Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CdapIO] HasOffset interface was implemented #22193

Merged
merged 3 commits into from
Jul 21, 2022

Conversation

sorokin-andrey
Copy link
Contributor

In SparkReceiverIO we will have different functions for working with Spark receivers that support offsets and those that don't. To determine if a receiver supports offsets, we need the HasOffset interface.

The goal of this PR is to implement it plus add some minor infrastructure needed changes.

Fixes #22148

@codecov
Copy link

codecov bot commented Jul 8, 2022

Codecov Report

Merging #22193 (98ef2c9) into master (4b5efc3) will increase coverage by 0.02%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master   #22193      +/-   ##
==========================================
+ Coverage   74.16%   74.18%   +0.02%     
==========================================
  Files         706      703       -3     
  Lines       93195    93084     -111     
==========================================
- Hits        69122    69059      -63     
+ Misses      22805    22758      -47     
+ Partials     1268     1267       -1     
Flag Coverage Δ
python 83.53% <ø> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
sdks/python/apache_beam/utils/interactive_utils.py 95.12% <0.00%> (-2.44%) ⬇️
...python/apache_beam/runners/worker/worker_status.py 77.53% <0.00%> (-2.18%) ⬇️
...ks/python/apache_beam/runners/worker/sdk_worker.py 88.46% <0.00%> (-0.48%) ⬇️
...eam/runners/interactive/interactive_environment.py 91.71% <0.00%> (-0.31%) ⬇️
...hon/apache_beam/runners/worker/bundle_processor.py 93.42% <0.00%> (ø)
...ache_beam/coders/proto2_coder_test_messages_pb2.py 100.00% <0.00%> (ø)
sdks/go/pkg/beam/io/fhirio/deidentify.go
.../pkg/beam/core/runtime/harness/diagnostics_hook.go
sdks/go/pkg/beam/util/harnessopts/heap_dump.go
sdks/go/pkg/beam/core/runtime/harness/harness.go 10.24% <0.00%> (+0.06%) ⬆️
... and 2 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4b5efc3...98ef2c9. Read the comment docs.

@elizaveta-lomteva
Copy link
Contributor

Run Java PreCommit

@elizaveta-lomteva
Copy link
Contributor

Run Python PreCommit

@sorokin-andrey
Copy link
Contributor Author

Run Java PreCommit

@Amar3tto
Copy link
Contributor

Run Python PreCommit

# Conflicts:
#	sdks/java/io/sparkreceiver/build.gradle
@sorokin-andrey
Copy link
Contributor Author

Run Python PreCommit

@chamikaramj
Copy link
Contributor

Thanks. LGTM.

@chamikaramj chamikaramj merged commit caaefc0 into apache:master Jul 21, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[CdapIO][Task]: Implement HasOffset interface
4 participants