Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BEAM-14053] [CdapIO] Add wrapper class for CDAP plugin #17150

Merged
merged 58 commits into from
May 24, 2022

Conversation

ktttnv
Copy link
Contributor

@ktttnv ktttnv commented Mar 22, 2022

Warning: This PR contains changes from PR "[BEAM-14048] [CdapIO] Add ConfigWrapper for building CDAP PluginConfigs" and should be merged after it

Changes:

  • Added wrapper classes for source and sink CDAP plugins
  • Added builder for plugin wrapper class
  • Added possibility to provide Hadoop configuration

pluginWrapper3


Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

  • Choose reviewer(s) and mention them in a comment (R: @username).
  • Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
  • Update CHANGES.md with noteworthy changes.
  • If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

Build python source distribution and wheels
Python tests
Java tests

See CI.md for more information about GitHub Actions CI.

@codecov
Copy link

codecov bot commented Mar 22, 2022

Codecov Report

Merging #17150 (94eca21) into master (a167424) will increase coverage by 0.16%.
The diff coverage is n/a.

❗ Current head 94eca21 differs from pull request most recent head 2766dca. Consider uploading reports for the commit 2766dca to get more accurate results

@@            Coverage Diff             @@
##           master   #17150      +/-   ##
==========================================
+ Coverage   73.96%   74.12%   +0.16%     
==========================================
  Files         693      680      -13     
  Lines       91721    89653    -2068     
==========================================
- Hits        67838    66454    -1384     
+ Misses      22634    22096     -538     
+ Partials     1249     1103     -146     
Flag Coverage Δ
python 83.71% <ø> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
sdks/go/pkg/beam/core/util/reflectx/types.go 0.00% <0.00%> (-94.60%) ⬇️
sdks/go/pkg/beam/core/util/reflectx/call.go 0.00% <0.00%> (-89.75%) ⬇️
sdks/go/pkg/beam/core/graph/coder/panes.go 0.00% <0.00%> (-81.25%) ⬇️
sdks/go/pkg/beam/core/util/reflectx/structs.go 0.00% <0.00%> (-76.93%) ⬇️
sdks/go/pkg/beam/core/runtime/graphx/cogbk.go 0.00% <0.00%> (-72.73%) ⬇️
sdks/go/pkg/beam/core/runtime/exec/discard.go 0.00% <0.00%> (-71.43%) ⬇️
sdks/go/pkg/beam/io/datastoreio/datastore.go 16.03% <0.00%> (-51.78%) ⬇️
sdks/go/pkg/beam/core/util/reflectx/functions.go 54.54% <0.00%> (-27.28%) ⬇️
sdks/go/pkg/beam/provision/provision.go 27.50% <0.00%> (-20.00%) ⬇️
sdks/go/pkg/beam/core/runtime/pipelinex/clone.go 21.42% <0.00%> (-18.58%) ⬇️
... and 69 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a167424...2766dca. Read the comment docs.

@chamikaramj
Copy link
Contributor

Retest this please

@elizaveta-lomteva
Copy link
Contributor

Run Java PreCommit

2 similar comments
@Amar3tto
Copy link
Contributor

Run Java PreCommit

@elizaveta-lomteva
Copy link
Contributor

Run Java PreCommit

@elizaveta-lomteva
Copy link
Contributor

@chamikaramj this PR is ready for review

Comment on lines 192 to 193
for (Annotation annotation : getPluginClass().getDeclaredAnnotations()) {
if (annotation.annotationType().equals(io.cdap.cdap.api.annotation.Plugin.class)) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a nitpick, but there is a method getDeclaredAnnotationsByType that should work as a shortcut.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, that's helpful. Done

@elizaveta-lomteva
Copy link
Contributor

Run Java PreCommit

Copy link
Contributor

@chamikaramj chamikaramj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. LGTM.

@chamikaramj
Copy link
Contributor

Run Java PreCommit

7 similar comments
@Amar3tto
Copy link
Contributor

Run Java PreCommit

@elizaveta-lomteva
Copy link
Contributor

Run Java PreCommit

@elizaveta-lomteva
Copy link
Contributor

Run Java PreCommit

@elizaveta-lomteva
Copy link
Contributor

Run Java PreCommit

@elizaveta-lomteva
Copy link
Contributor

Run Java PreCommit

@elizaveta-lomteva
Copy link
Contributor

Run Java PreCommit

@chamikaramj
Copy link
Contributor

Run Java PreCommit

@elizaveta-lomteva
Copy link
Contributor

@chamikaramj PR is ready to be merged

@chamikaramj chamikaramj merged commit 7547cb0 into apache:master May 24, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants