Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SNOW-843760] Update pom.xml corresponding to Wiz vulnerability scan #546

Merged
merged 12 commits into from
Jul 14, 2023

Conversation

sfc-gh-rcheng
Copy link
Collaborator

@sfc-gh-rcheng sfc-gh-rcheng commented Jul 10, 2023

Changes, full scan in slack (ping revi for link)

net.minidev:json-smart: 2.4.7 -> 2.4.9
com.nimbusds.nimbus-jose-jwt references json-smart
note: can't find reference to this anymore, even using lukas's process

com.google.protobuf:protobuf-java: 3.7.1 -> 3.16.3 and com.google.guava:guava: 30.1.1-jre -> 32.0.0
org.apache.hadoop/hadoop-common (exclude io.dropwizard.metrics.metrics-core to converge dependencies)
org.apache.hadoop.thirdparty/hadoop-shaded-guava-1.1.1: 3.71
org.apache.hadoop.thirdparty/hadoop-shaded-protobuf_3_7-1.1.1.jar: 3.7.1

com.fasterxml.jackson.core:jackson-databind
2.13.2.2 -> 2.13.4.2
org.apache.parquet.parquet-hadoop references jackson-databind
org.apache.commons/commons-configuration2: 2.13.3
org.apache.hadoop/hadoop-common
org.apache.parquet/parquet-hadoop
parquet-jackson-1.13.1: 2.13.4.2
org.apache.parquet/parquet-jackson

@sfc-gh-rcheng sfc-gh-rcheng marked this pull request as ready for review July 10, 2023 19:26
@sfc-gh-rcheng sfc-gh-rcheng requested review from a team and sfc-gh-tzhang as code owners July 10, 2023 19:26
@sfc-gh-rcheng sfc-gh-rcheng changed the title Update pom.xml corresponding to Wiz vulnerability scan [SNOW-843760] Update pom.xml corresponding to Wiz vulnerability scan Jul 10, 2023
@codecov
Copy link

codecov bot commented Jul 10, 2023

Codecov Report

Merging #546 (22a5dcd) into master (2999390) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #546   +/-   ##
=======================================
  Coverage   78.30%   78.30%           
=======================================
  Files          76       76           
  Lines        4734     4734           
  Branches      424      424           
=======================================
  Hits         3707     3707           
  Misses        846      846           
  Partials      181      181           

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

pom.xml Outdated Show resolved Hide resolved
Copy link
Contributor

@sfc-gh-tzhang sfc-gh-tzhang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Revi! Do you think we should provide a private jar to customers and ask them to run it with Blackduck in case there is still some differences? My concern is that we may need another round of SDK/KC release which is time consuming.

pom.xml Outdated Show resolved Hide resolved
@sfc-gh-rcheng
Copy link
Collaborator Author

sfc-gh-rcheng commented Jul 10, 2023

Thanks Revi! Do you think we should provide a private jar to customers and ask them to run it with Blackduck in case there is still some differences? My concern is that we may need another round of SDK/KC release which is time consuming.

Agreed - discussed offline, lets ask them to run a blackduck scan for us. Created jira https://snowflakecomputing.atlassian.net/browse/SNOW-860497

Copy link
Collaborator

@sfc-gh-japatel sfc-gh-japatel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, thanks!

@sfc-gh-rcheng
Copy link
Collaborator Author

Following error, so i think we need to retain the shaded hadoop jar. I'll add it back in, it was originally removed because I thought it caused the protobuf vulnerability. @sfc-gh-lsembera any thoughts?

The valid symbol is in org.apache.hadoop.thirdparty:hadoop-shaded-protobuf_3_7:jar:1.1.1 at net.snowflake:snowflake-ingest-sdk:jar:2.0.2-SNAPSHOT / org.apache.hadoop:hadoop-common:3.3.5 (compile) / org.apache.hadoop.thirdparty:hadoop-shaded-protobuf_3_7:1.1.1 (compile) but it was not selected because net.snowflake:snowflake-ingest-sdk:2.0.2-SNAPSHOT excludes org.apache.hadoop.thirdparty:hadoop-shaded-protobuf_3_7.

@sfc-gh-lsembera
Copy link
Contributor

@sfc-gh-rcheng Putting versions into <dependency> does not have any effect because versions of these dependencies are already defined in <dependencyManagement> section and dependencyManagement is the source of truth for dependency versions. You can verify it by running mvn dependency:list with and without your change and see there are no differences (output is generated in target/dependency-list.txt.

The Hadoop errors seem to be caused by the linkage checker, you can fix it by adding a rule to linkage-checker-exclusion-rules.xml.

@sfc-gh-rcheng
Copy link
Collaborator Author

Thanks Lukas, youre right adding the version makes no change on the dependency list. Closing this PR as the versions are high enough to resolve the vulnerabilities

@sfc-gh-rcheng
Copy link
Collaborator Author

sfc-gh-rcheng commented Jul 11, 2023

Discussing on slack - how to find shaded dependencies

@sfc-gh-rcheng sfc-gh-rcheng merged commit 2af18ec into master Jul 14, 2023
11 checks passed
@sfc-gh-rcheng sfc-gh-rcheng deleted the rcheng-wizfix branch July 14, 2023 22:57
sfc-gh-tzhang pushed a commit that referenced this pull request Jul 20, 2023
…546)

* update pom for vulns

* update pom

* use fasterxml version

* remove hadoop exclusion

* mvn install passes

* passes

* autoformatting

* remove dependency pom

* relocate parquet-hadoop

* dont relocate

* relocate just airlift not parquet-hadoop
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants