Skip to content

Conversation

@wangyum
Copy link
Member

@wangyum wangyum commented Aug 8, 2023

What changes were proposed in this pull request?

This is the third step to remove dependency on jackson-core-asl and jackson-mapper-asl.

Why are the changes needed?

To fix SPARK-44719: NoClassDefFoundError when using Hive UDF.

Does this PR introduce any user-facing change?

Is the change a dependency upgrade?

How was this patch tested?

@wangyum
Copy link
Member Author

wangyum commented Aug 8, 2023

cc @sunchao @pan3793

@pan3793
Copy link
Member

pan3793 commented Aug 8, 2023

Awesome! Thanks for your effort @wangyum

@wangyum
Copy link
Member Author

wangyum commented Aug 9, 2023

Add HIVE-22059 to fix:

mvn clean install -DskipTests=true
cd service && mvn test -DwildcardSuites=none -Dtest=org.apache.hive.service.cli.session.TestSessionHooks test

[INFO] Running org.apache.hive.service.cli.session.TestSessionHooks
[ERROR] Tests run: 2, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 3.236 s <<< FAILURE! - in org.apache.hive.service.cli.session.TestSessionHooks
[ERROR] org.apache.hive.service.cli.session.TestSessionHooks.testSessionHook  Time elapsed: 3.128 s  <<< ERROR!
java.lang.NoSuchFieldError: ALLOW_LEADING_DECIMAL_POINT_FOR_NUMBERS
        at com.fasterxml.jackson.core.json.JsonReadFeature.<clinit>(JsonReadFeature.java:121)
        at org.apache.hadoop.hive.ql.udf.UDFJson.<clinit>(UDFJson.java:69)
...

Copy link
Member

@sunchao sunchao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@sunchao sunchao merged commit d9e96c9 into apache:branch-2.3 Aug 30, 2023
@sunchao
Copy link
Member

sunchao commented Aug 30, 2023

Merged to branch-2.3, thanks @wangyum !

@wangyum wangyum deleted the HIVE-27580 branch August 30, 2023 23:36
charlso96 added a commit to charlso96/hive that referenced this pull request Sep 7, 2023
LuciferYang pushed a commit to apache/spark that referenced this pull request Jun 24, 2025
### What changes were proposed in this pull request?

CodeHaus Jackson dependencies were pulled from Hive, while in apache/hive#4564 (Hive 2.3.10), it migrated to Jackson 2.x, so we can remove them from Spark now.

### Why are the changes needed?

Remove unused and vulnerable dependencies.

### Does this PR introduce _any_ user-facing change?

No

### How was this patch tested?

Pass GA.

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes #46521 from pan3793/SPARK-48231.

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: yangjie01 <yangjie01@baidu.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants