[WIP][SPARK-36994][BUILD] Update Thrift to 0.15.0

[HyukjinKwon]

What changes were proposed in this pull request?

This PR proposes to upgrade Thrift to 0.15.0.

Why are the changes needed?

To addresses CVEs:

Component Name	Component Version Name	Vulnerability	Fixed version
Apache Thrift	0.11.0-4.	CVE-2019-0205	0.13.0
Apache Thrift	0.11.0-4.	CVE-2019-0210	0.13.0
Apache Thrift	0.11.0-4.	CVE-2020-13949	0.14.1

Does this PR introduce any user-facing change?

I think no. CVE-2019-0205 and CVE-2019-0210 are more a bug fix.
CVE-2020-13949 might be valid.

How was this patch tested?

Existing tests should cover.

[SparkQA]

Test build #144228 has finished for PR 34280 at commit 01d95b2.

• This patch fails to build.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Kubernetes integration test unable to build dist.

exiting with code: 1
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48708/

[dongjoon-hyun]

It seems to fail at some code. Is this a breaking change?

[error] /home/runner/work/spark/spark/sql/hive-thriftserver/src/main/java/org/apache/hive/service/auth/HiveAuthFactory.java:254:1:  error: incompatible types: String cannot be converted to TConfiguration
[error]     return new TSocket(host, port, loginTimeout);

cc @wangyum and @sunchao

[sunchao]

seems it's changed by this commit in 0.15.0

[SparkQA]

Test build #144347 has finished for PR 34280 at commit de3f92f.

• This patch fails to build.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Kubernetes integration test unable to build dist.

exiting with code: 1
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48825/

[HyukjinKwon]

@wangyum, I just noticed that we have some diff in Thriftserver. Just to confirm, have you made some modifications to match with Spark's Thriftversion before?

e.g.) https://github.com/apache/spark/blob/master/sql/hive-thriftserver/src/main/java/org/apache/hive/service/auth/HiveAuthFactory.java#L241-L283 vs https://github.com/apache/hive/blob/rel/release-2.3.7/service/src/java/org/apache/hive/service/auth/HiveAuthFactory.java#L240-L254

[wangyum]

It is copied from Spark 2.4.0, Not Hive 2.3.7.
Hive 1.2.1: https://github.com/apache/hive/blob/release-1.2.1/service/src/java/org/apache/hive/service/auth/HiveAuthFactory.java#L202-L244
Spark 2.4.0: https://github.com/apache/spark/blob/v2.4.0/sql/hive-thriftserver/src/main/java/org/apache/hive/service/auth/HiveAuthFactory.java#L237-L278

[SparkQA]

Test build #144351 has finished for PR 34280 at commit a796548.

• This patch fails to build.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Kubernetes integration test unable to build dist.

exiting with code: 1
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48829/

[wangyum]

@HyukjinKwon It seems we need to port https://issues.apache.org/jira/browse/HIVE-21498:

[HyukjinKwon]

Thanks @wangyum. Yeah, probably should better do that. Would you min taking a look when you find some time? I can give a shot too but that would take a while 😢

[wangyum]

It seems it only support Thrift 0.13: https://github.com/apache/spark/compare/master...wangyum:thrift-0.13?expand=1
Thrift 0.14.2 will throw exception:

[wangyum]

We can upgrade to Thrift 0.16 with these changes: https://github.com/apache/spark/compare/master...wangyum:thrift-0.15?expand=1

[HyukjinKwon]

Thanks @wangyum. feel free to take this over!

[HyukjinKwon]

@wangyum feel free to create a PR!