HIVE-27560: [2.3] Enhancing compatibility with Guava

[LuciferYang]

What changes were proposed in this pull request?

This PR made the following changes to allow Hive 2.3 to use the more compatible Guava API:

1. Copied com.google.common.base.Objects from Guava 14.0.1 version to org.apache.hive.common.guava.Objects and changed Hive to use org.apache.hive.common.guava.Objects. This change is to accommodate versions of Guava after 21, as Objects was renamed to MoreObjects after Guava 21.

2. Copied com.google.common.base.Stopwatch from Guava 14.0.1 version to org.apache.hive.common.guava.Stopwatch, and changed Hive to use org.apache.hive.common.guava.Stopwatch. This change was made to accommodate versions of Guava after 17, where Stopwatch underwent significant changes. Maintaining the use of a fixed API is simpler than using reflection for compatibility.

3. Copied org.spark_project.guava.util.concurrent.MoreExecutors.SameThreadExecutorService and its corresponding utility method from Guava 14.0.1 version to org.apache.hive.common.guava.SameThreadExecutorUtil, and changed Hive to use SameThreadExecutorUtil#sameThreadExecutor. This change was made to accommodate versions of Guava after 26, which no longer contains this API.

4. Used reflection in the TephraHBaseConnection#connect method to start the TransactionManager. This is to accommodate versions of Guava after 17. Prior to Guava 16, startAndWait was called via reflection, while from Guava 17 onwards startAsync and awaitRunning are called through reflection.

5. In order to ensure compatibility with Guava 21+, a test method was added for each implementation of the com.google.common.base.Predicate.

6. In order to ensure compatibility with Guava 26+, the Futures.addCallback method with an Executor parameter is uniformly used in Hive 2.3. The SameThreadExecutorUtil.sameThreadExecutor()is passed in according to the implementation of Guava 14.0.1.

7. In order to ensure compatibility with Guava 20+, Collections.emptyIterator() was used instead of Iterators.emptyIterator().

8. In order to ensure compatibility with Guava 16+, the Hasher#putString method with a Charset parameter is uniformly used.

9. The relocation behavior of Guava in the hive-exec module has been removed, this operation should no longer be necessary.

After this PR, Hive 2.3 uses APIs that exist from Guava 14.0.1 to 32.1.2-jre, which also makes it possible for downstream projects that depend on Hive to upgrade their own Guava dependencies.

Why are the changes needed?

Make downstream projects that rely on Hive to possibly upgrade the Guava version they depend on.

Does this PR introduce any user-facing change?

No

Is the change a dependency upgrade?

No

Need to add additional error_prone_annotations 2.0.12 dependency.

How was this patch tested?

• Compatibility tests were carried out on Hive 2.3 with multiple versions of Guava in the local environment. All the following tests build successfully.

mvn clean install -DskipTests -Pitests -Dguava.version=14.0.1
mvn clean install -DskipTests -Pitests -Dguava.version=15.0
mvn clean install -DskipTests -Pitests -Dguava.version=16.0.1
mvn clean install -DskipTests -Pitests -Dguava.version=17.0
mvn clean install -DskipTests -Pitests -Dguava.version=18.0
mvn clean install -DskipTests -Pitests -Dguava.version=19.0
mvn clean install -DskipTests -Pitests -Dguava.version=20.0 (Need to add additional error_prone_annotations 2.0.12 dependency to project)
mvn clean install -DskipTests -Pitests -Dguava.version=21.0 (Need to add additional error_prone_annotations 2.0.15 dependency to project)
mvn clean install -DskipTests -Pitests -Dguava.version=22.0
mvn clean install -DskipTests -Pitests -Dguava.version=23.6.1-jre
mvn clean install -DskipTests -Pitests -Dguava.version=24.1.1-jre
mvn clean install -DskipTests -Pitests -Dguava.version=25.1-jre
mvn clean install -DskipTests -Pitests -Dguava.version=26.0-jre
mvn clean install -DskipTests -Pitests -Dguava.version=27.1-jre
mvn clean install -DskipTests -Pitests -Dguava.version=28.2-jre
mvn clean install -DskipTests -Pitests -Dguava.version=29.0-jre
mvn clean install -DskipTests -Pitests -Dguava.version=30.1.1-jre
mvn clean install -DskipTests -Pitests -Dguava.version=31.1-jre
mvn clean install -DskipTests -Pitests -Dguava.version=32.1.2-jre

[LuciferYang]

cc @sunchao @pan3793 @wangyum

[wangyum]

cc @viirya

[pan3793]

This patch makes sense to me.

I believe it could help to unblock Spark upgrading Guava.

I tested it with the Spark master branch with Guava 32.0.1-jre [1], everything looks well on my internal YARN 3.3 cluster.

Note that, there are a few UTs failed because of IsolatedClientLoader[2], it's another story.

[1] https://github.com/pan3793/spark/tree/guava
[2] https://www.mail-archive.com/dev@spark.apache.org/msg30708.html

[pan3793]

@sunchao are we good to go? I think apache/spark#42493 at least proves this PR makes Hive 2.3.10 compatible with Guava 32+

[sunchao]

LGTM

[sunchao]

Merged to branch-2.3, thanks @LuciferYang @pan3793 !

[LuciferYang]

Thanks @sunchao @pan3793