YARN-11469. [BackPort] Refactor FederationStateStoreFacade Cache Code. (#5570)

[slfan1989]

Description of PR

We expect to upgrade ehcache from 3.3.1 to 3.8.2 in branch-3.3, but before upgrading this jar package version, we need to backport 2 pr.

• YARN-11469. Refactor FederationStateStoreFacade Cache Code. (YARN-11469. Refactor FederationStateStoreFacade Cache Code. #5570)
• YARN-11470. FederationStateStoreFacade Cache Support Guava Cache. (YARN-11470. FederationStateStoreFacade Cache Support Guava Cache. #5609)

How was this patch tested?

Add Junit Test.

For code changes:

• Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
• Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
• If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
• If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

[hadoop-yetus]

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	6m 46s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ branch-3.3 Compile Tests _
+1 💚	mvninstall	46m 23s		branch-3.3 passed
+1 💚	compile	0m 39s		branch-3.3 passed
+1 💚	checkstyle	0m 31s		branch-3.3 passed
+1 💚	mvnsite	0m 44s		branch-3.3 passed
+1 💚	javadoc	0m 39s		branch-3.3 passed
+1 💚	spotbugs	1m 26s		branch-3.3 passed
+1 💚	shadedclient	38m 12s		branch has no errors when building and testing our client artifacts.
-0 ⚠️	patch	38m 33s		Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 35s		the patch passed
+1 💚	compile	0m 31s		the patch passed
+1 💚	javac	0m 31s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 17s		hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common: The patch generated 0 new + 0 unchanged - 11 fixed = 0 total (was 11)
+1 💚	mvnsite	0m 33s		the patch passed
+1 💚	javadoc	0m 26s		hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common generated 0 new + 44 unchanged - 1 fixed = 44 total (was 45)
+1 💚	spotbugs	1m 21s		the patch passed
+1 💚	shadedclient	36m 9s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	2m 55s	/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt	hadoop-yarn-server-common in the patch passed.
+1 💚	asflicense	0m 35s		The patch does not generate ASF License warnings.
		142m 21s

Reason	Tests
Failed junit tests	hadoop.yarn.server.federation.policies.amrmproxy.TestLocalityMulticastAMRMProxyPolicy

Subsystem	Report/Notes
Docker	ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6796/1/artifact/out/Dockerfile
GITHUB PR	#6796
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 7c05cf879860 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	branch-3.3 / edccf15
Default Java	Private Build-1.8.0_362-8u372-gaus1-0ubuntu118.04-b09
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6796/1/testReport/
Max. process+thread count	557 (vs. ulimit of 5500)
modules	C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6796/1/console
versions	git=2.17.1 maven=3.6.0 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

[hadoop-yetus]

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 29s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ branch-3.3 Compile Tests _
+1 💚	mvninstall	47m 49s		branch-3.3 passed
+1 💚	compile	0m 37s		branch-3.3 passed
+1 💚	checkstyle	0m 29s		branch-3.3 passed
+1 💚	mvnsite	0m 42s		branch-3.3 passed
+1 💚	javadoc	0m 38s		branch-3.3 passed
+1 💚	spotbugs	1m 22s		branch-3.3 passed
+1 💚	shadedclient	36m 32s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 35s		the patch passed
+1 💚	compile	0m 31s		the patch passed
+1 💚	javac	0m 31s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 18s		hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common: The patch generated 0 new + 0 unchanged - 11 fixed = 0 total (was 11)
+1 💚	mvnsite	0m 34s		the patch passed
+1 💚	javadoc	0m 25s		hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common generated 0 new + 44 unchanged - 1 fixed = 44 total (was 45)
+1 💚	spotbugs	1m 21s		the patch passed
+1 💚	shadedclient	35m 20s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	2m 57s	/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt	hadoop-yarn-server-common in the patch passed.
+1 💚	asflicense	0m 38s		The patch does not generate ASF License warnings.
		134m 58s

Reason	Tests
Failed junit tests	hadoop.yarn.server.federation.policies.amrmproxy.TestLocalityMulticastAMRMProxyPolicy

Subsystem	Report/Notes
Docker	ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6796/2/artifact/out/Dockerfile
GITHUB PR	#6796
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 77569b6f3797 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	branch-3.3 / f417397
Default Java	Private Build-1.8.0_362-8u372-gaus1-0ubuntu118.04-b09
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6796/2/testReport/
Max. process+thread count	747 (vs. ulimit of 5500)
modules	C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6796/2/console
versions	git=2.17.1 maven=3.6.0 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

[hadoop-yetus]

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	6m 17s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ branch-3.3 Compile Tests _
+1 💚	mvninstall	46m 52s		branch-3.3 passed
+1 💚	compile	0m 39s		branch-3.3 passed
+1 💚	checkstyle	0m 31s		branch-3.3 passed
+1 💚	mvnsite	0m 45s		branch-3.3 passed
+1 💚	javadoc	0m 40s		branch-3.3 passed
+1 💚	spotbugs	1m 26s		branch-3.3 passed
+1 💚	shadedclient	35m 38s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 36s		the patch passed
+1 💚	compile	0m 31s		the patch passed
+1 💚	javac	0m 31s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 18s		hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common: The patch generated 0 new + 0 unchanged - 11 fixed = 0 total (was 11)
+1 💚	mvnsite	0m 34s		the patch passed
+1 💚	javadoc	0m 27s		hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common generated 0 new + 44 unchanged - 1 fixed = 44 total (was 45)
+1 💚	spotbugs	1m 22s		the patch passed
+1 💚	shadedclient	34m 59s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	3m 6s	/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt	hadoop-yarn-server-common in the patch passed.
+1 💚	asflicense	0m 37s		The patch does not generate ASF License warnings.
		139m 0s

Reason	Tests
Failed junit tests	hadoop.yarn.server.federation.policies.amrmproxy.TestLocalityMulticastAMRMProxyPolicy

Subsystem	Report/Notes
Docker	ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6796/3/artifact/out/Dockerfile
GITHUB PR	#6796
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 30922528e317 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	branch-3.3 / 8019977
Default Java	Private Build-1.8.0_362-8u372-gaus1-0ubuntu118.04-b09
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6796/3/testReport/
Max. process+thread count	552 (vs. ulimit of 5500)
modules	C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6796/3/console
versions	git=2.17.1 maven=3.6.0 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

[hadoop-yetus]

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	6m 25s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ branch-3.3 Compile Tests _
+1 💚	mvninstall	53m 43s		branch-3.3 passed
+1 💚	compile	0m 39s		branch-3.3 passed
+1 💚	checkstyle	0m 29s		branch-3.3 passed
+1 💚	mvnsite	0m 44s		branch-3.3 passed
+1 💚	javadoc	0m 40s		branch-3.3 passed
+1 💚	spotbugs	1m 25s		branch-3.3 passed
+1 💚	shadedclient	35m 24s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 35s		the patch passed
+1 💚	compile	0m 31s		the patch passed
+1 💚	javac	0m 31s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 18s		hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common: The patch generated 0 new + 0 unchanged - 11 fixed = 0 total (was 11)
+1 💚	mvnsite	0m 34s		the patch passed
+1 💚	javadoc	0m 26s		hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common generated 0 new + 44 unchanged - 1 fixed = 44 total (was 45)
+1 💚	spotbugs	1m 22s		the patch passed
+1 💚	shadedclient	34m 49s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	3m 12s		hadoop-yarn-server-common in the patch passed.
+1 💚	asflicense	0m 37s		The patch does not generate ASF License warnings.
		145m 34s

Subsystem	Report/Notes
Docker	ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6796/4/artifact/out/Dockerfile
GITHUB PR	#6796
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 945bbd036316 5.15.0-106-generic #116-Ubuntu SMP Wed Apr 17 09:17:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	branch-3.3 / e3cfb1a
Default Java	Private Build-1.8.0_362-8u372-gaus1-0ubuntu118.04-b09
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6796/4/testReport/
Max. process+thread count	552 (vs. ulimit of 5500)
modules	C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6796/4/console
versions	git=2.17.1 maven=3.6.0 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

[slfan1989]

@steveloughran @goiri We are backporting YARN-11469 to branch-3.3 in order to upgrade ehcache from 3.3.1 to 3.8.2. However, there are significant differences between the code in branch-3.3 and branch-3.4. I had to redevelop YARN-11469 based on the code in branch-3.3, which has resulted in some variances between YARN-11469 in branch-3.3 and branch-3.4, but they are generally consistent.

[steveloughran]

I don't write/review yarn code, or hdfs. better for everyone. fewer regressions and I can stay out of support escalations. sorry

[slfan1989]

I don't write/review yarn code, or hdfs. better for everyone. fewer regressions and I can stay out of support escalations. sorry

Thank you for feedback! I understand and respect your principles.