HADOOP-17912. ABFS: Support for Encryption Context (#6221)

[saxenapranav]

This is backport of 0b43026 in branch-3.3. The commit is result of the merge of #6221.

JIRA: https://issues.apache.org/jira/browse/HADOOP-17912

[hadoop-yetus]

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	10m 58s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+0 🆗	markdownlint	0m 1s		markdownlint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 13 new or modified test files.
			_ branch-3.3 Compile Tests _
+1 💚	mvninstall	46m 12s		branch-3.3 passed
+1 💚	compile	0m 38s		branch-3.3 passed
+1 💚	checkstyle	0m 31s		branch-3.3 passed
+1 💚	mvnsite	0m 43s		branch-3.3 passed
+1 💚	javadoc	0m 35s		branch-3.3 passed
+1 💚	spotbugs	1m 8s		branch-3.3 passed
+1 💚	shadedclient	35m 7s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 38s		the patch passed
+1 💚	compile	0m 29s		the patch passed
+1 💚	javac	0m 29s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	0m 19s	/results-checkstyle-hadoop-tools_hadoop-azure.txt	hadoop-tools/hadoop-azure: The patch generated 11 new + 7 unchanged - 0 fixed = 18 total (was 7)
+1 💚	mvnsite	0m 33s		the patch passed
+1 💚	javadoc	0m 23s		the patch passed
+1 💚	spotbugs	1m 5s		the patch passed
+1 💚	shadedclient	35m 0s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	2m 9s		hadoop-azure in the patch passed.
+1 💚	asflicense	0m 35s		The patch does not generate ASF License warnings.
		140m 13s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6401/1/artifact/out/Dockerfile
GITHUB PR	#6401
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint
uname	Linux 74ab27fd079a 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	branch-3.3 / 5a0af85
Default Java	Private Build-1.8.0_362-8u372-gaus1-0ubuntu118.04-b09
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6401/1/testReport/
Max. process+thread count	557 (vs. ulimit of 5500)
modules	C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6401/1/console
versions	git=2.17.1 maven=3.6.0 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

[saxenapranav]

---

:::: AGGREGATED TEST RESULT ::::

HNS-OAuth

[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] TestAccountConfiguration.testConfigPropNotFound:386->testMissingConfigKey:399 Expected a org.apache.hadoop.fs.azurebfs.contracts.exceptions.TokenAccessProviderException to be thrown, but got the result: : "org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider"
[INFO]
[ERROR] Tests run: 141, Failures: 1, Errors: 0, Skipped: 5
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] ITestAzureBlobFileSystemRandomRead.testSkipBounds:218->Assert.assertTrue:42->Assert.fail:89 There should not be any network I/O (elapsedTimeMs=30).
[ERROR] Errors:
[ERROR] ITestAzureBlobFileSystemLease.testAcquireRetry:329 » TestTimedOut test timed o...
[INFO]
[ERROR] Tests run: 566, Failures: 1, Errors: 1, Skipped: 26
[INFO] Results:
[INFO]
[WARNING] Tests run: 339, Failures: 0, Errors: 0, Skipped: 41

HNS-SharedKey

[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] TestAccountConfiguration.testConfigPropNotFound:386->testMissingConfigKey:399 Expected a org.apache.hadoop.fs.azurebfs.contracts.exceptions.TokenAccessProviderException to be thrown, but got the result: : "org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider"
[ERROR] TestAbfsClientThrottlingAnalyzer.testManySuccessAndErrorsAndWaiting:181->fuzzyValidate:64 The actual value 34 is not within the expected range: [5.60, 8.40].
[INFO]
[ERROR] Tests run: 141, Failures: 2, Errors: 0, Skipped: 5
[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR] ITestAzureBlobFileSystemLease.testAcquireRetry:344->lambda$testAcquireRetry$6:345 » TestTimedOut
[INFO]
[ERROR] Tests run: 579, Failures: 0, Errors: 1, Skipped: 26
[INFO] Results:
[INFO]
[WARNING] Tests run: 339, Failures: 0, Errors: 0, Skipped: 41

NonHNS-SharedKey

[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] TestAccountConfiguration.testConfigPropNotFound:386->testMissingConfigKey:399 Expected a org.apache.hadoop.fs.azurebfs.contracts.exceptions.TokenAccessProviderException to be thrown, but got the result: : "org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider"
[INFO]
[ERROR] Tests run: 141, Failures: 1, Errors: 0, Skipped: 11
[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR] ITestAzureBlobFileSystemLease.testAcquireRetry:344->lambda$testAcquireRetry$6:345 » TestTimedOut
[INFO]
[ERROR] Tests run: 563, Failures: 0, Errors: 1, Skipped: 269
[INFO] Results:
[INFO]
[WARNING] Tests run: 339, Failures: 0, Errors: 0, Skipped: 44

AppendBlob-HNS-OAuth

[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] TestAccountConfiguration.testConfigPropNotFound:386->testMissingConfigKey:399 Expected a org.apache.hadoop.fs.azurebfs.contracts.exceptions.TokenAccessProviderException to be thrown, but got the result: : "org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider"
[INFO]
[ERROR] Tests run: 141, Failures: 1, Errors: 0, Skipped: 5
[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR] ITestAzureBlobFileSystemLease.testAcquireRetry:329 » TestTimedOut test timed o...
[ERROR] ITestAzureBlobFileSystemLease.testTwoWritersCreateAppendWithInfiniteLeaseEnabled:186->twoWriters:154 » TestTimedOut
[INFO]
[ERROR] Tests run: 560, Failures: 0, Errors: 2, Skipped: 26
[INFO] Results:
[INFO]
[WARNING] Tests run: 339, Failures: 0, Errors: 0, Skipped: 41

Time taken: 46 mins 48 secs.
azureuser@Hadoop-VM-EAST2:/hadoop/hadoop-tools/hadoop-azure$
azureuser@Hadoop-VM-EAST2:/hadoop/hadoop-tools/hadoop-azure$ git log
commit 5a0af85 (HEAD -> saxenapranav/HADOOP-17912-branch-3.3, origin/saxenapranav/HADOOP-17912-branch-3.3)
Author: Pranav Saxena 108325433+saxenapranav@users.noreply.github.com
Date: Mon Jan 1 11:09:44 2024 -0800

HADOOP-17912. ABFS: Support for Encryption Context (#6221)

Contributed by Pranav Saxena and others.

[saxenapranav]

@steveloughran , this is backport PR to branch-3.3 of #6221. Requesting your kind help in review and merge. Thank you so much.