HADOOP-17194. Adding Context class for AbfsClient in ABFS #2216

mehakmeet · 2020-08-11T12:42:10Z

Tested on: mvn -T 1C -Dparallel-tests=abfs clean verify
Region: East US

[INFO] Results:
[INFO]
[INFO] Tests run: 87, Failures: 0, Errors: 0, Skipped: 0

[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR]   ITestAzureBlobFileSystemCheckAccess.<init>:82->setTestUserFs:104 » IllegalArgument
[ERROR]   ITestAzureBlobFileSystemCheckAccess.<init>:82->setTestUserFs:104 » IllegalArgument
[ERROR]   ITestAzureBlobFileSystemCheckAccess.<init>:82->setTestUserFs:104 » IllegalArgument
[ERROR]   ITestAzureBlobFileSystemCheckAccess.<init>:82->setTestUserFs:104 » IllegalArgument
[ERROR]   ITestAzureBlobFileSystemCheckAccess.<init>:82->setTestUserFs:104 » IllegalArgument
[ERROR]   ITestAzureBlobFileSystemCheckAccess.<init>:82->setTestUserFs:104 » IllegalArgument
[ERROR]   ITestAzureBlobFileSystemCheckAccess.<init>:82->setTestUserFs:104 » IllegalArgument
[ERROR]   ITestAzureBlobFileSystemCheckAccess.<init>:82->setTestUserFs:104 » IllegalArgument
[ERROR]   ITestAzureBlobFileSystemCheckAccess.<init>:82->setTestUserFs:104 » IllegalArgument
[ERROR]   ITestAzureBlobFileSystemCheckAccess.<init>:82->setTestUserFs:104 » IllegalArgument
[ERROR]   ITestAzureBlobFileSystemCheckAccess.<init>:82->setTestUserFs:104 » IllegalArgument
[ERROR]   ITestAzureBlobFileSystemCheckAccess.<init>:82->setTestUserFs:104 » IllegalArgument
[INFO]
[ERROR] Tests run: 451, Failures: 0, Errors: 12, Skipped: 61

[INFO] Results:
[INFO]
[WARNING] Tests run: 206, Failures: 0, Errors: 0, Skipped: 29

The errors are being discussed in HADOOP-17203(https://issues.apache.org/jira/browse/HADOOP-17203).

…rations to limit number of parameters

hadoop-yetus · 2020-08-11T16:52:21Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 5s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s	The patch appears to include 1 new or modified test files.
		_ trunk Compile Tests _
+1 💚	mvninstall	31m 10s	trunk passed
+1 💚	compile	0m 36s	trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
+1 💚	compile	0m 29s	trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
+1 💚	checkstyle	0m 22s	trunk passed
+1 💚	mvnsite	0m 32s	trunk passed
+1 💚	shadedclient	16m 36s	branch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 26s	trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
+1 💚	javadoc	0m 24s	trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
+0 🆗	spotbugs	0m 56s	Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚	findbugs	0m 54s	trunk passed
		_ Patch Compile Tests _
+1 💚	mvninstall	0m 27s	the patch passed
+1 💚	compile	0m 28s	the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
+1 💚	javac	0m 28s	the patch passed
+1 💚	compile	0m 24s	the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
+1 💚	javac	0m 24s	the patch passed
+1 💚	checkstyle	0m 15s	the patch passed
+1 💚	mvnsite	0m 27s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	shadedclient	15m 29s	patch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 22s	the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
+1 💚	javadoc	0m 19s	the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
+1 💚	findbugs	0m 57s	the patch passed
		_ Other Tests _
+1 💚	unit	1m 19s	hadoop-azure in the patch passed.
+1 💚	asflicense	0m 28s	The patch does not generate ASF License warnings.
		75m 31s

Subsystem	Report/Notes
Docker	ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2216/1/artifact/out/Dockerfile
GITHUB PR	#2216
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname	Linux 500ec22de4cc 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `6c2ce3d`
Default Java	Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2216/1/testReport/
Max. process+thread count	308 (vs. ulimit of 5500)
modules	C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2216/1/console
versions	git=2.17.1 maven=3.6.0 findbugs=4.0.6
Powered by	Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

mukund-thakur · 2020-08-12T07:12:01Z

Overall looks good to me.

steveloughran

patch looks OK, just move to a builder + immutable context

...ols/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClientContext.java

...tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AzureBlobFileSystemStore.java

hadoop-yetus · 2020-08-24T13:13:58Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 30s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 1s	No case conflicting files found.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s	The patch appears to include 1 new or modified test files.
		_ trunk Compile Tests _
+1 💚	mvninstall	28m 46s	trunk passed
+1 💚	compile	0m 37s	trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
+1 💚	compile	0m 34s	trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
+1 💚	checkstyle	0m 27s	trunk passed
+1 💚	mvnsite	0m 40s	trunk passed
+1 💚	shadedclient	14m 58s	branch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 31s	trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
+1 💚	javadoc	0m 28s	trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
+0 🆗	spotbugs	0m 58s	Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚	findbugs	0m 57s	trunk passed
		_ Patch Compile Tests _
+1 💚	mvninstall	0m 30s	the patch passed
+1 💚	compile	0m 29s	the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
+1 💚	javac	0m 29s	the patch passed
+1 💚	compile	0m 25s	the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
+1 💚	javac	0m 25s	the patch passed
+1 💚	checkstyle	0m 18s	the patch passed
+1 💚	mvnsite	0m 28s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	shadedclient	13m 50s	patch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 25s	the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
+1 💚	javadoc	0m 24s	the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
+1 💚	findbugs	0m 57s	the patch passed
		_ Other Tests _
+1 💚	unit	1m 20s	hadoop-azure in the patch passed.
+1 💚	asflicense	0m 34s	The patch does not generate ASF License warnings.
		70m 4s

Subsystem	Report/Notes
Docker	ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2216/2/artifact/out/Dockerfile
GITHUB PR	#2216
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname	Linux eb7b47b2765e 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `17cd8a1`
Default Java	Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2216/2/testReport/
Max. process+thread count	415 (vs. ulimit of 5500)
modules	C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2216/2/console
versions	git=2.17.1 maven=3.6.0 findbugs=4.0.6
Powered by	Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

...ols/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClientContext.java

steveloughran · 2020-08-26T16:25:00Z

ok, patch LGTM. Just make that context constructor private so everyone MUST use the builder. thanks

hadoop-yetus · 2020-08-27T03:00:47Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 47s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s	The patch appears to include 1 new or modified test files.
		_ trunk Compile Tests _
+1 💚	mvninstall	30m 12s	trunk passed
+1 💚	compile	0m 40s	trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
+1 💚	compile	0m 34s	trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
+1 💚	checkstyle	0m 26s	trunk passed
+1 💚	mvnsite	0m 39s	trunk passed
+1 💚	shadedclient	14m 40s	branch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 31s	trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
+1 💚	javadoc	0m 29s	trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
+0 🆗	spotbugs	0m 59s	Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚	findbugs	0m 56s	trunk passed
		_ Patch Compile Tests _
+1 💚	mvninstall	0m 30s	the patch passed
+1 💚	compile	0m 29s	the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
+1 💚	javac	0m 29s	the patch passed
+1 💚	compile	0m 24s	the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
+1 💚	javac	0m 24s	the patch passed
+1 💚	checkstyle	0m 16s	the patch passed
+1 💚	mvnsite	0m 28s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	shadedclient	13m 51s	patch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 25s	the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
+1 💚	javadoc	0m 23s	the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
+1 💚	findbugs	0m 58s	the patch passed
		_ Other Tests _
+1 💚	unit	1m 29s	hadoop-azure in the patch passed.
+1 💚	asflicense	0m 32s	The patch does not generate ASF License warnings.
		71m 36s

Subsystem	Report/Notes
Docker	ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2216/3/artifact/out/Dockerfile
GITHUB PR	#2216
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname	Linux 54451f27d972 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `5e52955`
Default Java	Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2216/3/testReport/
Max. process+thread count	460 (vs. ulimit of 5500)
modules	C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2216/3/console
versions	git=2.17.1 maven=3.6.0 findbugs=4.0.6
Powered by	Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

steveloughran · 2020-08-27T10:09:27Z

+1, merging

Contributed by Mehakmeet Singh. Change-Id: I120c9a068d758d8e5d071c878a3b7fbeb95e4de6

Contributed by Mehakmeet Singh.

HADOOP-17194. Adding Context class for AbfsClient to pass AbfsConfigu…

4bafb6a

…rations to limit number of parameters

steveloughran requested changes Aug 22, 2020

View reviewed changes

HADOOP-17194. Splitting the builder and context class.

13f9cee

mehakmeet requested a review from steveloughran August 24, 2020 12:04

steveloughran requested changes Aug 26, 2020

View reviewed changes

...ols/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClientContext.java Outdated Show resolved Hide resolved

...ols/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClientContext.java Show resolved Hide resolved

HADOOP-17194. making AbfsClientContext constructor package private

07bf092

mehakmeet requested a review from steveloughran August 27, 2020 01:48

steveloughran merged commit d1c60a5 into apache:trunk Aug 27, 2020

asfgit pushed a commit that referenced this pull request Aug 27, 2020

HADOOP-17194. Adding Context class for AbfsClient in ABFS (#2216)

f6e1ed4

Contributed by Mehakmeet Singh. Change-Id: I120c9a068d758d8e5d071c878a3b7fbeb95e4de6

snvijaya pushed a commit to snvijaya/hadoop that referenced this pull request May 23, 2021

HADOOP-17194. Adding Context class for AbfsClient in ABFS (apache#2216)

c252365

Contributed by Mehakmeet Singh.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HADOOP-17194. Adding Context class for AbfsClient in ABFS #2216

HADOOP-17194. Adding Context class for AbfsClient in ABFS #2216

mehakmeet commented Aug 11, 2020

hadoop-yetus commented Aug 11, 2020

mukund-thakur commented Aug 12, 2020

steveloughran left a comment

hadoop-yetus commented Aug 24, 2020

steveloughran commented Aug 26, 2020

hadoop-yetus commented Aug 27, 2020

steveloughran commented Aug 27, 2020

HADOOP-17194. Adding Context class for AbfsClient in ABFS #2216

HADOOP-17194. Adding Context class for AbfsClient in ABFS #2216

Conversation

mehakmeet commented Aug 11, 2020

hadoop-yetus commented Aug 11, 2020

mukund-thakur commented Aug 12, 2020

steveloughran left a comment

Choose a reason for hiding this comment

hadoop-yetus commented Aug 24, 2020

steveloughran commented Aug 26, 2020

hadoop-yetus commented Aug 27, 2020

steveloughran commented Aug 27, 2020