HDFS-2139. Fast copy for HDFS.FastCopy (This is used for test case . And in the futuer this will split to subtask submission. ) #6950

LiuGuH · 2024-07-19T09:26:04Z

Description of PR

https://issues.apache.org/jira/browse/HDFS-2139

This is used for compile to verify this is correct. It is for test case . And in the futuer this will split to subtask submission.

It support now:
(1) hdfs dfs -fastcp
(2) Distcp with fastcp

How was this patch tested?

Add test case

hadoop-yetus · 2024-07-19T18:05:01Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 29s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+0 🆗	buf	0m 1s		buf was not available.
+0 🆗	buf	0m 1s		buf was not available.
+0 🆗	xmllint	0m 1s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 11 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	14m 27s		Maven dependency ordering for branch
+1 💚	mvninstall	32m 9s		trunk passed
+1 💚	compile	16m 45s		trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 💚	compile	15m 21s		trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 💚	checkstyle	4m 25s		trunk passed
+1 💚	mvnsite	5m 4s		trunk passed
+1 💚	javadoc	3m 54s		trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 💚	javadoc	4m 19s		trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 💚	spotbugs	9m 29s		trunk passed
+1 💚	shadedclient	33m 36s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 31s		Maven dependency ordering for patch
+1 💚	mvninstall	3m 21s		the patch passed
+1 💚	compile	16m 23s		the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 💚	cc	16m 23s		the patch passed
+1 💚	javac	16m 23s		the patch passed
+1 💚	compile	15m 37s		the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 💚	cc	15m 37s		the patch passed
+1 💚	javac	15m 37s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	4m 14s	/results-checkstyle-root.txt	root: The patch generated 47 new + 1030 unchanged - 6 fixed = 1077 total (was 1036)
+1 💚	mvnsite	4m 59s		the patch passed
+1 💚	javadoc	3m 56s		the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 💚	javadoc	4m 17s		the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 💚	spotbugs	10m 23s		the patch passed
+1 💚	shadedclient	33m 47s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	19m 21s	/patch-unit-hadoop-common-project_hadoop-common.txt	hadoop-common in the patch passed.
+1 💚	unit	2m 46s		hadoop-hdfs-client in the patch passed.
-1 ❌	unit	222m 25s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	unit	29m 47s		hadoop-distcp in the patch passed.
+1 💚	asflicense	1m 8s		The patch does not generate ASF License warnings.
		517m 44s

Reason	Tests
Failed junit tests	hadoop.fs.TestHarFileSystem
	hadoop.fs.TestFilterFileSystem
	hadoop.hdfs.server.namenode.TestFSNamesystemLockReport

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6950/1/artifact/out/Dockerfile
GITHUB PR	#6950
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets cc buflint bufcompat xmllint
uname	Linux 65eed2f25d58 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `6cbf7c9`
Default Java	Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6950/1/testReport/
Max. process+thread count	3470 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-tools/hadoop-distcp U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6950/1/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

zeekling

We already has hadoop distcp, Why do we need hdfs dfs -distcp?

zeekling · 2024-07-20T16:20:59Z

hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java

  }

+  public void fastCopy(Path src, Path dst, boolean overwrite) throws IOException {
+


Why empty method ?

hadoop-yetus · 2024-07-22T16:59:32Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	11m 44s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	buf	0m 0s		buf was not available.
+0 🆗	buf	0m 0s		buf was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 13 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	14m 36s		Maven dependency ordering for branch
+1 💚	mvninstall	33m 8s		trunk passed
+1 💚	compile	16m 55s		trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 💚	compile	15m 34s		trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 💚	checkstyle	4m 22s		trunk passed
+1 💚	mvnsite	5m 9s		trunk passed
+1 💚	javadoc	4m 0s		trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 💚	javadoc	4m 18s		trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 💚	spotbugs	9m 36s		trunk passed
+1 💚	shadedclient	33m 32s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 32s		Maven dependency ordering for patch
+1 💚	mvninstall	3m 13s		the patch passed
+1 💚	compile	16m 26s		the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 💚	cc	16m 26s		the patch passed
+1 💚	javac	16m 26s		the patch passed
+1 💚	compile	15m 41s		the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 💚	cc	15m 41s		the patch passed
+1 💚	javac	15m 41s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	4m 18s	/results-checkstyle-root.txt	root: The patch generated 49 new + 1205 unchanged - 6 fixed = 1254 total (was 1211)
+1 💚	mvnsite	5m 6s		the patch passed
+1 💚	javadoc	3m 59s		the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 💚	javadoc	4m 20s		the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 💚	spotbugs	10m 23s		the patch passed
+1 💚	shadedclient	34m 15s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	19m 23s		hadoop-common in the patch passed.
+1 💚	unit	2m 43s		hadoop-hdfs-client in the patch passed.
-1 ❌	unit	220m 27s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	unit	29m 43s		hadoop-distcp in the patch passed.
+1 💚	asflicense	1m 7s		The patch does not generate ASF License warnings.
		529m 15s

Reason	Tests
Failed junit tests	hadoop.hdfs.server.namenode.TestFSNamesystemLockReport

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6950/2/artifact/out/Dockerfile
GITHUB PR	#6950
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets cc buflint bufcompat xmllint
uname	Linux a24adcfdc185 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `3bad2c1`
Default Java	Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6950/2/testReport/
Max. process+thread count	3264 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-tools/hadoop-distcp U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6950/2/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2024-07-23T11:21:34Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 30s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	buf	0m 0s		buf was not available.
+0 🆗	buf	0m 0s		buf was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 13 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	14m 53s		Maven dependency ordering for branch
+1 💚	mvninstall	32m 32s		trunk passed
+1 💚	compile	17m 35s		trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 💚	compile	16m 5s		trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 💚	checkstyle	4m 34s		trunk passed
+1 💚	mvnsite	5m 10s		trunk passed
+1 💚	javadoc	3m 54s		trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 💚	javadoc	4m 18s		trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 💚	spotbugs	9m 44s		trunk passed
+1 💚	shadedclient	36m 20s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 31s		Maven dependency ordering for patch
+1 💚	mvninstall	3m 22s		the patch passed
+1 💚	compile	17m 4s		the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 💚	cc	17m 4s		the patch passed
+1 💚	javac	17m 4s		the patch passed
+1 💚	compile	15m 56s		the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 💚	cc	15m 56s		the patch passed
+1 💚	javac	15m 56s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	4m 39s	/results-checkstyle-root.txt	root: The patch generated 49 new + 1205 unchanged - 6 fixed = 1254 total (was 1211)
+1 💚	mvnsite	5m 1s		the patch passed
+1 💚	javadoc	3m 51s		the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 💚	javadoc	4m 14s		the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 💚	spotbugs	10m 21s		the patch passed
+1 💚	shadedclient	35m 8s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	19m 20s		hadoop-common in the patch passed.
+1 💚	unit	2m 42s		hadoop-hdfs-client in the patch passed.
-1 ❌	unit	217m 35s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	unit	29m 58s		hadoop-distcp in the patch passed.
+1 💚	asflicense	1m 2s		The patch does not generate ASF License warnings.
		521m 21s

Reason	Tests
Failed junit tests	hadoop.hdfs.server.namenode.TestFSNamesystemLockReport

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6950/3/artifact/out/Dockerfile
GITHUB PR	#6950
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets cc buflint bufcompat xmllint
uname	Linux 6918e2d2e0f0 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `db19346`
Default Java	Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6950/3/testReport/
Max. process+thread count	3889 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-tools/hadoop-distcp U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6950/3/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

…out federation.

hadoop-yetus · 2024-07-26T18:07:44Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 30s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	buf	0m 0s		buf was not available.
+0 🆗	buf	0m 0s		buf was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 13 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	14m 28s		Maven dependency ordering for branch
-1 ❌	mvninstall	32m 54s	/branch-mvninstall-root.txt	root in trunk failed.
+1 💚	compile	17m 13s		trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 💚	compile	18m 11s		trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 💚	checkstyle	4m 46s		trunk passed
+1 💚	mvnsite	5m 3s		trunk passed
+1 💚	javadoc	3m 35s		trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 💚	javadoc	3m 50s		trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 💚	spotbugs	8m 53s		trunk passed
+1 💚	shadedclient	36m 31s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 29s		Maven dependency ordering for patch
+1 💚	mvninstall	3m 30s		the patch passed
+1 💚	compile	17m 15s		the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 💚	cc	17m 15s		the patch passed
+1 💚	javac	17m 15s		the patch passed
+1 💚	compile	15m 31s		the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 💚	cc	15m 31s		the patch passed
+1 💚	javac	15m 31s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	4m 27s	/results-checkstyle-root.txt	root: The patch generated 49 new + 1205 unchanged - 6 fixed = 1254 total (was 1211)
+1 💚	mvnsite	5m 10s		the patch passed
+1 💚	javadoc	3m 46s		the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 💚	javadoc	4m 17s		the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 💚	spotbugs	11m 3s		the patch passed
+1 💚	shadedclient	40m 41s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	19m 53s		hadoop-common in the patch passed.
+1 💚	unit	2m 47s		hadoop-hdfs-client in the patch passed.
+1 💚	unit	222m 28s		hadoop-hdfs in the patch passed.
+1 💚	unit	30m 38s		hadoop-distcp in the patch passed.
+1 💚	asflicense	1m 8s		The patch does not generate ASF License warnings.
		533m 39s

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6950/4/artifact/out/Dockerfile
GITHUB PR	#6950
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets cc buflint bufcompat xmllint
uname	Linux eb29cb108240 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `205d342`
Default Java	Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6950/4/testReport/
Max. process+thread count	3245 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-tools/hadoop-distcp U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6950/4/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2024-08-09T19:36:29Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	12m 6s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	buf	0m 0s		buf was not available.
+0 🆗	buf	0m 0s		buf was not available.
+0 🆗	xmllint	0m 1s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 13 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	14m 36s		Maven dependency ordering for branch
+1 💚	mvninstall	32m 3s		trunk passed
+1 💚	compile	16m 46s		trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04
+1 💚	compile	15m 38s		trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05
+1 💚	checkstyle	4m 18s		trunk passed
+1 💚	mvnsite	5m 8s		trunk passed
+1 💚	javadoc	4m 3s		trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04
+1 💚	javadoc	4m 23s		trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05
+1 💚	spotbugs	9m 35s		trunk passed
+1 💚	shadedclient	34m 3s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 31s		Maven dependency ordering for patch
+1 💚	mvninstall	3m 18s		the patch passed
+1 💚	compile	16m 20s		the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04
+1 💚	cc	16m 20s		the patch passed
+1 💚	javac	16m 20s		the patch passed
+1 💚	compile	15m 42s		the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05
+1 💚	cc	15m 42s		the patch passed
+1 💚	javac	15m 42s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	4m 15s	/results-checkstyle-root.txt	root: The patch generated 53 new + 1205 unchanged - 6 fixed = 1258 total (was 1211)
+1 💚	mvnsite	5m 7s		the patch passed
+1 💚	javadoc	3m 57s		the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04
+1 💚	javadoc	4m 26s		the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05
+1 💚	spotbugs	10m 18s		the patch passed
+1 💚	shadedclient	34m 7s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	19m 23s		hadoop-common in the patch passed.
+1 💚	unit	2m 41s		hadoop-hdfs-client in the patch passed.
+1 💚	unit	218m 2s		hadoop-hdfs in the patch passed.
+1 💚	unit	29m 16s		hadoop-distcp in the patch passed.
+1 💚	asflicense	1m 8s		The patch does not generate ASF License warnings.
		525m 59s

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6950/5/artifact/out/Dockerfile
GITHUB PR	#6950
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets cc buflint bufcompat xmllint
uname	Linux fe34b1067a1f 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `e5022fe`
Default Java	Private Build-1.8.0_422-8u422-b05-1~20.04-b05
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6950/5/testReport/
Max. process+thread count	3814 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-tools/hadoop-distcp U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6950/5/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

tomscut · 2024-12-26T05:59:33Z

...-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java

-        Token<BlockTokenIdentifier> accessToken = getBlockAccessToken(b,
-            EnumSet.of(BlockTokenIdentifier.AccessMode.WRITE),
-            targetStorageTypes, targetStorageIds);
+        Token<BlockTokenIdentifier> accessToken =


There's a lot of code formatting here.

github-actions · 2025-09-22T00:24:13Z

We're closing this stale PR because it has been open for 100 days with no activity. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you feel like this was a mistake, or you would like to continue working on it, please feel free to re-open it and ask for a committer to remove the stale tag and review again.
Thanks all for your contribution.

github-actions bot added HDFS DistCp trunk Common TOOLS labels Jul 19, 2024

LiuGuH changed the title ~~FastCopy~~ HDFS-2139. Fast copy for HDFS.FastCopy (This is used for test case . And in the futuer this will split to subtask submission. ) Jul 19, 2024

zeekling reviewed Jul 21, 2024

View reviewed changes

LiuGuH force-pushed the trunk-fastcopy branch from 6cbf7c9 to 3bad2c1 Compare July 22, 2024 08:09

LiuGuH added 3 commits July 26, 2024 17:08

HDFS-16757. Add a new method copyBlockCrossNamespace to DataNode

3e2e2c3

HDFS-17581. Add FastCopy tool and support dfs -fastcp command

3af0f21

HDFS-17582. Distcp support fastcopy

205d342

LiuGuH force-pushed the trunk-fastcopy branch from db19346 to 205d342 Compare July 26, 2024 09:12

HDFS-17592. FastCopy support data copy in different nameservices with…

e5022fe

…out federation.

tomscut reviewed Dec 26, 2024

View reviewed changes

github-actions bot added the Stale label Sep 22, 2025

github-actions bot closed this Sep 23, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

HDFS-2139. Fast copy for HDFS.FastCopy (This is used for test case . And in the futuer this will split to subtask submission. ) #6950

HDFS-2139. Fast copy for HDFS.FastCopy (This is used for test case . And in the futuer this will split to subtask submission. ) #6950

Uh oh!

LiuGuH commented Jul 19, 2024

Uh oh!

hadoop-yetus commented Jul 19, 2024

Uh oh!

zeekling left a comment

Uh oh!

zeekling Jul 20, 2024

Uh oh!

hadoop-yetus commented Jul 22, 2024

Uh oh!

hadoop-yetus commented Jul 23, 2024

Uh oh!

hadoop-yetus commented Jul 26, 2024

Uh oh!

hadoop-yetus commented Aug 9, 2024

Uh oh!

tomscut Dec 26, 2024

Uh oh!

github-actions bot commented Sep 22, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

		}

		public void fastCopy(Path src, Path dst, boolean overwrite) throws IOException {

HDFS-2139. Fast copy for HDFS.FastCopy (This is used for test case . And in the futuer this will split to subtask submission. ) #6950

HDFS-2139. Fast copy for HDFS.FastCopy (This is used for test case . And in the futuer this will split to subtask submission. ) #6950

Uh oh!

Conversation

LiuGuH commented Jul 19, 2024

Description of PR

How was this patch tested?

Uh oh!

hadoop-yetus commented Jul 19, 2024

Uh oh!

zeekling left a comment

Choose a reason for hiding this comment

Uh oh!

zeekling Jul 20, 2024

Choose a reason for hiding this comment

Uh oh!

hadoop-yetus commented Jul 22, 2024

Uh oh!

hadoop-yetus commented Jul 23, 2024

Uh oh!

hadoop-yetus commented Jul 26, 2024

Uh oh!

hadoop-yetus commented Aug 9, 2024

Uh oh!

tomscut Dec 26, 2024

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Sep 22, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants