HDFS-16274. Improve error msg for FSNamesystem#startFileInt #3554

tomscut · 2021-10-14T16:06:41Z

When the blocksize of a file is smaller than dfs.namenode.fs-limits.min-block-size, an IOE will be thrown. In current exception messages, it is easy to confuse the value of blocksize with the value of dfs.namenode.fs-limits.min-block-size.

Before the change:

After the change:

hadoop-yetus · 2021-10-14T21:32:08Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 44s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 1s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+1 💚	mvninstall	31m 29s		trunk passed
+1 💚	compile	1m 22s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	1m 16s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	checkstyle	1m 0s		trunk passed
+1 💚	mvnsite	1m 24s		trunk passed
+1 💚	javadoc	0m 58s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	1m 27s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	3m 7s		trunk passed
+1 💚	shadedclient	21m 12s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 10s		the patch passed
+1 💚	compile	1m 12s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javac	1m 12s		the patch passed
+1 💚	compile	1m 9s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	javac	1m 9s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 50s		the patch passed
+1 💚	mvnsite	1m 15s		the patch passed
+1 💚	javadoc	0m 46s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	1m 18s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	3m 5s		the patch passed
+1 💚	shadedclient	20m 54s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	229m 38s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 47s		The patch does not generate ASF License warnings.
		324m 12s

Reason	Tests
Failed junit tests	hadoop.hdfs.server.namenode.TestFileLimit

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3554/1/artifact/out/Dockerfile
GITHUB PR	#3554
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux 607a07c724c1 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19 23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `23fc0a0`
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3554/1/testReport/
Max. process+thread count	3436 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3554/1/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2021-10-15T06:41:51Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 40s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	31m 38s		trunk passed
+1 💚	compile	1m 23s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	1m 15s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	checkstyle	1m 2s		trunk passed
+1 💚	mvnsite	1m 23s		trunk passed
+1 💚	javadoc	0m 57s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	1m 27s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	3m 9s		trunk passed
+1 💚	shadedclient	21m 17s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 12s		the patch passed
+1 💚	compile	1m 12s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javac	1m 12s		the patch passed
+1 💚	compile	1m 6s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	javac	1m 6s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 51s		the patch passed
+1 💚	mvnsite	1m 12s		the patch passed
+1 💚	javadoc	0m 46s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	1m 20s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	3m 10s		the patch passed
+1 💚	shadedclient	21m 5s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	223m 56s		hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 47s		The patch does not generate ASF License warnings.
		318m 31s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3554/2/artifact/out/Dockerfile
GITHUB PR	#3554
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux 4edb2e14c654 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19 23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `9211428`
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3554/2/testReport/
Max. process+thread count	3271 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3554/2/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

tomscut · 2021-10-15T06:53:05Z

Hi @ferhui @ayushtkn @Hexiaoqiao @jojochuang @tasanuma , could you please review this PR. Thanks a lot.

aajisaka

LGTM.

tomscut · 2021-10-15T09:33:54Z

Thanks @aajisaka @tasanuma for your review.

tomscut · 2021-10-15T10:57:19Z

Thanks @ayushtkn for your review.

hemanthboyina · 2021-10-15T17:11:23Z

+1

virajjasani · 2021-10-15T19:43:30Z

Belated +1 (non-binding)

tomscut · 2021-10-15T20:25:09Z

Thanks @hemanthboyina @virajjasani for your review.

) * HDFS-16274. Improve log for FSNamesystem#startFileInt * fix failed unit test

HDFS-16274. Improve log for FSNamesystem#startFileInt

23fc0a0

tomscut changed the title ~~HDFS-16274. Improve log for FSNamesystem#startFileInt~~ HDFS-16274. Improve error msg for FSNamesystem#startFileInt Oct 15, 2021

fix failed unit test

9211428

aajisaka approved these changes Oct 15, 2021

View reviewed changes

tasanuma approved these changes Oct 15, 2021

View reviewed changes

ayushtkn approved these changes Oct 15, 2021

View reviewed changes

hemanthboyina merged commit d72e2cb into apache:trunk Oct 15, 2021

HarshitGupta11 pushed a commit to HarshitGupta11/hadoop that referenced this pull request Nov 28, 2022

HDFS-16274. Improve error msg for FSNamesystem#startFileInt (apache#3554

e5105fc

) * HDFS-16274. Improve log for FSNamesystem#startFileInt * fix failed unit test

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

HDFS-16274. Improve error msg for FSNamesystem#startFileInt #3554

HDFS-16274. Improve error msg for FSNamesystem#startFileInt #3554

Uh oh!

tomscut commented Oct 14, 2021

Uh oh!

hadoop-yetus commented Oct 14, 2021

Uh oh!

hadoop-yetus commented Oct 15, 2021

Uh oh!

tomscut commented Oct 15, 2021

Uh oh!

aajisaka left a comment

Uh oh!

tomscut commented Oct 15, 2021

Uh oh!

tomscut commented Oct 15, 2021 •

edited

Loading

Uh oh!

hemanthboyina commented Oct 15, 2021

Uh oh!

virajjasani commented Oct 15, 2021

Uh oh!

tomscut commented Oct 15, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

HDFS-16274. Improve error msg for FSNamesystem#startFileInt #3554

HDFS-16274. Improve error msg for FSNamesystem#startFileInt #3554

Uh oh!

Conversation

tomscut commented Oct 14, 2021

Uh oh!

hadoop-yetus commented Oct 14, 2021

Uh oh!

hadoop-yetus commented Oct 15, 2021

Uh oh!

tomscut commented Oct 15, 2021

Uh oh!

aajisaka left a comment

Choose a reason for hiding this comment

Uh oh!

tomscut commented Oct 15, 2021

Uh oh!

tomscut commented Oct 15, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hemanthboyina commented Oct 15, 2021

Uh oh!

virajjasani commented Oct 15, 2021

Uh oh!

tomscut commented Oct 15, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

tomscut commented Oct 15, 2021 •

edited

Loading