HBASE-29465 [hbase-thirdparty] Update yetus to a newer version #144

NihalJain · 2025-07-25T09:40:33Z

Upgrades yetus to 0.15.0, making required changes in sync with hbase main repo
Improves error handling and logging of wrapper maven script
Unrelated changes: Fix typos and git urls

NihalJain · 2025-07-25T10:02:05Z

Based off apache/hbase@17ce7c3 and apache/hbase@4c83128

Apache-HBase · 2025-07-25T10:49:32Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 24s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	shelldocs	0m 0s		Shelldocs was not available.
+0 🆗	hadolint	0m 0s		hadolint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
			_ master Compile Tests _
+0 🆗	mvndep	0m 3s		Maven dependency ordering for branch
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 3s		Maven dependency ordering for patch
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	shellcheck	0m 1s		No new issues.
			_ Other Tests _
+0 🆗	asflicense	0m 0s		ASF License check generated no output?
		0m 36s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-Thirdparty-PreCommit/job/PR-144/6/artifact/yetus-precommit-check/output/Dockerfile
GITHUB PR	#144
Optional Tests	dupname asflicense codespell detsecrets shellcheck shelldocs hadolint
uname	Linux fb1f6db9ecde 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	/home/jenkins/jenkins-home/workspace/Base-Thirdparty-PreCommit_PR-144/yetus-precommit-check/src/.yetus/personality.sh
git revision	master / `ad8071a`
Max. process+thread count	36 (vs. ulimit of 2000)
modules	C: U:
Console output	https://ci-hbase.apache.org/job/HBase-Thirdparty-PreCommit/job/PR-144/6/console
versions	git=2.34.1 maven=/home/jenkins/jenkins-home/workspace/Base-Thirdparty-PreCommit_PR-144/yetus-precommit-check/src/dev-support/toolchains-jenkins.xml shellcheck=0.8.0
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

NihalJain · 2025-07-25T10:54:54Z

Ok so now the code is building but the tests run are not as expected! Do we need to provide test filter with new yetus.?

NihalJain · 2025-07-25T17:35:56Z

Ah now I understand! It was working good all this while, since we did not have any change in code hence build did not run javac, etc. Introduced some changes in pom.xml to trigger those.

Apache-HBase · 2025-07-25T17:38:07Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 35s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	shelldocs	0m 0s		Shelldocs was not available.
+0 🆗	hadolint	0m 0s		hadolint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-0 ⚠️	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ master Compile Tests _
-1 ❌	mvninstall	0m 17s	/branch-mvninstall-root.txt	root in master failed.
-1 ❌	compile	0m 18s	/branch-compile-root.txt	root in master failed.
-1 ❌	javadoc	0m 17s	/branch-javadoc-root.txt	root in master failed.
			_ Patch Compile Tests _
-1 ❌	mvninstall	0m 17s	/patch-mvninstall-root.txt	root in the patch failed.
-1 ❌	compile	0m 17s	/patch-compile-root.txt	root in the patch failed.
-1 ❌	javac	0m 17s	/patch-compile-root.txt	root in the patch failed.
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	shellcheck	0m 0s		No new issues.
+1 💚	xmllint	0m 0s		No new issues.
-1 ❌	javadoc	0m 17s	/patch-javadoc-root.txt	root in the patch failed.
			_ Other Tests _
-1 ❌	unit	0m 18s	/patch-unit-root.txt	root in the patch failed.
+0 🆗	asflicense	0m 17s		ASF License check generated no output?
		3m 17s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-Thirdparty-PreCommit/job/PR-144/9/artifact/yetus-precommit-check/output/Dockerfile
GITHUB PR	#144
Optional Tests	dupname asflicense codespell detsecrets shellcheck shelldocs hadolint javac javadoc unit xmllint compile
uname	Linux 09c4e26e54af 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	/home/jenkins/jenkins-home/workspace/Base-Thirdparty-PreCommit_PR-144/yetus-precommit-check/src/.yetus/personality.sh
git revision	master / `2b32064`
Default Java	Eclipse Adoptium-17.0.11+9
Test Results	https://ci-hbase.apache.org/job/HBase-Thirdparty-PreCommit/job/PR-144/9/testReport/
Max. process+thread count	9 (vs. ulimit of 2000)
modules	C: . U: .
Console output	https://ci-hbase.apache.org/job/HBase-Thirdparty-PreCommit/job/PR-144/9/console
versions	git=2.34.1 maven=/home/jenkins/jenkins-home/workspace/Base-Thirdparty-PreCommit_PR-144/yetus-precommit-check/src/dev-support/toolchains-jenkins.xml shellcheck=0.8.0 xmllint=20913
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

NihalJain · 2025-07-25T17:41:03Z

xmllint is fixed but other maven stuff failing now 🤦

Apache-HBase · 2025-07-25T18:40:06Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 26s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	shelldocs	0m 0s		Shelldocs was not available.
+0 🆗	hadolint	0m 0s		hadolint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-0 ⚠️	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ master Compile Tests _
+1 💚	mvninstall	0m 53s		master passed
+1 💚	compile	0m 16s		master passed
+1 💚	javadoc	0m 7s		master passed
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 40s		the patch passed
+1 💚	compile	0m 18s		the patch passed
+1 💚	javac	0m 18s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	shellcheck	0m 0s		No new issues.
+1 💚	xmllint	0m 0s		No new issues.
+1 💚	javadoc	0m 6s		the patch passed
			_ Other Tests _
+1 💚	unit	0m 48s		root in the patch passed.
+1 💚	asflicense	0m 7s		The patch does not generate ASF License warnings.
		3m 50s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-Thirdparty-PreCommit/job/PR-144/14/artifact/yetus-precommit-check/output/Dockerfile
GITHUB PR	#144
Optional Tests	dupname asflicense codespell detsecrets shellcheck shelldocs hadolint javac javadoc unit xmllint compile
uname	Linux a124cb79bd45 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	/home/jenkins/jenkins-home/workspace/Base-Thirdparty-PreCommit_PR-144/yetus-precommit-check/src/.yetus/personality.sh
git revision	master / `e8bb9ac`
Default Java	Eclipse Adoptium-17.0.11+9
Test Results	https://ci-hbase.apache.org/job/HBase-Thirdparty-PreCommit/job/PR-144/14/testReport/
Max. process+thread count	418 (vs. ulimit of 2000)
modules	C: . U: .
Console output	https://ci-hbase.apache.org/job/HBase-Thirdparty-PreCommit/job/PR-144/14/console
versions	git=2.34.1 maven=called shellcheck=0.8.0 xmllint=20913
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

NihalJain · 2025-07-25T18:58:08Z

dev-support/jenkins/Dockerfile

+# Try multiple possible locations for the toolchains file
+TOOLCHAIN_LOCATIONS=(
+  "${BASEDIR}/dev-support/toolchains-jenkins.xml"
+  "$(pwd)/dev-support/toolchains-jenkins.xml"


may be we do not need to try for different dirs as it should found in 1st dir itself? but why not, not a problem too? WDYT?

dev-support/jenkins/Dockerfile

NihalJain · 2025-07-25T18:58:50Z

dev-support/jenkins/Dockerfile

-TOOLCHAIN="${BASEDIR}/dev-support/toolchains-jenkins.xml"
-if [ -f "$TOOLCHAIN" ]; then
+
+echo "Maven wrapper called with args: $@"


I think its good to keep the change as this improves logging and error handling. without this the out files were empty and swallowing errors making difficult to debug

NihalJain · 2025-07-25T19:01:06Z

Now the issue is fixed, problem was usage of exec and no error handling, hence logs were empty. Now we have better error handling and proper logging in wrapper code.

Also added a dummy commit in each module to trigger all possible build combinations. Will revert before commit!

NihalJain · 2025-07-25T19:09:48Z

dev-support/jenkins/Dockerfile

+  done
+  echo "Proceeding without toolchains configuration..."
+  echo "Executing: ${MAVEN_HOME}/bin/mvn-original $@"
+  "${MAVEN_HOME}/bin/mvn-original" "$@"


this fallback at least builds code, should not happen again though, but maybe with some yetus upgrade or dockerfile change

Apache-HBase · 2025-07-25T19:18:07Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 26s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	shelldocs	0m 0s		Shelldocs was not available.
+0 🆗	hadolint	0m 0s		hadolint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-0 ⚠️	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ master Compile Tests _
+0 🆗	mvndep	0m 8s		Maven dependency ordering for branch
+1 💚	mvninstall	0m 40s		master passed
+1 💚	compile	1m 27s		master passed
+1 💚	javadoc	1m 6s		master passed
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 5s		Maven dependency ordering for patch
+1 💚	mvninstall	2m 33s		the patch passed
+1 💚	compile	1m 26s		the patch passed
+1 💚	javac	1m 26s		the patch passed
-1 ❌	blanks	0m 0s	/blanks-eol.txt	The patch has 13 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
+1 💚	shellcheck	0m 0s		No new issues.
+1 💚	xmllint	0m 0s		No new issues.
+1 💚	javadoc	1m 6s		the patch passed
			_ Other Tests _
+1 💚	unit	0m 11s		hbase-shaded-protobuf in the patch passed.
+1 💚	unit	0m 6s		hbase-shaded-netty in the patch passed.
+1 💚	unit	0m 5s		hbase-shaded-netty-tcnative in the patch passed.
+1 💚	unit	0m 5s		hbase-shaded-gson in the patch passed.
+1 💚	unit	0m 5s		hbase-shaded-miscellaneous in the patch passed.
+1 💚	unit	0m 5s		hbase-shaded-jetty in the patch passed.
+1 💚	unit	0m 5s		hbase-shaded-jetty-12-plus-core in the patch passed.
+1 💚	unit	0m 5s		hbase-shaded-jetty-12-plus-ee8 in the patch passed.
+1 💚	unit	0m 5s		hbase-shaded-jersey in the patch passed.
+1 💚	unit	0m 5s		hbase-shaded-jackson-jaxrs-json-provider in the patch passed.
+1 💚	unit	0m 37s		hbase-noop-htrace in the patch passed.
+1 💚	unit	0m 6s		hbase-unsafe in the patch passed.
+1 💚	unit	0m 45s		root in the patch passed.
+1 💚	asflicense	1m 2s		The patch does not generate ASF License warnings.
		13m 20s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-Thirdparty-PreCommit/job/PR-144/15/artifact/yetus-precommit-check/output/Dockerfile
GITHUB PR	#144
Optional Tests	dupname asflicense codespell detsecrets shellcheck shelldocs hadolint javac javadoc unit xmllint compile
uname	Linux a620d3af4693 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	/home/jenkins/jenkins-home/workspace/Base-Thirdparty-PreCommit_PR-144/yetus-precommit-check/src/.yetus/personality.sh
git revision	master / `083b0fd`
Default Java	Eclipse Adoptium-17.0.11+9
Test Results	https://ci-hbase.apache.org/job/HBase-Thirdparty-PreCommit/job/PR-144/15/testReport/
Max. process+thread count	419 (vs. ulimit of 2000)
modules	C: hbase-shaded-protobuf hbase-shaded-netty hbase-shaded-netty-tcnative hbase-shaded-gson hbase-shaded-miscellaneous hbase-shaded-jetty hbase-shaded-jetty-12-plus-core hbase-shaded-jetty-12-plus-ee8 hbase-shaded-jersey hbase-shaded-jackson-jaxrs-json-provider hbase-noop-htrace hbase-unsafe . U: .
Console output	https://ci-hbase.apache.org/job/HBase-Thirdparty-PreCommit/job/PR-144/15/console
versions	git=2.34.1 maven=called shellcheck=0.8.0 xmllint=20913
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

NihalJain · 2025-07-25T19:21:59Z

ok looks good, blanks is due to the dummy patch

NihalJain · 2025-07-25T19:33:19Z

reverted dummy patch, fixed some typos, changed message to error from warning

Apache-HBase · 2025-07-25T19:34:43Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hbase.apache.org/job/HBase-Thirdparty-PreCommit/job/PR-144/16/console in case of problems.

Apache-HBase · 2025-07-25T19:40:32Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 31s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+0 🆗	shelldocs	0m 1s		Shelldocs was not available.
+0 🆗	hadolint	0m 1s		hadolint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-0 ⚠️	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ master Compile Tests _
+0 🆗	mvndep	0m 5s		Maven dependency ordering for branch
+1 💚	mvninstall	0m 40s		master passed
+1 💚	compile	0m 33s		master passed
+1 💚	javadoc	0m 21s		master passed
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 5s		Maven dependency ordering for patch
+1 💚	mvninstall	1m 5s		the patch passed
+1 💚	compile	0m 32s		the patch passed
+1 💚	javac	0m 32s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	shellcheck	0m 0s		No new issues.
+1 💚	xmllint	0m 1s		No new issues.
+1 💚	javadoc	0m 21s		the patch passed
			_ Other Tests _
+1 💚	unit	0m 5s		hbase-shaded-jetty in the patch passed.
+1 💚	unit	0m 5s		hbase-shaded-jetty-12-plus-core in the patch passed.
+1 💚	unit	0m 5s		hbase-shaded-jetty-12-plus-ee8 in the patch passed.
+1 💚	unit	0m 43s		root in the patch passed.
+1 💚	asflicense	0m 19s		The patch does not generate ASF License warnings.
		5m 52s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-Thirdparty-PreCommit/job/PR-144/16/artifact/yetus-precommit-check/output/Dockerfile
GITHUB PR	#144
Optional Tests	dupname asflicense codespell detsecrets shellcheck shelldocs hadolint javac javadoc unit xmllint compile
uname	Linux 29bef013284f 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	/home/jenkins/jenkins-home/workspace/Base-Thirdparty-PreCommit_PR-144/yetus-precommit-check/src/.yetus/personality.sh
git revision	master / `95550b6`
Default Java	Eclipse Adoptium-17.0.11+9
Test Results	https://ci-hbase.apache.org/job/HBase-Thirdparty-PreCommit/job/PR-144/16/testReport/
Max. process+thread count	397 (vs. ulimit of 2000)
modules	C: hbase-shaded-jetty hbase-shaded-jetty-12-plus-core hbase-shaded-jetty-12-plus-ee8 . U: .
Console output	https://ci-hbase.apache.org/job/HBase-Thirdparty-PreCommit/job/PR-144/16/console
versions	git=2.34.1 maven=called shellcheck=0.8.0 xmllint=20913
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

HBASE-29465 [hbase-thirdparty] Update yetus for hbase-thirdparty

29f841f

NihalJain changed the title ~~HBASE-29465 [hbase-thirdparty] Update yetus for hbase-thirdparty~~ HBASE-29465 [hbase-thirdparty] Update yetus to a newer version Jul 25, 2025

NihalJain mentioned this pull request Jul 25, 2025

[DO NOT MERGE] PR just to test build #143

Closed

Copy cache-apache-project-artifact.sh from hbase and sync download logic

40f421d

NihalJain marked this pull request as draft July 25, 2025 10:14

NihalJain added 2 commits July 25, 2025 15:49

Drop as we already have the cache-apache-project-artifact.sh

e45edf1

Download as "noop htrace drop in" doesnot happen first

ad8071a