HADOOP-19466. Fix Dockerfile_windows_10 #7417

mgmgwi · 2025-02-20T14:20:57Z

Description of PR

Fix a few outdated dependencies.

How was this patch tested?

For code changes:

Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

hadoop-yetus · 2025-02-20T14:23:46Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7417/1/console in case of problems.

hadoop-yetus · 2025-02-20T15:40:43Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 34s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
			_ trunk Compile Tests _
+0 🆗	mvndep	6m 1s		Maven dependency ordering for branch
+1 💚	shadedclient	40m 52s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 32s		Maven dependency ordering for patch
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	shadedclient	34m 8s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	asflicense	0m 37s		The patch does not generate ASF License warnings.
		78m 24s

Subsystem	Report/Notes
Docker	ClientAPI=1.48 ServerAPI=1.48 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7417/1/artifact/out/Dockerfile
GITHUB PR	#7417
Optional Tests	dupname asflicense
uname	Linux 3963ac263972 5.15.0-131-generic #141-Ubuntu SMP Fri Jan 10 21:18:28 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `9a47c0d`
Max. process+thread count	545 (vs. ulimit of 5500)
modules	C: U:
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7417/1/console
versions	git=2.25.1 maven=3.6.3
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

slfan1989 · 2025-02-20T22:46:45Z

@GauthamBanasandra Could you please review this PR? Thank you very much!

GauthamBanasandra · 2025-02-21T00:39:06Z

@mgmgwi Thank you so much for the PR. I've started the CI run for Windows - https://ci-hadoop.apache.org/view/Hadoop/job/hadoop-qbt-trunk-java8-win10-x86_64/806/console.
We can merge this once the CI run is completed. FYI, it'll take about a day to finish.

mgmgwi · 2025-02-23T00:00:58Z

Hmm, that build runs now for 2 days already. @GauthamBanasandra can you check what's going on?

mgmgwi · 2025-02-24T08:53:09Z

Can someone give me a hint, what went wrong?

GauthamBanasandra · 2025-02-24T08:58:29Z

@mgmgwi I guess there was something wrong with the CI node thus it was stuck at the mvn install step.

I had thus started the CI run on a couple of other nodes and it has passed the required steps - https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-win10-x86_64/813/

GauthamBanasandra · 2025-02-24T08:59:27Z

Please see - https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-win10-x86_64/813/artifact/out/report.html

GauthamBanasandra · 2025-02-24T09:05:43Z

The ones in green are expected to pass and it's looking fine.

@mgmgwi could you please address the minor comment that I've added? We can then proceed with the merge.

GauthamBanasandra · 2025-02-21T00:36:12Z

dev-support/docker/Dockerfile_windows_10

    && git checkout 7ffa425e1db8b0c3edf9c50f2f3a0f25a324541d \
    && .\bootstrap-vcpkg.bat
+
+# fix wrong download link


@mgmgwi could you please elaborate on why we're installing 7-zip here? Please add the same as a comment.

We are not installing 7-zip here, we are replacing the download link that is configured in .\vcpkg\scripts\vcpkgTools.xml using the Linux tool sed.

https://www.nuget.org/api/v2/package/7-Zip.CommandLine/18.1.0 (not working) is replaced with

https://globalcdn.nuget.org/packages/7-zip.commandline.18.1.0.nupkg?packageVersion=18.1.0 (working)

hadoop-yetus · 2025-02-24T09:40:37Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7417/2/console in case of problems.

hadoop-yetus · 2025-02-24T09:40:42Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7417/3/console in case of problems.

hadoop-yetus · 2025-02-24T09:46:13Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7417/4/console in case of problems.

mgmgwi · 2025-02-24T09:46:46Z

@GauthamBanasandra thanks for the feedback. I updated the comment explaining what we do there.

hadoop-yetus · 2025-02-24T10:26:18Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 19s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
			_ trunk Compile Tests _
+0 🆗	mvndep	5m 28s		Maven dependency ordering for branch
+1 💚	shadedclient	25m 29s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 21s		Maven dependency ordering for patch
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	shadedclient	18m 55s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	asflicense	0m 23s		The patch does not generate ASF License warnings.
		46m 24s

Subsystem	Report/Notes
Docker	ClientAPI=1.48 ServerAPI=1.48 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7417/3/artifact/out/Dockerfile
GITHUB PR	#7417
Optional Tests	dupname asflicense
uname	Linux d1a346334361 5.15.0-130-generic #140-Ubuntu SMP Wed Dec 18 17:59:53 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `b6b9539`
Max. process+thread count	553 (vs. ulimit of 5500)
modules	C: U:
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7417/3/console
versions	git=2.25.1 maven=3.6.3
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2025-02-24T10:38:36Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	6m 57s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
			_ trunk Compile Tests _
+0 🆗	mvndep	5m 26s		Maven dependency ordering for branch
+1 💚	shadedclient	25m 19s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 23s		Maven dependency ordering for patch
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	shadedclient	19m 17s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	asflicense	0m 25s		The patch does not generate ASF License warnings.
		53m 16s

Subsystem	Report/Notes
Docker	ClientAPI=1.48 ServerAPI=1.48 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7417/4/artifact/out/Dockerfile
GITHUB PR	#7417
Optional Tests	dupname asflicense
uname	Linux f13f89fac95b 5.15.0-130-generic #140-Ubuntu SMP Wed Dec 18 17:59:53 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `0d3e08e`
Max. process+thread count	677 (vs. ulimit of 5500)
modules	C: U:
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7417/4/console
versions	git=2.25.1 maven=3.6.3
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2025-02-24T11:01:03Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 33s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
			_ trunk Compile Tests _
+0 🆗	mvndep	5m 49s		Maven dependency ordering for branch
+1 💚	shadedclient	41m 26s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 32s		Maven dependency ordering for patch
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	shadedclient	37m 6s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	asflicense	0m 38s		The patch does not generate ASF License warnings.
		81m 43s

Subsystem	Report/Notes
Docker	ClientAPI=1.48 ServerAPI=1.48 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7417/2/artifact/out/Dockerfile
GITHUB PR	#7417
Optional Tests	dupname asflicense
uname	Linux 19b7ec36eb35 5.15.0-131-generic #141-Ubuntu SMP Fri Jan 10 21:18:28 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `b6b9539`
Max. process+thread count	553 (vs. ulimit of 5500)
modules	C: U:
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7417/2/console
versions	git=2.25.1 maven=3.6.3
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

GauthamBanasandra

Thanks for the fix @mgmgwi. LGTM.

* This PR fixes a few outdated dependencies needed for the Hadoop build environment on Windows.

gabry-lab · 2025-09-29T09:10:02Z

@GauthamBanasandra
This CI job never runs successfully recently, can you help to fix it ?
https://ci-hadoop.apache.org/view/Hadoop/job/hadoop-qbt-trunk-java8-win10-x86_64/

GauthamBanasandra · 2025-10-04T08:41:16Z

@gabry-lab there's some issue with disk space on the Windows VMs used for the Jenkins runs. It's been failing for several months now.

I've been working on a Powershell script that sets up the Windows docker image locally and runs the full suite of the CI tests. With this, we won't need to rely on the Jenkins CI anymore. All one needs is a Windows VM with docker installed on it.

I'll create a PR once ready. It should take about 2 weeks.

gabry-lab · 2025-10-09T01:49:40Z

@GauthamBanasandra Thanks very much , it will be helpful if it's fixed.

github-actions bot added trunk Infra labels Feb 20, 2025

slfan1989 requested a review from GauthamBanasandra February 20, 2025 22:45

GauthamBanasandra requested changes Feb 24, 2025

View reviewed changes

mgmgwi force-pushed the HADOOP-19466-fix-windows-dockerfile branch 2 times, most recently from 9707cb0 to b6b9539 Compare February 24, 2025 09:38

HADOOP-19466. fix Dockerfile_windows_10

0d3e08e

mgmgwi force-pushed the HADOOP-19466-fix-windows-dockerfile branch from b6b9539 to 0d3e08e Compare February 24, 2025 09:44

mgmgwi requested a review from GauthamBanasandra February 24, 2025 11:57

GauthamBanasandra approved these changes Feb 24, 2025

View reviewed changes

GauthamBanasandra merged commit acc3e50 into apache:trunk Feb 24, 2025
4 checks passed

GauthamBanasandra assigned mgmgwi Feb 24, 2025

mgmgwi deleted the HADOOP-19466-fix-windows-dockerfile branch February 24, 2025 14:47

adideshpande pushed a commit to adideshpande/hadoop that referenced this pull request Feb 27, 2025

HADOOP-19466. fix Dockerfile_windows_10 (apache#7417)

d2e964c

* This PR fixes a few outdated dependencies needed for the Hadoop build environment on Windows.

YanivKunda pushed a commit to YanivKunda/hadoop that referenced this pull request Mar 23, 2025

HADOOP-19466. fix Dockerfile_windows_10 (apache#7417)

95b1f91

* This PR fixes a few outdated dependencies needed for the Hadoop build environment on Windows.

HADOOP-19466. Fix Dockerfile_windows_10 #7417

HADOOP-19466. Fix Dockerfile_windows_10 #7417

Uh oh!

Conversation

mgmgwi commented Feb 20, 2025

Description of PR

How was this patch tested?

For code changes:

Uh oh!

hadoop-yetus commented Feb 20, 2025

Uh oh!

hadoop-yetus commented Feb 20, 2025

Uh oh!

slfan1989 commented Feb 20, 2025

Uh oh!

GauthamBanasandra commented Feb 21, 2025

Uh oh!

mgmgwi commented Feb 23, 2025

Uh oh!

mgmgwi commented Feb 24, 2025

Uh oh!

GauthamBanasandra commented Feb 24, 2025

Uh oh!

GauthamBanasandra commented Feb 24, 2025

Uh oh!

GauthamBanasandra commented Feb 24, 2025

Uh oh!

GauthamBanasandra Feb 21, 2025

Choose a reason for hiding this comment

Uh oh!

mgmgwi Feb 24, 2025

Choose a reason for hiding this comment

Uh oh!

hadoop-yetus commented Feb 24, 2025

Uh oh!

hadoop-yetus commented Feb 24, 2025

Uh oh!

hadoop-yetus commented Feb 24, 2025

Uh oh!

mgmgwi commented Feb 24, 2025

Uh oh!

hadoop-yetus commented Feb 24, 2025

Uh oh!

hadoop-yetus commented Feb 24, 2025

Uh oh!

hadoop-yetus commented Feb 24, 2025

Uh oh!

GauthamBanasandra left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

gabry-lab commented Sep 29, 2025

Uh oh!

GauthamBanasandra commented Oct 4, 2025

Uh oh!

gabry-lab commented Oct 9, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants