Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

New Machine requirement: Windows server 2022 build systems #2938

Closed
sxa opened this issue Feb 15, 2023 · 12 comments · Fixed by #2993
Closed

New Machine requirement: Windows server 2022 build systems #2938

sxa opened this issue Feb 15, 2023 · 12 comments · Fixed by #2993
Assignees
@sxa @steelhead31
Labels
Machine Request
Milestone
2023-03 (March)

Comments

@sxa
Copy link
Member

sxa commented Feb 15, 2023

I need to request a new machine: As discussed in this week's PMC meeting we intende to move from building on Windows Server 2012 to the latest version now that @andrew-m-leonard has determined that we can produce binary identical builds of the latest JDKs across both versions.

  • New machine operating system (e.g. linux/windows/macos/solaris/aix): windows
  • New machine architecture (e.g. x64/aarch32/arm32/ppc64/ppc64le/sparc): x64
  • Provider (leave blank if it does not matter): Azure
  • Desired usage: build (and some test!)
  • Any unusual specification/setup required: We'll want to use the VS2019 fixed layout for the install
  • How many of them are required: 2 for build, maybe more for test (to replace existing ones)

Please explain what this machine is needed for: Building and testing Windows Server 2022 when 2012 goes out of support later this year.
@sxa sxa added this to the 2023-03 (March) milestone Feb 23, 2023
@sxa
Copy link
Member Author

sxa commented Mar 2, 2023

Provisioned build-azure-win2022-x64-1 (172.187.129.163)
No setup done yet (Will discuss with @andrew-m-leonard prior to deploying compiler)
Nagios and WinRM ports are enabled

@sxa
Copy link
Member Author

sxa commented Mar 2, 2023

Provisioned a 4 core version at 172.187.176.15 (Earlier one was 2 core which would be very low for our typical build machines. WinRM and NSClient (12489) ports are enables.

@sxa
Copy link
Member Author

sxa commented Mar 7, 2023 •
edited
Loading

These machines need to be deployed using the VS2019 layout from /Vendor_Files/windows/VS2019_layout.16.11.32106.67.tar.gz on the AWX server. This issue covers what was done to create and deploy that image. We need to distill that into the commands required to copy the layout across from AWX to the windows box and deploy it in place of the existing VS2019 role (ideally falling back to the community edition install/download if we don't have a layout file so that others can still use the playbooks)
This will enable reproducibility of the builds.

@steelhead31 steelhead31 moved this to In Progress in Adoptium 1Q 2023 Plan Mar 9, 2023
@steelhead31
Copy link
Contributor

steelhead31 commented Mar 10, 2023 •
edited
Loading

TLDR; - Install from layout into default program files(x86) path , after discussion detailed below.

Summary Of Slack Discussion About Install Path From Layout.  

All, Im currently implementing some changes to the ansible infrastructure windows playbooks regarding the Visual Studio 2019 install , currently the Visual Studio 2019 is installed into its default location, ie c:\program files\visual studio\2019,  however as part of the windows reproducible build work, the default installation , when installing from a pre-defined layout is c:\vs2019 ...   the new playbook changes, has the capability to install from a predefined layout, where available, and "fall back" to the download and install ( which has issues for reproducibility )...   are there any thoughts on whether the installation paths should be aligned for both scenarios?, and if so , which way should they be aligned ?,  a preference for c:\program files OR c:\vs2019 ?

leonarda  23 hours ago
I always prefer short vs2019, but that’s a personal setup, if one needs to find it the first place to look is Program Files…

sxa  23 hours ago
For the latter of those two reasons, and avoiding any risk of it not being detected properly by any of the processes, my preference would be to keep it in the current directory under Program Files - avoids any issues with the detection and any risk of ending up with two conflicting versions on the machine which could make problem determination harder.

@sxa
Copy link
Member Author

sxa commented Mar 14, 2023 •
edited
Loading

Testing (Note that in the absence of the win2012 build tag this machine will not automatically be used for builds yet. Verifying a few here by labelling them directly targetting the new machine:

FYI @andrew-m-leonard These aren't reproducible tests, but just verifying that three of the versions will build ok (Correct compilers available etc.) but they will let us see what the performance of the builds are like on the new system.

@steelhead31 steelhead31 mentioned this issue Mar 15, 2023
Merged
4 tasks
@sxa
Copy link
Member Author

sxa commented Mar 15, 2023

Running on the -2 machine:

Failed due to potential permissions issue:

13:27:03  [CHECKOUT] Checking out Adopt Pipelines https://github.com/adoptium/ci-jenkins-pipelines.git : master
[Pipeline] checkout
13:27:03  The recommended git tool is: git
13:27:09  No credentials specified
13:27:09  Cloning the remote Git repository
13:27:09  Cloning repository https://github.com/adoptium/ci-jenkins-pipelines.git
13:27:10  ERROR: Error cloning remote repo 'origin'
13:27:10  hudson.plugins.git.GitException: Command "git fetch --tags --force --progress -- https://github.com/adoptium/ci-jenkins-pipelines.git +refs/heads/*:refs/remotes/origin/*" returned status code 128:
13:27:10  stdout: 
13:27:10  stderr: fatal: detected dubious ownership in repository at '/cygdrive/c/workspace/openjdk-build'
13:27:10  To add an exception for this directory, call:
13:27:10  
13:27:10  	git config --global --add safe.directory /cygdrive/c/workspace/openjdk-build
13:27:10

@sxa
Copy link
Member Author

sxa commented Mar 15, 2023

Second machine now fixed:

@sxa
Copy link
Member Author

sxa commented Mar 16, 2023 •
edited
Loading

Both of those are showing as aborted but it looks like that might have been something downstream in the test jobs.

I'll run again:

There should be new runs of both of those jobs in tonight's scheduled run so we will be able to directly compare.

@andrew-m-leonard
Copy link
Contributor

@sxa Test job Aborted due to this.. Windows docker node ??? :

18:54:49  There are no nodes with the label ‘[ci.role.test&&hw.arch.x86&&sw.os.windows&&sw.tool.docker](https://ci.adoptium.net/label/ci.role.test&&hw.arch.x86&&sw.os.windows&&sw.tool.docker/)’

@sxa
Copy link
Member Author

sxa commented Mar 17, 2023

@sxa Test job Aborted due to this.. Windows docker node ??? :

Are you talking about the sanity.external one? I don't think that's generally run on Windows so can be ignored

@sxa
Copy link
Member Author

sxa commented Mar 17, 2023

Although sanity.openjdk isn't looking too happy either - some java_util stream failures.

@sxa
Copy link
Member Author

sxa commented Mar 20, 2023

adoptium/ci-jenkins-pipelines#659 will allow the JDK20+ jobs to run on these new machines

@github-project-automation github-project-automation bot moved this from In Progress to Done in Adoptium 1Q 2023 Plan Mar 20, 2023
@steelhead31 steelhead31 moved this from Done to In Progress in Adoptium 1Q 2023 Plan Mar 23, 2023
@steelhead31 steelhead31 moved this from In Progress to Done in Adoptium 1Q 2023 Plan Mar 23, 2023
@adamfarley adamfarley mentioned this issue May 7, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@sxa sxa

@steelhead31 steelhead31

Labels
Machine Request
Projects
No open projects
Adoptium 1Q 2023 Plan
Status: Done
Milestone
2023-03 (March)
3 participants
@sxa @andrew-m-leonard @steelhead31