Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

run_tests on Windows VPC runs failing #3707

Closed
sxa opened this issue Aug 13, 2024 · 6 comments
Closed

run_tests on Windows VPC runs failing #3707

sxa opened this issue Aug 13, 2024 · 6 comments
Assignees

Comments

@sxa
Copy link
Member

sxa commented Aug 13, 2024

Related: #3288 which shows failures on MBCS tests on Linux

Latest run from a branch of mine: https://ci.adoptium.net/view/Tooling/job/VagrantPlaybookCheck/OS=Win2022,label=vagrant/1950/console

I looked back in the history and the previous run that got as far as executing tests seems to have been https://ci.adoptium.net/view/Tooling/job/VagrantPlaybookCheck/OS=Win2022,label=vagrant/1854/console (although that didn't run a build, but the errors are the same as my new one)

c:/tmp/jdk-23+13
LIB_DIR is s/bin/sh: -c: line 4: syntax error: unexpected end of file
make: *** [makefile:105: compileTools] Error 2
make[1]: c:/tmp/jdk-22: Permission denied
make[1]: *** [makeGen.mk:45: autogen] Error 127
make: *** [makefile:60: _extended.system] Error 2
Warning: there was a problem converting the Powershell error message: syntax error: line 1, column 0
@steelhead31
Copy link
Contributor

Working ok in my new vagrant POC..

 https://ci.adoptium.net/job/SFR-VPC-AZURE-POC/OS=Win2022,label=vagrantX/29/console

06:55:18.850 PASSED test targets: 06:55:18.850 MachineInfo_0 06:55:18.850 MachineInfo_0 06:55:18.850 MachineInfo_0 06:55:18.850 MachineInfo_0 06:55:18.850 MathLoadTest_all_5m_0 06:55:18.850 MathLoadTest_all_5m_1 06:55:18.850 MachineInfo_0 06:55:18.850 MachineInfo_0 06:55:18.850 MachineInfo_0 06:55:18.850 MiniMix_5m_0 06:55:18.850 MiniMix_5m_1 06:55:18.850 MiniMix_10m_0 06:55:18.850 MiniMix_10m_1 06:55:18.850 MiniMix_aot_5m_0 06:55:18.850 ConcurrentLoadTest_5m_0 06:55:18.850 ConcurrentLoadTest_5m_1 06:55:18.850 DBBLoadTest_5m_0 06:55:18.850 DBBLoadTest_5m_1 06:55:18.850 LangLoadTest_5m_0 06:55:18.850 LangLoadTest_5m_1 06:55:18.850 LockingLoadTest_0 06:55:18.850 LockingLoadTest_1 06:55:18.850 UtilLoadTest_5m_0 06:55:18.850 UtilLoadTest_5m_1 06:55:18.850 MachineInfo_0 06:55:18.850 MachineInfo_0 06:55:18.850 06:55:18.850 TOTAL: 41   EXECUTED: 26   PASSED: 26   FAILED: 0   DISABLED: 7   SKIPPED: 8 06:55:18.850 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

@steelhead31
Copy link
Contributor

A single test failed in this run ( on the existing machine.. )

https://ci.adoptium.net/job/VagrantPlaybookCheck/OS=Win2022,label=vagrant/1951/console

This has been problematic in the past I believe, as seen in ( #3689 )

MiniMix_10m_1 => deep history 2/3 passed | possible issues

@sxa
Copy link
Member Author

sxa commented Aug 15, 2024

Not sure what happened with that run - if you look at the deep history link it shows it was from the job with parent build 343 which is https://ci.adoptium.net/job/AQA_Test_Pipeline/343/console and the extended.openjdk run stopped with:
``
Test_openjdk11_hs_extended.system_x86-64_linux result is ABORTED

It looks like it was one of @Haroon-Khel's runs while testing on Fedora 40.

Having said that, your runs are looking a bit better than mine, and if we've got somethignng intermittent in there (Maybe we should just kick off a modified VPC run that runs the tests 100 times over the weekend and see how many fail?) then we should look at switching it for something else.

@steelhead31
Copy link
Contributor

Run with 5x tests running here... will likely occupy the VPC win2022 for a while..
https://ci.adoptium.net/job/VagrantPlaybookCheck/1952/

@sxa
Copy link
Member Author

sxa commented Aug 16, 2024

Summary (Noting that this build was based on openjdk/jdk21u):

  • MiniMix_10m_1 failed in 4/5 runs
  • ConcurrentLoadTest_5m_0 failed in 2/5
  • LangLoadTest_5m_1 failed in 1/5
  • MiniMix_aot_5m_0 failed 1/5

All of the above are included in extended.system and there have been no failures of that job in the last five runs.

sxa@fedora:/dev/shm$ curl -s https://ci.adoptium.net/job/VagrantPlaybookCheck/OS=Win2022,label=vagrant/1952/consoleText | grep -A5 'FAILED test targets'
FAILED test targets:
	MiniMix_10m_1

TOTAL: 41   EXECUTED: 26   PASSED: 25   FAILED: 1   DISABLED: 7   SKIPPED: 8
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

--
FAILED test targets:
	ConcurrentLoadTest_5m_0

TOTAL: 41   EXECUTED: 26   PASSED: 25   FAILED: 1   DISABLED: 7   SKIPPED: 8
+++++++++++++++++++++++++++cat: 'C:/tmp/testLocation/openjdk-tests/TKGoutput_17237613599926../AQACert.log': No such file or directory
cat: 'C:/tmp/testLocation/openjdk-tests/TKGoutput_17237613599926../SHA.txt': No such file or directory
--
FAILED test targets:
	MiniMix_5m_1
	MiniMix_aot_5m_0

TOTAL: 41   EXECUTED: 26   PASSED: 24   FAILED: 2   DISABLED: 7   SKIPPED: 8
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
--
FAILED test targets:
	MiniMix_5m_0

TOTAL: 41   EXECUTED: 26   PASSED: 25   FAILED: 1   DISABLED: 7   SKIPPED: 8
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

--
FAILED test targets:
	MiniMix_10m_0
	ConcurrentLoadTest_5m_1
	LangLoadTest_5m_1

TOTAL: 41   EXECUTED: 26   PASSED: 23   FAILED: 3   DISABLED: 7   SKIPPED: 8
sxa@fedora:/dev/shm$ 

I suspect that we should look at taking these out of the VPC runs since they generally seem to be getting stuck with **POSSIBLE HANG DETECTED** messages maybe due to the spec of the VMs we are using for the playbook tests.

@smlambert The full extended_system target was originally chosen and added in this PR as it was considered a good validation of the playbooks as was a target that exercised perl, but I presume we could get away with a smaller subset of that which would still provide the validation that perl is working as expected. If there are particular tests that you'd like to see, whether in the system suite or elsewhere, when we run the Windows playbook validation let me know sine it's been 4½ years since we chose these ones.

For comparison on Linux (in testJDK.sh) we only run _MBCS_Tests_pref_ja_JP_linux_0 and some of the newer RH SSL tests (although I'm a bit nervous that it seems to be extracting the tests into $HOME/testLocation/ssl-tests instead of a directory within $WORKSPACE - IMHO we should change that too...)

@sxa sxa added this to the 2024-08 (August) milestone Aug 16, 2024
@sxa sxa moved this to In Progress in 2024 3Q Adoptium Plan Aug 16, 2024
@steelhead31
Copy link
Contributor

steelhead31 commented Sep 5, 2024

The current set of tests, now pass in the refreshed Vagrant environment. See https://ci.adoptium.net/job/VagrantPlaybookCheck/OS=Win2022,label=vagrant/1957/console

@github-project-automation github-project-automation bot moved this from In Progress to Done in 2024 3Q Adoptium Plan Sep 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
No open projects
Status: Done
Development

No branches or pull requests

2 participants