{devel}[SYSTEM] Reframe v4.6.2

[casparvl]

Split off from #21269 since this requires a bit more work at the system toolchain level...

As discussed with @bedroge on chat, the problem here is that Python 3.12 would need a newer setuptools (at least 66.1.0) to function. To keep at least python 3.7 support, it could be at most 68.0.0. We can not make something that works for both Python 3.6 and Python 3.12.

The only approach here would be to make two EasyConfigs: one that would work on Python 3.6-3.11 (based on setuptools 59.6.0, and one that would work on Python 3.7-3.12 (based on setuptools 68.0.0, and probably also a bumped version for wheel). I'll implement that here, though we should think of a relevant suffix... Maybe I'll just do one without suffix, and one with -python-3.6. I.e. the one that uses the newer setuptools would be considered the 'default' one, and only if you really still have a system python 3.6 do you use the older one.

WARNING: this PR requires easybuilders/easybuild-easyblocks#3431 in order to support the max_py_majver and max_py_minver keywords.

[bedroge]

Test report by @bedroge
SUCCESS
Build succeeded for 1 out of 1 (1 easyconfigs in total)
bob-Latitude-5300 - Linux Ubuntu 24.04.1 LTS (Noble Numbat), x86_64, Intel(R) Core(TM) i7-8665U CPU @ 1.90GHz, Python 3.12.3
See https://gist.github.com/bedroge/3962200fe7c226dedc392dce0e8ea30e for a full test report.

[casparvl]

@boegelbot please test @ generoso

[boegelbot]

@casparvl: Request for testing this PR well received on login1

PR test command 'EB_PR=21307 EB_ARGS= EB_CONTAINER= EB_REPO=easybuild-easyconfigs /opt/software/slurm/bin/sbatch --job-name test_PR_21307 --ntasks=4 ~/boegelbot/eb_from_pr_upload_generoso.sh' executed!

• exit code: 0
• output:

Submitted batch job 14181

Test results coming soon (I hope)...

- notification for comment with ID 2326583819 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

[boegelbot]

Test report by @boegelbot
FAILED
Failed during parsing of the easyconfigs, so no ecs were built (2 easyconfigs in total)
cns2 - Linux Rocky Linux 8.9, x86_64, Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz (haswell), Python 3.6.8
See https://gist.github.com/boegelbot/bc971f73f3037ad273a397eb18e6943f for a full test report.

[casparvl]

Ah, yes, we already found out in the other PR that both our test clusters are missing some os dependencies. So this failure should not be considered a problem, it is correct behavior of the EasyConfig.

[casparvl]

Test report by @casparvl
FAILED
Build succeeded for 1 out of 2 (2 easyconfigs in total)
cn175.karolina.it4i.cz - Linux Rocky Linux 8.9, x86_64, AMD EPYC 7H12 64-Core Processor (zen2), Python 3.6.8
See https://gist.github.com/casparvl/b222dae57cf698aeec1c0604adc7295c for a full test report.

[casparvl]

Test report by @casparvl
SUCCESS
Build succeeded for 2 out of 2 (2 easyconfigs in total)
tcn1.local.snellius.surf.nl - Linux RHEL 9.4, x86_64, AMD EPYC 7H12 64-Core Processor, Python 3.9.18
See https://gist.github.com/casparvl/91ee70e54442f569c4cb977d448865b1 for a full test report.

[casparvl]

The failure of ReFrame-4.6.2.eb is expected, as there it does not meet the python version requirement. The error is a bit unexpected though, I would have expected a clear error saying the Python version doesn't meet the requirement, but instead I get:

== 2024-09-03 18:45:54,654 filetools.py:541 WARNING Could not find command 'python' (with permissions to read/execute it) in $PATH (['/apps/all/EasyBuild/4.9.2/bin', '/apps/all/XALT/3.0.2/bin', '/opt/clmgr/sbin', '/opt/clmgr/bin', '/opt/sgi/sbin', '/opt/sgi/bin', '/home/it4i-casparl/.local/bin', '/home/it4i-casparl/bin', '/usr/share/Modules/bin', '/usr/local/bin', '/usr/bin', '/usr/local/sbin', '/usr/sbin', '/opt/c3/bin', '/sbin', '/bin', '/opt/slurm/bin', '/opt/icommands'])
== 2024-09-03 18:45:54,655 filetools.py:541 WARNING Could not find command 'python' (with permissions to read/execute it) in $PATH (['/apps/all/EasyBuild/4.9.2/bin', '/apps/all/XALT/3.0.2/bin', '/opt/clmgr/sbin', '/opt/clmgr/bin', '/opt/sgi/sbin', '/opt/sgi/bin', '/home/it4i-casparl/.local/bin', '/home/it4i-casparl/bin', '/usr/share/Modules/bin', '/usr/local/bin', '/usr/bin', '/usr/local/sbin', '/usr/sbin', '/opt/c3/bin', '/sbin', '/bin', '/opt/slurm/bin', '/opt/icommands'])
== 2024-09-03 18:45:54,655 filetools.py:526 INFO Command python3 found at /usr/bin/python3
== 2024-09-03 18:45:54,655 run.py:247 INFO running cmd: python3 -c 'import sys; print("%s.%s.%s" % sys.version_info[:3])'
== 2024-09-03 18:45:54,685 filetools.py:541 WARNING Could not find command 'python3.7' (with permissions to read/execute it) in $PATH (['/apps/all/EasyBuild/4.9.2/bin', '/apps/all/XALT/3.0.2/bin', '/opt/clmgr/sbin', '/opt/clmgr/bin', '/opt/sgi/sbin', '/opt/sgi/bin', '/home/it4i-casparl/.local/bin', '/home/it4i-casparl/bin', '/usr/share/Modules/bin', '/usr/local/bin', '/usr/bin', '/usr/local/sbin', '/usr/sbin', '/opt/c3/bin', '/sbin', '/bin', '/opt/slurm/bin', '/opt/icommands'])
== 2024-09-03 18:45:54,685 run.py:247 INFO running cmd: /usr/bin/python3 -c 'import sys; print("%s.%s.%s" % sys.version_info[:3])'
== 2024-09-03 18:45:54,713 run.py:247 INFO running cmd: python -c 'import sys; print("%s.%s.%s" % sys.version_info[:3])'
== 2024-09-03 18:45:54,752 build_log.py:171 ERROR EasyBuild crashed with an error (at easybuild/base/exceptions.py:126 in __init__): cmd "python -c 'import sys; print("%s.%s.%s" % sys.version_info[:3])'" exited with exit code 127 and output:
/bin/bash: python: command not found

I would have expected easybuild to just use the info it obtained here:

== 2024-09-03 18:45:54,655 run.py:247 INFO running cmd: python3 -c 'import sys; print("%s.%s.%s" % sys.version_info[:3])'

to conclude the python version isn't up to the specified requirement. Oh well, that's not an issue of this EasyConfig PR.

[casparvl]

Ok, I investigated a bit more with --debug. I now see:

== 2024-09-03 18:57:18,972 run.py:689 DEBUG cmd "/usr/bin/python3 -c 'import sys; print("%s.%s.%s" % sys.version_info[:3])'" exited with exit code 0 and output:
3.6.8

== 2024-09-03 18:57:18,972 run.py:725 DEBUG Using default regular expression: (?<![(,-]|\w)(?:error|segmentation fault|failed)(?![(,-]|\.?\w)
== 2024-09-03 18:57:18,972 pythonpackage.py:130 DEBUG Minimal requirement for minor Python version not satisfied: 3.6.8 vs 3.7
== 2024-09-03 18:57:18,972 pythonpackage.py:159 DEBUG Python command '/usr/bin/python3' does not satisfy version requirements (maj: 3, min: 7), moving on
== 2024-09-03 18:57:18,973 run.py:221 DEBUG run_cmd: running cmd python -c 'import sys; print("%s.%s.%s" % sys.version_info[:3])' (in /home/it4i-casparl/.local/easybuild/build/ReFrame/4.6.2/system-system)
== 2024-09-03 18:57:18,973 run.py:247 INFO running cmd: python -c 'import sys; print("%s.%s.%s" % sys.version_info[:3])'
== 2024-09-03 18:57:19,000 build_log.py:171 ERROR EasyBuild crashed with an error (at easybuild/base/exceptions.py:126 in __init__): cmd "python -c 'import sys; print("%s.%s.%s" % sys.version_info[:3])'" exited with exit code 127 and output:
/bin/bash: python: command not found

Why is https://github.com/easybuilders/easybuild-easyblocks/blob/cc5b95dab1aabae2a64360c492a4f068f8eab267/easybuild/easyblocks/generic/pythonpackage.py#L158 only writing to the debug log and continuing? I would assume that if a certain requirement is set in the EasyConfig file, and if that isn't met, EasyBuild would shout at me, and then exit, or at least, I'd think that'd be the appropriate behavior :\

[casparvl]

Ok, easybuilders/easybuild-easyblocks#3430 should fix that. Let's test...

[casparvl]

Test report by @casparvl
Using easyblocks from PR(s) easybuilders/easybuild-easyblocks#3430
FAILED
Build succeeded for 0 out of 1 (1 easyconfigs in total)
cn051.karolina.it4i.cz - Linux Rocky Linux 8.9, x86_64, AMD EPYC 7H12 64-Core Processor (zen2), Python 3.6.8
See https://gist.github.com/casparvl/2be4191e699207e43ceb19c6665c77c4 for a full test report.

[casparvl]

Ok, that failure was actually to prove the EasyBlock change in easybuilders/easybuild-easyblocks#3430 :) It is an expected failure, since the OS python there doesn't meet the requirement set in this EasyConfig.

[casparvl]

Test report by @casparvl
SUCCESS
Build succeeded for 2 out of 2 (2 easyconfigs in total)
gcn6.local.snellius.surf.nl - Linux RHEL 9.4, x86_64, Intel(R) Xeon(R) Platinum 8360Y CPU @ 2.40GHz, 4 x NVIDIA NVIDIA A100-SXM4-40GB, 555.42.06, Python 3.9.18
See https://gist.github.com/casparvl/34b00ed5b28d0e4d3600af71939a4a78 for a full test report.

[casparvl]

Test report by @casparvl
SUCCESS
Build succeeded for 1 out of 1 (1 easyconfigs in total)
cn051.karolina.it4i.cz - Linux Rocky Linux 8.9, x86_64, AMD EPYC 7H12 64-Core Processor (zen2), Python 3.6.8
See https://gist.github.com/casparvl/2ad5c3a81c784a2721f25e27593d6aba for a full test report.

[casparvl]

Test report by @casparvl
FAILED
Build succeeded for 0 out of 1 (1 easyconfigs in total)
vglogin0001.vega.izum.si - Linux RHEL 8.8, x86_64, AMD EPYC 7502 32-Core Processor, Python 3.6.8
See https://gist.github.com/casparvl/10f7a1d5d033b01470f2a8014d85d6ed for a full test report.

[casparvl]

Ugh, system python has a setuptools that is too old on Vega, and it seems that is the one used. I'm not sure why though. :\ It doesn't seem to be the case on another system (Karolina).

[casparvl]

Test report by @casparvl
Using easyblocks from PR(s) easybuilders/easybuild-easyblocks#3431
SUCCESS
Build succeeded for 2 out of 2 (2 easyconfigs in total)
gcn6.local.snellius.surf.nl - Linux RHEL 9.4, x86_64, Intel(R) Xeon(R) Platinum 8360Y CPU @ 2.40GHz, 4 x NVIDIA NVIDIA A100-SXM4-40GB, 555.42.06, Python 3.9.18
See https://gist.github.com/casparvl/8b4b3b96790241d84700d6837912d1d5 for a full test report.

[casparvl]

Test report by @casparvl
SUCCESS
Build succeeded for 1 out of 1 (1 easyconfigs in total)
cn0056 - Linux RHEL 8.8, x86_64, AMD EPYC 7H12 64-Core Processor, Python 3.6.8
See https://gist.github.com/casparvl/8f35e32a8260be27a052ab0988f2858e for a full test report.

[casparvl]

Ugh, system python has a setuptools that is too old on Vega, and it seems that is the one used. I'm not sure why though. :\ It doesn't seem to be the case on another system (Karolina).

I'm not sure what kind of funny stuff is happening on the Vega login nodes, but on the batch nodes, this seems to work fine (see this test report).

I'm not going to spend my time debugging that behavior on the login node, I think we have enough successes to show that this set of EasyConfigs is sane. There is always some degree of unexpected behavior possible for EasyConfigs at the system toolchain level...

[bedroge]

Test report by @bedroge
Using easyblocks from PR(s) easybuilders/easybuild-easyblocks#3431
FAILED
Build succeeded for 0 out of 1 (1 easyconfigs in total)
bob-Latitude-5300 - Linux Ubuntu 24.04.1 LTS (Noble Numbat), x86_64, Intel(R) Core(TM) i7-8665U CPU @ 1.90GHz, Python 3.12.3
See https://gist.github.com/bedroge/bd31e7e2c540e3561147111847661303 for a full test report.

Expected failure when using Python 3.12 while only versions up to 3.11 are specified as supported.

[bedroge]

Test report by @bedroge
Using easyblocks from PR(s) easybuilders/easybuild-easyblocks#3431
SUCCESS
Build succeeded for 1 out of 1 (1 easyconfigs in total)
bob-Latitude-5300 - Linux Ubuntu 24.04.1 LTS (Noble Numbat), x86_64, Intel(R) Core(TM) i7-8665U CPU @ 1.90GHz, Python 3.12.3
See https://gist.github.com/bedroge/63b97257a046f2d2409e982bec50a9c9 for a full test report.

[bedroge]

Lgtm

[bedroge]

Going in, thanks @casparvl!