Skip to content

Conference call notes 20230705

Jump to bottom
Kenneth Hoste edited this page Aug 2, 2023 · 3 revisions

(back to Conference calls)

Notes on the 224th EasyBuild conference call, Wednesday 5 July 2023 (15:00 UTC)

Attendees

List of attendees (15):

  • Sebastian Achilles (JSC, Germany)
  • Simon Branford (Univ. of Birmingham, UK)
  • John Dey (Fred Hutch, Seattle, US
  • Alex Domingo (Vrije Universiteit Brussel, Belgium)
  • Jasper Grimm (University of York, UK)
  • Thomas Hayward-Schneider (Max Planck Institute for Plasma Physics (IPP))
  • Adam Huffman (Big Data Institute, Oxford, UK)
  • Kenneth Hoste (HPC-UGent, Belgium)
  • Kurt Lust (UAntwerpen, Belgium + LUMI User Support Team)
  • Sebastien Moretti (SIB, Switzerland)
  • Mikael Öhman (Chalmers University of Technology, Sweden)
  • Bart Oldeman (Digital Research Alliance of Canada)
  • Jörg Saßmannshausen (Imperial College London, UK)
  • Alexandre Strube (JSC, Germany
  • Davide Vanzo (Microsoft)

Agenda

  • overview of recent developments + outlook to next release
  • progress on 2023a update of common toolchains
  • Q&A

Recent developments

  • easyconfigs merge sprint: 21 + 28 June 2023
    • was successful, managed to bring down number open easyconfig PRs from ~850 to ~700
      • ~90 PRs closed
      • ~180 PRs merged
      • => ~270 PRs processed
    • we should probably have an easyconfigs merge sprint on a regular basis (monthly? every other month?)
  • request for feedback (in context of EasyBuild 5.0 effort)
    • minimum supported Lmod version (issue #871)
      • we will probably start requiring a recent Lmod 8.x version (8.3+) in EasyBuild 5.0 - more feedback still welcome!
      • Debian stable (12.x) has Lmod 8.x packaged
    • toolchain support policy (issue #872)
      • common ground found on this, being carved in stone in docs PR #200
        • 6 most recent (common) toolchain versions supported (easyconfigs actively maintained + accepting PRs)
        • next 2 toolchain versions deprecated (PRs are closed, but still included in EasyBuild releases)
        • easyconfigs using older toolchain versions are archived (=> still there for reference, not supported)
        • sliding window as new common toolchain versions are defined (twice a year)
  • release timeline
    • latest release: EasyBuild v4.7.2 (28 May 2023)
    • ETA next release (EasyBuild v4.7.3?): end of June'23
  • progress on EasyBuild 5.0
  • recent changes
    • docs (merged PRs)
      • ...
    • framework (merged PRs)
      • bug fixes
        • fix regex for extracting glibc version from output of 'ldd --version' in Gentoo Linux (PR #4290)
        • allow versionsuffix to be set to None in det_full_ec_version (PR #4292, fixes issue #4281)
      • enhancements
        • use version suffixes in PR title instead of Python versions when using --new-pr (PR #4253 + PR #4286)
      • changes
        • ...
      • EasyBuild 5.0 (to 5.0.x branch)
        • enable --trace by default (PR #4250)
        • update check_python_version for removed Python 2 support (PR #4270)
    • easyblocks (merged PRs)
      • bug fixes
        • reset modules loaded by PythonPackage to let ExtensionEasyBlock handle multi_deps correctly (PR #2951)
      • enhancements
        • add support for install_cmds in Binary easyblock (PR #2953)
      • updates
        • ...
      • changes
        • ...
      • new easyblocks
        • add PerlBundle generic easyblock for installing a bundle of Perl modules (PR #2945)
        • add custom easyblock for Rust (PR #2952)
      • EasyBuild 5.0 (to 5.0.x branch)
        • ...
    • easyconfigs (merged PRs)
  • work in progress
    • docs (open PRs + issues)
    • framework (open PRs + issues)
      • reported bugs / bug fixes
        • re-initializing easyblocks modifies easyconfig input parameter (issue #4289)
        • nested temporary directories for easystack items (issue #4291)
      • enhancements
        • suggest default title in --review-pr (PR #4287)
      • changes
        • ...
      • EasyBuild 5.0 (to 5.0.x branch)
        • initial implementation of run function (WIP) (PR #4284, see also issue #4252)
        • small PRs to clean up deprecated stuff:
          • remove support for 32-bit targets (PR #4272)
          • remove deprecated desc option for simple_option (PR #4273)
          • remove deprecated Toolchain.add_dependencies (PR #4274)
          • remove deprecated functionality from filetools (PR #4275)
          • remove deprecated log_error option from which command (PR #4276)
          • remove deprecated skip_lower option from template_constant_dict (PR #4277)
          • removed deprecated mod_exists_regex_template to exist (PR #4279)
          • remove deprecated options from easyconfig.py (PR #4278)
    • easyblocks (open PRs + issues)
      • bug reports/fixes
        • fixes issue in sympy test step when tmpdir is a symlink (PR #2949)
          • awaiting test report to confirm the fix + accompanying PR for sympy easyconfigs to make them use the new custom easyblock
        • disable disk space check in STAR-CCM+ installer (PR #2956)
      • enhancements
        • improve cpu-arch optimization settings for OpenCV (PR #2954)
      • updates
        • update MXNet easyblock + don't try to install R extension by default (PR #2955)
      • new easyblocks
        • ...
      • changes
        • drop unnecessary CUDA stub libraries from $LIBRARY_PATH (PR #2793)
    • easyconfigs (open PRs + issues)
      • still over 800 open easyconfig PRs...
      • bug fixes/reports
        • ...
      • enhancements
        • improve check for multi-variant dependencies per generation of easyconfigs (PR #12687)
      • new software
      • noteworthy software updates
        • PyTorch 1.13.1 (PR #17155 + PR #17156)
          • Are these good to go?
            • Probably just missing some patches that were added recently in existing PyTorch easyconfigs
          • Is anyone working on PyTorch 2.x?
            • Simon has briefly looked into this
            • CPU version is pretty much a copy of PyTorch 1.13.1
            • GPU distributed tests were failing, related to NCCL
              • BEAR is currently falling back to binary wheel for PyTorch 2.0
      • changes
        • ...

2023a update of common toolchains

  • foss/2023a and intel/2023a toochains should be included in EasyBuild v4.7.3 release
    • foss/2023.05 can be promoted in-place to foss/2023a (PR #18264)
      • GCC 12.3.0 (latest, except for GCC 13.1.0)
      • OpenMPI 4.1.5 (latest)
      • OpenBLAS 0.3.23 + FlexiBLAS 3.3.1 (latest)
      • FFTW 3.3.10 (latest)
      • ScaLAPACK 2.2.0 (latest)
    • intel-compilers 2023 update 1 is still altest release, so intel/2023.03 can be copied to intel/2023a only after GCCcore base compiler is changed to 12.3.0 first (PR #18263)
      • GCCcore/12.3.0 as base
      • Intel oneAPI compilers 2023.1.0 (latest)
      • Intel MPI 2021.9.0 (latest)
      • Intel MKL 2023.1.0 (latest)
  • see PRs marked with 2023a label
  • open PRs
    • bundles for Python extensions (PR #18208)
      • cryptography should be in a stand-alone easyconfig to have full control over installation using CargoPythonPackage?
      • will most likely not be included yet in EasyBuild v4.7.3 release
  • CUDA version for 2023a?
    • CUDA 12.1.x should be compatible with GCC 12.x, so OK
    • support for sm35 (K40) + sm37 (K80) were dropped (since CUDA 11.7)
    • easyconfig for CUDA 12.1.1 merged (PR #18092)

Q&A

  • ...
Clone this wiki locally