Skip to content

Releases: reframe-hpc/reframe

ReFrame 2.15

20 Nov 10:26
e9a298a
Compare
Choose a tag to compare

ReFrame 2.15 Release Notes

New features and enhancements

  • Update retries stage/output directory naming convention (#580)
  • Full support for flexible task allocation (#522)
  • Add new command-line option for more verbose test listing (#506)
  • List only the checks that are supported on the current system (#487)
  • Use different stage and output directories for different retries of the same test (#464)

Bug fixes

  • Directly assign to num_tasks if --flex-alloc-tasks has an integer value (#558)
  • Show also hidden partitions in Slurm, when trying to flexibly allocate tasks (#553)
  • Fix redundant export statements (#448)
  • Fix retry report when test fails before the setup phase (#535)
  • Do not silently abort tasks in hold when all currently running tasks fail (#533)
  • Remove the validation of the name of the test's base classes (#480)
  • Remove time limit from build jobs (#474)

Regression tests

  • Update NAMD test (#574)
  • Run libsci_acc and libsci_resolve tests on compute nodes as well (#581)
  • Update GROMACS test for Dom (#579)
  • Ruby 2.2.2 NArray test (#575)
  • Add test checking if CRAY_CPU_TARGET is set (#572)
  • Update MAGMA tests (#571)
  • Update CP2K test (#559)
  • Adapt the GPU burn test for multi-gpu nodes (#564)
  • Disable Trilinos tests which fail due to the corresponding module (#551)
  • Adapt usertool tests to new syntax (#552)
  • Update GROMACS performance numbers (#548)
  • Remove Monch from supported systems (#537)
  • Remove read-only files attribute from the CP2K check (#529)
  • Remove wildcards from programming environment specification in tests (#534)
  • Fix NetCDF test on Kesch (#523)
  • Sync OpenMP versions for Dom/Daint in notool test (#519)
  • Add MCH CUDA stress test (#514)
  • Adapt boost tests to latest pe18.08 (#518)
  • Adapt the TensorFlow module name for PE 18.08 (#516)
  • Adapt Score-P modules to PE 18.08 (#515)
  • Add MCH halo exchange benchmark (#509)
  • Adapt IOR checks to build systems (#505)
  • Add MCH alltoallv check (#347)
  • Adjust LAMMPS performance values (#504)
  • Adapt the LAMMPS test to PE18.08 (#502)
  • Adapt QE test to PE18.08 (#501)
  • Remove PrgEnv-gnu support from the Scalapack test on Kesch (#500)
  • Adapt NetCDF test to Kesch (#482)
  • New OpenACC/CUDA/C++ test from MCH (#342)
  • Adapt Kesch tests to final CSCS prog. environments (#479)
  • Publish MAGMA tests and adapt to PE 18.08 (#463)
  • Remove the matrix file from version control (#484)
  • Set required version for tests using build systems (#476)
  • Add matrix file add20.rua in the src dir (#481)
  • Adapt trilinos regression test to PE18.08 and build systems (#466)
  • Adapt gpu direct test to build systems (#462)
  • Adapt CUDA NVML test to build systems (#461)
  • Increase GROMACS sanity check tolerance to 0.1% (#459)
  • Replace FFLAGS with FCFLAGS in Makefiles (#457)

Other

  • Add C2SM environments on Kesch (#512)
  • Re-enable running modified tests on Daint (#517)
  • Adapt Kesch settings (#477)
  • Automate deployment and generation of release notes (#460)
  • Document the --flex-alloc-tasks feature (#582)
  • Add User Lab Day slides to documentation (#520)
  • Update documentation on the topic of filtering tests by system (#492)

ReFrame 2.14

14 Sep 08:54
9ca7e89
Compare
Choose a tag to compare

ReFrame 2.14 Release Notes

New features and enhancements

  • Add support for Autotools-based projects (#452)
  • Add support for CMake (#447)
  • Add a new infrastructure for build systems (#340)
    • Setting compilation flags directly in the programming environment is now deprecated.
    • Passing arguments to the compile() method is now deprecated.
    • For more information on how to use the new build system feature, please refer to the tutorial and the reference guide.
  • Evaluate and log all performance values before asserting them (#359)
  • Allow purging modules before running regression tests (#399)

Bug fixes

  • Fix reading of logdir setting from system config (#445)
  • Convert message for inexistent test files to warning (#440)
  • Do not load environments twice (#441)
  • Add a persistent version of rmtree() (#437)
  • Filter out empty strings before writing preamble (#410)
  • Fix bug in unloading of conflicting modules (#405)
  • Enable sphinx versioning only for releases (#361)
  • Change the resources dir path (#358)
  • Fix expected OpenMP versions for Dom (#350)
  • Comment out Graylog handler from CSCS config (#352)
  • Fix site_configuration in docs (#344)

Regression tests

  • Adapt stream microbenchmark test to build systems (#420)
  • Adapt prgenv test to build systems (#449)
  • Fix sourcesdir for the scorep test (#455)
  • Update OSU microbenchmark test to new syntax and build system (#442)
  • Adapt GpuBandwidthTest to build_systems (#438)
  • Adapt perftools to use build systems (#439)
  • Adapt helloworld tests to build systems (#416)
  • Adapt netcdf tests to build systems (#434)
  • Adapt cdo test to PE 18.07 (#431)
  • Adapt ddt regression test to PE 18.07 (#425)
  • Adapt cuda_gdb to build systems (#428)
  • Adapt notools check to build systems (#427)
  • Adapt hdf5 tests to build systems (#422)
  • Adapt libsci_resolve test to PE 18.07 (#424)
  • Adapt scorep test to PE 18.07 (#426)
  • Adapt cuda checks to build systems (#421)
  • Adapt dgemm test to use build systems (#418)
  • Adapt the scalapack tests to use build systems (#417)
  • Adapt haswell_fma to buildsystems style (#415)
  • Adapt nvprof test to PE 18.07 (#414)
  • Adapt the nco test for PE 18.07 (#412)
  • Adapt Tensorflow tests for PE 18.07 (#409)
  • Adapt the Boost Python test for PE 18.07 (#408)
  • Adapt Paraview check for PE 18.07 (#401)
  • Adapt the petsc test for PE 18.07 (#404)
  • Update PrgEnv-pgi definitions on Kesch (#348)
  • New flexible CUDA memtest check (#346)

Other

  • Add testing for python3.7 in travis (#356)
  • Remove monch from the machinesList (#436)
  • New programming environments for Kesch (#432)
  • Temporarily disable running of checks on Daint (#406)
  • Check added/modified regression tests w.r.t master (#353)
  • Update documentation for the build systems feature (#433)

ReFrame 2.13

22 Jun 12:54
Compare
Choose a tag to compare

ReFrame 2.13 Release Notes

API Changes

  • Enhanced syntax of regression tests using decorators that eliminate the need of boilerplate code (#264, #294)
  • Support for specifying compatible ReFrame versions for tests (#287)
  • Enhance behavior of pre_run and post_run attributes, allowing to specify multiple job steps (#333)

Framework

  • Redesign of performance logging (#213)
    • Performance logging can now be configured from the configuration file
    • Two different backends: file logging, Graylog
  • More logging attributes added for formatting log messages (#213, #337)
  • Support for PBS job scheduler (#316)
  • Support for TMod4 (#282)
  • Include name of the regression test file to the list of checks (#288)
  • Better filtering of tests (#319)
  • Add a knob for controlling the 'sacct'/'squeue' ratio in Slurm 'sacct' backend (#289)
  • Migrate to pytest (#231)
  • Bug fixes: #293, #305, #307, #308, #312, #318, #323, #325

Regression tests

ReFrame 2.12

04 May 09:54
Compare
Choose a tag to compare

ReFrame 2.12 Release Notes

API Changes

  • post_run and pre_run attributes removed from the Job class
  • Support for wildcards in valid_prog_environs

Framework

  • Support for automatic retry of failing tests (new --max-retries option)
  • Support for custom configuration files (new -C option)
  • Poll rate of the asynchronous execution policy is now lowered
  • Unique test names are now enforced
  • Environment variables are expanded in the -c option
  • Documentation fixes and updates
  • Internal abstraction layer for treating ReFrame versions
  • Bug fixes: #192, #207, #221, #266, #274, #279

Regression tests

  • New regression test for TensorFlow
  • New flexible MPI Hello World test
  • Add numerical sanity checks for Amber, CP2K, CPMD, GROMACS, LAMMPS, NAMD
  • Corrected use of scoped dict key updates in the multi-GPU CUDA test
  • Bug fixes: #242

ReFrame 2.11

08 Mar 09:43
e201184
Compare
Choose a tag to compare

ReFrame 2.11 Release Notes

API Changes

None

Framework

  • New feature: support for module mappings
  • Submit jobs from the stage directory
  • Allow thresholds >= 1.0 in assert_reference() sanity function
  • Flexible allocation take into account the --exclude-nodes option
  • Add prefix rfm_ to the job names of ReFrame tests
  • Fix crash when an unknown modules system is set in the configuration
  • Fix and update documentation on the check_info logging attribute
  • Fix math domain error in the polling function of the asynchronous execution policy
  • Fix failure of the asynchronous execution policy's unit tests
  • Fix confusing error messages in serial execution policy
  • Fix emission of Slurm's --exclusive flag
  • Fix logging configuration in generic settings

Regression tests

  • Add regression tests for GPU direct
  • Add a G2G meteoswiss regression test
  • Add ICON RRTMGP regression test
  • Add numerical sanity check in quantum espresso test
  • Enable the switches resource certain tests
  • Fix deprecation warnings in regression tests

ReFrame 2.10

14 Feb 11:53
557e930
Compare
Choose a tag to compare

ReFrame 2.10 Release Notes

API Changes

  • RegressionTest.sourcesdir is now always copied to the stage directory
  • Support for specifying Git repositories in RegressionTest.sourcesdir
  • Job.pre_run and Job.post_run fields are deprecated in favour of the new ones RegressionTest.pre_run and RegressionTest.post_run

Framework

  • Support for the LMod modules system
  • Support for module aliases by the internal modules system abstraction
  • New more robust design of the regression test execution policies
  • More robust and efficient error checking for the Tmod modules system backend
  • Fix framework behaviour when trying to run interactive commands
  • Fir random unit tests failures with Python 3.5
  • Documentation fixes

ReFrame 2.9

24 Jan 14:11
53874c2
Compare
Choose a tag to compare

ReFrame 2.9 Release Notes

API Changes

  • Old sanity and performance patterns syntax completely removed
  • New extended syntax of extra_resources
  • Allow sourcesdir to be None even for compiled regression tests
  • More advanced examples in documentation

Framework

  • Redesign of the exception hierarchy
  • Version information moved away from settings
  • Support for flexible job allocations that span a full reservation
  • Support for easy deprecation of fields
  • Fix ungraceful exit when a job is blocked in in unrecoverable state
  • Fix regex group references to be compatible with Python 3.5
  • Fix random failures in unit tests
  • Fix behaviour of --partition option to override any similar option passed in site configuration

ReFrame 2.8.1

22 Dec 14:33
92c03e6
Compare
Choose a tag to compare

ReFrame 2.8.1 Release Notes

API Changes

None.

Framework

  • Suppress SKIP messages in the output
  • Support of an squeue-only Slurm backend
  • ReFrame version is now printed in the performance logs
  • New sanity function wrappers for itertools.chain(), glob.glob() and glob.iglob() functions
  • Sanity functions reading files now accept an encoding option
  • Fix typos in documentation
  • Fix broken images in documentation

ReFrame 2.8

08 Dec 00:06
d11a049
Compare
Choose a tag to compare

ReFrame 2.8 Release Notes

API Changes

  • New RegressionTest attribute, extra_resources, for specifying custom job resources
  • New API for retrieving and setting the parallel program launchers
  • New API for retrieving the job scheduler of a system partition

Framework

  • Redesign of the modules system backend
  • Tmod is no more required to run ReFrame
  • New system configuration option for setting the modules system
  • More detailed output in the performance log
  • Redesign of the job scheduler and parallel program launcher backends
  • Expand support of several new parallel program launchers (mpirun, mpiexec etc.)
  • Documentation fixes

ReFrame 2.7

09 Nov 11:04
2f51c5d
Compare
Choose a tag to compare

ReFrame 2.7 Release Notes

API Changes

  • New composable and flexible syntax for sanity and performance checking
  • Old syntax of RegressionTest.sanity_patterns and RegressionTest.perf_patterns is deprecated
  • RegressionTest.strict_check attribute behavior changed

Framework

  • New complete documentation with tutorial and reference guides
  • New command-line option --strict
  • New syntax in the configuration file
  • New resourcesdir entry in the systems section of the configuration file
  • Removed command-line option --relax-performance-check
  • More robust unit tests of the asynchronous execution policy
  • prefix_apps was removed from the settings.
  • Standardised the behaviour of scoped dictionaries
  • Code quality improvements
  • Fixed misleading failure reports in asynchronous execution policy
  • Fixed behaviour of the --nocolor option
  • Fixed frontend crash during failure report when a test fails early in its setup phase
  • Fixed crash when using execution modes