Releases: reframe-hpc/reframe
Releases · reframe-hpc/reframe
ReFrame 2.15
ReFrame 2.15 Release Notes
New features and enhancements
- Update retries stage/output directory naming convention (#580)
- Full support for flexible task allocation (#522)
- Add new command-line option for more verbose test listing (#506)
- List only the checks that are supported on the current system (#487)
- Use different stage and output directories for different retries of the same test (#464)
Bug fixes
- Directly assign to
num_tasks
if--flex-alloc-tasks
has an integer value (#558) - Show also hidden partitions in Slurm, when trying to flexibly allocate tasks (#553)
- Fix redundant export statements (#448)
- Fix retry report when test fails before the setup phase (#535)
- Do not silently abort tasks in hold when all currently running tasks fail (#533)
- Remove the validation of the name of the test's base classes (#480)
- Remove time limit from build jobs (#474)
Regression tests
- Update NAMD test (#574)
- Run libsci_acc and libsci_resolve tests on compute nodes as well (#581)
- Update GROMACS test for Dom (#579)
- Ruby 2.2.2 NArray test (#575)
- Add test checking if
CRAY_CPU_TARGET
is set (#572) - Update MAGMA tests (#571)
- Update CP2K test (#559)
- Adapt the GPU burn test for multi-gpu nodes (#564)
- Disable Trilinos tests which fail due to the corresponding module (#551)
- Adapt usertool tests to new syntax (#552)
- Update GROMACS performance numbers (#548)
- Remove Monch from supported systems (#537)
- Remove read-only files attribute from the CP2K check (#529)
- Remove wildcards from programming environment specification in tests (#534)
- Fix NetCDF test on Kesch (#523)
- Sync OpenMP versions for Dom/Daint in notool test (#519)
- Add MCH CUDA stress test (#514)
- Adapt boost tests to latest pe18.08 (#518)
- Adapt the TensorFlow module name for PE 18.08 (#516)
- Adapt Score-P modules to PE 18.08 (#515)
- Add MCH halo exchange benchmark (#509)
- Adapt IOR checks to build systems (#505)
- Add MCH
alltoallv
check (#347) - Adjust LAMMPS performance values (#504)
- Adapt the LAMMPS test to PE18.08 (#502)
- Adapt QE test to PE18.08 (#501)
- Remove
PrgEnv-gnu
support from the Scalapack test on Kesch (#500) - Adapt NetCDF test to Kesch (#482)
- New OpenACC/CUDA/C++ test from MCH (#342)
- Adapt Kesch tests to final CSCS prog. environments (#479)
- Publish MAGMA tests and adapt to PE 18.08 (#463)
- Remove the matrix file from version control (#484)
- Set required version for tests using build systems (#476)
- Add matrix file add20.rua in the src dir (#481)
- Adapt trilinos regression test to PE18.08 and build systems (#466)
- Adapt gpu direct test to build systems (#462)
- Adapt CUDA NVML test to build systems (#461)
- Increase GROMACS sanity check tolerance to 0.1% (#459)
- Replace FFLAGS with FCFLAGS in Makefiles (#457)
Other
- Add C2SM environments on Kesch (#512)
- Re-enable running modified tests on Daint (#517)
- Adapt Kesch settings (#477)
- Automate deployment and generation of release notes (#460)
- Document the
--flex-alloc-tasks
feature (#582) - Add User Lab Day slides to documentation (#520)
- Update documentation on the topic of filtering tests by system (#492)
ReFrame 2.14
ReFrame 2.14 Release Notes
New features and enhancements
- Add support for Autotools-based projects (#452)
- Add support for CMake (#447)
- Add a new infrastructure for build systems (#340)
- Setting compilation flags directly in the programming environment is now deprecated.
- Passing arguments to the
compile()
method is now deprecated. - For more information on how to use the new build system feature, please refer to the tutorial and the reference guide.
- Evaluate and log all performance values before asserting them (#359)
- Allow purging modules before running regression tests (#399)
Bug fixes
- Fix reading of
logdir
setting from system config (#445) - Convert message for inexistent test files to warning (#440)
- Do not load environments twice (#441)
- Add a persistent version of rmtree() (#437)
- Filter out empty strings before writing preamble (#410)
- Fix bug in unloading of conflicting modules (#405)
- Enable sphinx versioning only for releases (#361)
- Change the resources dir path (#358)
- Fix expected OpenMP versions for Dom (#350)
- Comment out Graylog handler from CSCS config (#352)
- Fix
site_configuration
in docs (#344)
Regression tests
- Adapt stream microbenchmark test to build systems (#420)
- Adapt prgenv test to build systems (#449)
- Fix sourcesdir for the scorep test (#455)
- Update OSU microbenchmark test to new syntax and build system (#442)
- Adapt GpuBandwidthTest to build_systems (#438)
- Adapt perftools to use build systems (#439)
- Adapt helloworld tests to build systems (#416)
- Adapt netcdf tests to build systems (#434)
- Adapt cdo test to PE 18.07 (#431)
- Adapt ddt regression test to PE 18.07 (#425)
- Adapt cuda_gdb to build systems (#428)
- Adapt notools check to build systems (#427)
- Adapt hdf5 tests to build systems (#422)
- Adapt libsci_resolve test to PE 18.07 (#424)
- Adapt scorep test to PE 18.07 (#426)
- Adapt cuda checks to build systems (#421)
- Adapt dgemm test to use build systems (#418)
- Adapt the scalapack tests to use build systems (#417)
- Adapt haswell_fma to buildsystems style (#415)
- Adapt nvprof test to PE 18.07 (#414)
- Adapt the nco test for PE 18.07 (#412)
- Adapt Tensorflow tests for PE 18.07 (#409)
- Adapt the Boost Python test for PE 18.07 (#408)
- Adapt Paraview check for PE 18.07 (#401)
- Adapt the petsc test for PE 18.07 (#404)
- Update PrgEnv-pgi definitions on Kesch (#348)
- New flexible CUDA memtest check (#346)
Other
- Add testing for python3.7 in travis (#356)
- Remove monch from the machinesList (#436)
- New programming environments for Kesch (#432)
- Temporarily disable running of checks on Daint (#406)
- Check added/modified regression tests w.r.t master (#353)
- Update documentation for the build systems feature (#433)
ReFrame 2.13
ReFrame 2.13 Release Notes
API Changes
- Enhanced syntax of regression tests using decorators that eliminate the need of boilerplate code (#264, #294)
- Support for specifying compatible ReFrame versions for tests (#287)
- Enhance behavior of
pre_run
andpost_run
attributes, allowing to specify multiple job steps (#333)
Framework
- Redesign of performance logging (#213)
- Performance logging can now be configured from the configuration file
- Two different backends: file logging, Graylog
- More logging attributes added for formatting log messages (#213, #337)
- Support for PBS job scheduler (#316)
- Support for TMod4 (#282)
- Include name of the regression test file to the list of checks (#288)
- Better filtering of tests (#319)
- Add a knob for controlling the 'sacct'/'squeue' ratio in Slurm 'sacct' backend (#289)
- Migrate to
pytest
(#231) - Bug fixes: #293, #305, #307, #308, #312, #318, #323, #325
Regression tests
ReFrame 2.12
ReFrame 2.12 Release Notes
API Changes
post_run
andpre_run
attributes removed from theJob
class- Support for wildcards in
valid_prog_environs
Framework
- Support for automatic retry of failing tests (new
--max-retries
option) - Support for custom configuration files (new
-C
option) - Poll rate of the asynchronous execution policy is now lowered
- Unique test names are now enforced
- Environment variables are expanded in the
-c
option - Documentation fixes and updates
- Internal abstraction layer for treating ReFrame versions
- Bug fixes: #192, #207, #221, #266, #274, #279
Regression tests
- New regression test for TensorFlow
- New flexible MPI Hello World test
- Add numerical sanity checks for Amber, CP2K, CPMD, GROMACS, LAMMPS, NAMD
- Corrected use of scoped dict key updates in the multi-GPU CUDA test
- Bug fixes: #242
ReFrame 2.11
ReFrame 2.11 Release Notes
API Changes
None
Framework
- New feature: support for module mappings
- Submit jobs from the stage directory
- Allow thresholds >= 1.0 in
assert_reference()
sanity function - Flexible allocation take into account the
--exclude-nodes
option - Add prefix
rfm_
to the job names of ReFrame tests - Fix crash when an unknown modules system is set in the configuration
- Fix and update documentation on the
check_info
logging attribute - Fix math domain error in the polling function of the asynchronous execution policy
- Fix failure of the asynchronous execution policy's unit tests
- Fix confusing error messages in serial execution policy
- Fix emission of Slurm's
--exclusive
flag - Fix logging configuration in generic settings
Regression tests
- Add regression tests for GPU direct
- Add a G2G meteoswiss regression test
- Add ICON RRTMGP regression test
- Add numerical sanity check in quantum espresso test
- Enable the
switches
resource certain tests - Fix deprecation warnings in regression tests
ReFrame 2.10
ReFrame 2.10 Release Notes
API Changes
RegressionTest.sourcesdir
is now always copied to the stage directory- Support for specifying Git repositories in
RegressionTest.sourcesdir
Job.pre_run
andJob.post_run
fields are deprecated in favour of the new onesRegressionTest.pre_run
andRegressionTest.post_run
Framework
- Support for the LMod modules system
- Support for module aliases by the internal modules system abstraction
- New more robust design of the regression test execution policies
- More robust and efficient error checking for the Tmod modules system backend
- Fix framework behaviour when trying to run interactive commands
- Fir random unit tests failures with Python 3.5
- Documentation fixes
ReFrame 2.9
ReFrame 2.9 Release Notes
API Changes
- Old sanity and performance patterns syntax completely removed
- New extended syntax of
extra_resources
- Allow
sourcesdir
to beNone
even for compiled regression tests - More advanced examples in documentation
Framework
- Redesign of the exception hierarchy
- Version information moved away from settings
- Support for flexible job allocations that span a full reservation
- Support for easy deprecation of fields
- Fix ungraceful exit when a job is blocked in in unrecoverable state
- Fix regex group references to be compatible with Python 3.5
- Fix random failures in unit tests
- Fix behaviour of
--partition
option to override any similar option passed in site configuration
ReFrame 2.8.1
ReFrame 2.8.1 Release Notes
API Changes
None.
Framework
- Suppress SKIP messages in the output
- Support of an squeue-only Slurm backend
- ReFrame version is now printed in the performance logs
- New sanity function wrappers for
itertools.chain()
,glob.glob()
andglob.iglob()
functions - Sanity functions reading files now accept an encoding option
- Fix typos in documentation
- Fix broken images in documentation
ReFrame 2.8
ReFrame 2.8 Release Notes
API Changes
- New
RegressionTest
attribute,extra_resources
, for specifying custom job resources - New API for retrieving and setting the parallel program launchers
- New API for retrieving the job scheduler of a system partition
Framework
- Redesign of the modules system backend
- Tmod is no more required to run ReFrame
- New system configuration option for setting the modules system
- More detailed output in the performance log
- Redesign of the job scheduler and parallel program launcher backends
- Expand support of several new parallel program launchers (
mpirun
,mpiexec
etc.) - Documentation fixes
ReFrame 2.7
ReFrame 2.7 Release Notes
API Changes
- New composable and flexible syntax for sanity and performance checking
- Old syntax of
RegressionTest.sanity_patterns
andRegressionTest.perf_patterns
is deprecated RegressionTest.strict_check
attribute behavior changed
Framework
- New complete documentation with tutorial and reference guides
- New command-line option
--strict
- New syntax in the configuration file
- New
resourcesdir
entry in the systems section of the configuration file - Removed command-line option
--relax-performance-check
- More robust unit tests of the asynchronous execution policy
prefix_apps
was removed from the settings.- Standardised the behaviour of scoped dictionaries
- Code quality improvements
- Fixed misleading failure reports in asynchronous execution policy
- Fixed behaviour of the
--nocolor
option - Fixed frontend crash during failure report when a test fails early in its setup phase
- Fixed crash when using execution modes