Test Simulation Mode

[wxtim]

Synopsis

Simulation Mode is currently largely untested. To ensure that #5641 doesn't break existing interfaces I intend to write a thorough set of tests for similar existing functionality. This ticket is a way of keeping notes about what I need to test.

Although the target of the work in #5641 is a new feature, fixing testing for existing features should be counted as a bug-fix, hence the 8.2.1 milestone.

Current state of play

There are functional tests in tests/f/modes for dummy mode, but not for simulation mode, at least on the surface. On closer insepection it looks like they rely on settings in [runtime][namespace]simulation which describes itself as

task configuration for workflow simulation and dummy run modes.

What should simulation mode do?

• Prevent a run directory being created for tasks.
• Take a time specified by:
  • [simulation]speedup factor * execution time limit if both are set.
  • else [simulation]default run length.
• Config item time limit buffer should not affect simulation mode (Docs claim that it is only used in dummy mode).
• Tasks should fail at cycle points specified by [simulation]fail cycle points.
• Tasks should fail on the first attempt, but not retries if [simulation]fail try 1 only set.
• Task event handers will only be called if disable task event handlers is set True.
• [ ]

[wxtim]

Bug found in testing

# flow.cylc
[scheduling]
    initial cycle point = 1651
    [[graph]]
        R1 = foo => bar

[runtime]
    [[foo]]
        execution time limit = PT20S
        [[[simulation]]]
            default run length = PT5S
            speedup factor = 5
    [[bar]]
        execution time limit = PT30S
        [[[simulation]]]
            speedup factor = 10
            default run length = PT1S

Using a stopwatch or by adding LOG.warning([t.rtconfig['job']['simulated run length'] for t in self.taskdefs.values()]) to the end of configure_sim_modes in cylc/flow/config.py you can see that for all itasks the simulated run length is the first item defined (you can change the definition order to demonstrate this) for all tasks.