Skip to content

Adding Batch Mode and Chain Mode

Latest
Compare
Choose a tag to compare
@geyang geyang released this 28 Dec 04:43
· 44 commits to main since this release

Jaynes has been updated to v0.8.0. This is a major re-write that adds support for chaining multiple scripts in a single SLURM job, and batch mode that makes a single API / SSH call to launch many srun/ GCE VM jobs.

The support is extensive: these are supported across SSH, GCE, AWS, and the Manager mode, and on all runners (Docker, SLURM, etc).

You can look for new examples in the [jaynes-starter-kit] under each folder:

https://github.com/geyang/jaynes-starter-kit/blob/master/04_slurm_configuration/launch_multiple_entries.py

if __name__ == "__main__":
    import jaynes

    # instr wrapper automatically sets the launch name.
    jaynes.config()
    thunk = instr(main, seed=100)
    jaynes.add(thunk)
    thunk = instr(main, seed=200)
    jaynes.chain(thunk)
    thunk = instr(main, seed=300)
    jaynes.add(thunk)
    thunk = instr(main, seed=400)
    jaynes.chain(thunk)

    jaynes.execute()

    jaynes.listen(100)

The new jaynes==0.8.0 requires ml-logger==0.8.60 to work.