BEE Release 0.1.9

Major features:

• Updated the Graph Database to use one instance of Neo4j for all workflows for a user vs. using one for each workflow.
• Added Python API for generating cwl specification of a workflow.
• Added coverage metrics for unit tests.
• Added resiliency to Task Manager to update tasks status when coming back up
• Enhanced pre/post script capabilities (added flux capability) and added checks for shell compatibility.
• Added capability to export dag of a workflow before and during execution

What's Changed

• Ensure failed tasks cause dependent tasks to fail by @jtronge in #806
• Update Neo4j to 5.x by @jtronge in #819
• Archive failed checkpoint restart workflows by @jtronge in #824
• Add pre/post script support to flux worker by @kchilleri in #818
• Add integration test for pre/post scripts (beeflow:ScriptRequirement) by @jtronge in #838
• Update Task Manager for task update resiliency by @jtronge in #827
• Restart GDB by @rstyd in #759
• Allow removal of a workflow with any Archive status by @pagrubel in #828
• Add Pytest coverage by @aquan9 in #843
• Test slurm commands worker in CI by @jtronge in #839
• Check for missing workflow inputs by @jtronge in #842
• Validate shell environment in pre/post script files by @kchilleri in #844
• Add Script Option to Pre/Post Script & Change MPI Call by @rstyd in #856
• Add MPI integration test code by @jtronge in #858
• Fix edge case in badge generation. by @aquan9 in #851
• Security concrete by @aquan9 in #852
• Remove enhanced client from github by @rstyd in #865
• Bump gunicorn from 20.1.0 to 22.0.0 by @dependabot in #862
• Bump urllib3 from 1.26.18 to 1.26.19 by @dependabot in #863
• Enable beeflow submit from working directory by @jtronge in #854
• Update work_dir path by @Leahh02 in #869
• Issue735/clear workflow by @kabir-vats in #872
• Pause running workflows when stopping beeflow with 'beeflow core stop' by @pagrubel in #830
• Bump certifi from 2024.6.2 to 2024.7.4 by @dependabot in #878
• Enable shell option on flux worker by @kchilleri in #866
• Issue871/delete_use_archive_config by @Leahh02 in #879
• Update documentation for the integration tests by @jtronge in #887
• Validate shell option in cwl file; ensure it matches shell interpreter in pre/post scripts by @kchilleri in #880
• Stop checking job state when it's completed by @Leahh02 in #892
• Add WorkerError exception handling by @jtronge in #888
• Add check for /tmp or /var/tmp by @Leahh02 in #885
• Generate CWL by @rstyd in #841
• Only Run One Neo4j Instance for a BEE Instance by @kabir-vats in #882
• Remove shebang lines from pre/post scripts by @kchilleri in #895
• Add "beeflow core info" by @aquan9 in #908
• Fix Reexecute by @kabir-vats in #902
• Update Poetry developer's guide by @Leahh02 in #899
• Update Git workflow documentation by @Leahh02 in #897
• Bump cryptography from 42.0.8 to 43.0.1 by @dependabot in #916
• Add Beeflow connect - A REST API for BEEflow itself. by @aquan9 in #837
• Add No Start for workflows submitted but not started by @Leahh02 in #914
• Add task state when creating task for checkpoint by @pagrubel in #923
• Export dag by @Leahh02 in #909
• Release 0.1.9 by @pagrubel in #925

Full Changelog: 0.1.8...0.1.9

BEE Release 0.1.8

Features: Fixes sphinx version to enable publishing documentation, now includes CI for testing documentation builds

• Update sphinx version, update actions and release docs (#812)
• Add separate action for testing docs
• Fix beeflow config new error

BEE release 0.1.7

Major features: adds the capability to include post- and pre-processing scripts to tasks, fixes the Checkpoint/Restart capability, increases logging, and adds some features to the client.

• Initial task manager resiliency and error handling (#789)
• Add pre/post script support (#788)
• Fix LOCALE error for systems where redis container failed to start
• Add logging to workflow interface (#764)
  • Enable logging in neo4j_cypher.py, neo4j_driver.py, and gdb_driver.py
• Add beeflow remove command to client
  • Enables removal of archived or cancelled workflows and associated artifacts
• Update minimum Charliecloud version to 0.36
• CI refactor to allow running jobs on runners other than github
• Add sumbit command options to workflow artifact for archive purposes
• Increase maximum version of python to 3.12
• Fix Checkpoint/Restart capability
• Add testing for Checkpoint/Restart
• Adds capability to reset the beeflow files (deletes all artifacts) especially useful for developers.

BEE release 0.1.6

Release 0.1.6 is mainly a clean up of processes, logs and directory space

• Eliminates extraneous Neo4j instances from cancelled/failed tasks
• Cleans up log entries for query
• Improves start time for celery
• Makes start time configurable
• Decreases the number of celery processes
• Fixes capability to specify a main cwl file and/or yaml file not in the CWL directory
• Parses CWL after packaging the directory
• Moves temporary files for unit tests out of $HOME

BEE release 0.1.5

• Combined beeflow, beeclient and beecfg commands. All commands now are invoked via beeflow.
• Fixed an obscure dependency issue between tasks
• Simplified config file, deleted duplications of bee_workdir
• CWL Parser was moved to the client
  • CwlParser is now instantiated in bee_client.py
  • CwlParser no longer invokes Workflow Interface, now returns Workflow and Task objects
  • Allows verification of CWL specification without running the workflow
• Added support for Flux scheduler

What's Changed

• Flux support by @jtronge in #696
• Move parser to client by @Boogie3D in #697
• Remove bee_workdir duplication in config file by @jtronge in #709
• Fix Dependencies Bug by @Boogie3D in #713
• Combine beeflow, beeclient, and beecfg by @jtronge in #710

Full Changelog: 0.1.4...0.1.5

BEE release 0.1.4

What's Changed

• Scheduler options added for time-limit, account and partitions as CWL extensions
• Fixes for MPI
• Jinja file no longer required
• Merge submit and start commands
• Improved usability of 'beecfg new'
• Combined gdbs
• Add restart code to beeflow
• Checkpoint restart fix
• Allow Absolute/Relative Paths for Main CWL and YAML Files
• Minimum version of Charliecloud required is now 0.32

Full Changelog: 0.1.3...0.1.4

BEE release 0.1.3

BEE now accepts stdout and stderr CWL specifications to direct those outputs for each task.