utils: fix total job count to exclude restored jobs from workspace

Amend the total number of jobs in a workflow so that when Snakemake detects that one or more jobs can be restored from the workspace (same workflow caching) they are not considered in the total count, reflecting Snakemake's local behaviour. Closes #62.
reanahub · Sep 19, 2023 · bc9cfee · bc9cfee
1 parent 3a8f770
commit bc9cfee
Show file tree

Hide file tree

Showing 2 changed files with 4 additions and 1 deletion.
diff --git a/CHANGES.rst b/CHANGES.rst
@@ -4,6 +4,7 @@ Changes
 Version 0.9.1 (UNRELEASED)
 --------------------------
 
+- Fixes the total job count reported to workflow controller by excluding jobs restored from the workspace and not executed by Snakemake.
 - Fixes an issue where some workflows would be stuck waiting for already-finished jobs.
 
 Version 0.9.0 (2023-01-19)

diff --git a/reana_workflow_engine_snakemake/utils.py b/reana_workflow_engine_snakemake/utils.py
@@ -17,7 +17,9 @@ def publish_workflow_start(
     workflow_uuid: str, publisher: WorkflowStatusPublisher, job: Job
 ):
     """Publish to MQ the start of the workflow."""
-    job_count = len([rule for rule in job.dag.rules if not rule.norun])
+    job_count = len(
+        [j for j in (job.dag._needrun | job.dag._finished) if not j.rule.norun]
+    )
     total_jobs = {"total": job_count, "job_ids": []}
     status_running = 1
     publisher.publish_workflow_status(