Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support nested subworkflows. #4822

Closed
wants to merge 2 commits into from
Closed
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 38 additions & 0 deletions cylc/flow/etc/job.sh
Original file line number Diff line number Diff line change
Expand Up @@ -356,3 +356,41 @@ cylc__job__dummy_result() {
return 1
fi
}

cylc__job__subworkflow() {
# Create and play a new instance of a subworkflow, from its installed
# source in the main workflow run dir.
local NAME="$1"
local DONE="$2" # last task instance in sub-workflow, e.g. "1/foo"
if [[ "$NAME" != "sub-"* ]]; then
>&2 echo "ERROR: subworkflow name must begin with 'sub-'"
exit 1
fi
if [[ "$DONE" != *"/"* ]]; then
>&2 echo "ERROR: subworkflow final task must be 'point/name'"
exit 1
fi
# Set subworkflow paths and ID (includes main cycle point):
local SRC_DIR="${CYLC_WORKFLOW_RUN_DIR}/${NAME}"
local RUN_DIR="${SRC_DIR}/${CYLC_TASK_CYCLE_POINT}"
local ID="${RUN_DIR#*/cylc-run/}"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CYLC_WORKFLOW_ID?

mkdir "$RUN_DIR"

# Symlink to the installed flow.cylc after renaming it to avoid
# detection of the template as a workflow (if not already renamed).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this still needed? Otherwise could we build in sub- awareness to the check causing the issue?

mv "${SRC_DIR}/flow.cylc" "${SRC_DIR}/sub-flow.cylc" || true
ln -s "${SRC_DIR}/sub-flow.cylc" "${RUN_DIR}/flow.cylc"

cylc message "Subworkflow: installed $ID from $SRC_DIR"

# Start the new sub-workflow instance (detaching).
cylc play "$ID"

# Wait for the subworkflow final task (allows subworkflow to stop and restart).
cylc message "Subworkflow: waiting for ${ID}//${DONE}"
cylc workflow-state \
--max-polls=10 --interval=10 \
-p "${DONE%/*}" -t "${DONE#*/}" --status succeeded "$ID"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure how this will behave with "failed" and "submit-failed" outcomes.


cylc message "Subworkflow: finished ${ID}//${DONE}"
}