Skip to content

Conversation

@nmdefries
Copy link
Contributor

@nmdefries nmdefries commented Jan 17, 2023

Description

Modify params dirs (for convenience of working with user-provided paths) only inside the docker container so that the original user-provided params file is preserved regardless of pipeline exit status or other unintended behavior.

Changelog

  • Makefile

Fixes

Previously modifications to dir paths were performed on the original params.json outside the docker container. Restoring it to its original state relied on a teardown step that didn't run if the pipeline exited with an error.

Instead of solving this by capturing and rethrowing errors, it was easier to limit params modifications to a copy of the mounted params.json inside the docker container. The copy is not linked to any file on the host, so changes to it are not propagated back to the original params.json.

I can't find any built-in support for mounting a file from the host but NOT propagating in-container changes back to the host, so params.json is mounted under another name and then copied manually.

@nmdefries nmdefries requested a review from krivard January 17, 2023 17:58
Copy link
Contributor

@krivard krivard left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice find! It's always great when you can drop a target without loss of functionality.

Suggestion optional; probably works either way.

Co-authored-by: Katie Mazaitis <krivard@cs.cmu.edu>
@nmdefries nmdefries marked this pull request as ready for review January 18, 2023 17:06
@nmdefries
Copy link
Contributor Author

@krivard This is ready to merge

@krivard krivard merged commit 4641604 into main Jan 18, 2023
@krivard krivard deleted the ndefries/backfill/restore-params-finally branch January 18, 2023 21:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants