From c42abb639ac20c28fbf922eccd6f639c37fdf7fe Mon Sep 17 00:00:00 2001 From: Benjamin Bay <48391872+ben-bay@users.noreply.github.com> Date: Tue, 4 Aug 2020 11:59:15 -0600 Subject: [PATCH] 1.7.2 (#264) * locked maestro version * incremented to 1.5.3 * Update CHANGELOG.md * Feature/koning/vhost no leading slash (#217) * The broker name can now be amqps (with ssl) or amqp (without ssl). The default rabbitmq vhost is now instead of /. * Undo fix typos. * Fix spacing. * Add amqps conn for self doc. * Add inital running workers check to monitor. This should eliminate any (#218) race conditions. * updated timeout (#223) * updated default task timeout * Added to CHANGELOG * Feature/koning/create encrypt key init (#219) * Add init_key to create the encryption key when merlin config is run. * Move import to point of use for encryption. * Feature/koning/monitor upgrade (#221) * Fix sleep variable. * Change another sleep variable. * Change brackets. * Fix clarity for jobs but no consumers condition. * Fix comment. * Update comment. * Move check_merlin_status to router. * Run fix-style. * Bugfix/ben/expand name (#220) * working on bugfix * achieved desired behavior * made expansion anf filewriting conditional * fixed style * updated CHANGELOG * fixed samples and restart * fixed style * used variable * added name expansion test * temporarily commented out one test * fixed style * added debug block * changes * corrected bug * updated example; scripts are now copied into merlin_info * Feature/ben/expose visibility timeout (#224) * exposed visibility timeout in config file * added visibility timeouts to default config files * fixed style * added seconds to clarify name * fixed attribute name * locked celery version at 4.4.2 * updated CHANGELOG * v1.5.4 * adjusted CHANGELOG * adjusted CHANGELOG * Feature/koning/info conn timeout (#226) * Add a timeout check for the merlin info kombu connection test. The redis server will not comply with the connection_timeout config. * Update docs. * Load default timeout if non in config file (#229) * 1.6.1 * bugfix (#232) * bugfix * removed repetition in logic * Log sample generation to merlin_info (#234) * sample generation is now logged to merlin_info * Added to CHANGELOG * fixed style * Maestro 1.1.8+ update (#233) * working on adding maestro's schema validation to merlin * updated schemas * adding merlin_schema to spec logic * added debug prints * added print * fixing vestigial Maestro calls * adjusted merlin_schemas * fixed json file * validation progress * added from_dict replacement logic * adjusted maestrowf version req * fixed string * schema improvements * merlinsection schema updates * merlin section error parsing appears to work * updated CHANGELOG * fixed merge error * Added walltime to json schema * Add the bank and walltime keywords to the batch slurm launch, these (#236) will not alter the lsf launch. * Bugfix for new celery versions (#231) * Change the expand_tasks_with_samples task to use an immutable signature, no return code is neccessary. * Change only expand_tasks_with_samples to si() * Increase celery version requirement to the latest (4.4.5). * Update CHANGELOG. Co-authored-by: Benjamin Bay <48391872+ben-bay@users.noreply.github.com> * hidden test specs (#237) * added tests to run_tests.py * updated merlinsection schema * added full_spec test * added to CHANGELOG * merlinspec change * use regex to find variable tokens (#239) * now using regex to find variable tokens * added shell ref function * added to CHANGELOG * Revert maestro 1.1.8 support (#241) * reverted support of Maestro 1.1.8 * removed tests that use maestro validation * fixed style * fixed cli tests, reactivated 3 old ones (#242) * fixed cli tests, reactivated 3 old ones * added to CHANGELOG * More provenance specs + logic improvements (#240) * added orig.yaml and expanded.yaml * added partial provenance spec * fixed style * added to CHANGELOG * removed comments * made env replacement more precise * fixed restart bug * changed provenance file names * logic update * fixed style * expansion now working * added cli test logic to test all 3 provenance file types * updated CHANGELOG * expansion sans io (#243) * reverted support of Maestro 1.1.8 * removed tests that use maestro validation * fixed style * added sections property to spec * reducing number of file writes and reads * provenance specs are correct * fixed style * removed unused imports * cleaning up logic * fixed style * final logic improvements * removed unused property * updated CHANGELOG * v1.6.2 * debugging study * added provenance tests * fixed path in test * added check for workflow name expanding to invalid filename * Docs/ben/level max dirs (#250) * added to documentation * added to CHANGELOG * added default * updated docs to reflect recent provenance spec update (#251) * Feature/ben/pgen (#248) * added basic pgen support * added to CHANGELOG * pgen, pargs, and env appear to work * added to CHANGELOG * fixed typo * added pgen cli test * added faq entry on pgen * Update Dockerfile (#254) * Update CHANGELOG.md * pretty yaml (#252) * starting * building pretty_dump function * updates * fixed style * tweaks * valid yaml * added yaml_sections * added to CHANGELOG * removed old yaml representer code * added cli test for equality of provenance specs * fixed pgen test * disabled test while not working * fixed yaml lists for source, paths, git * tweaks / fixes * env variable bugfix (#247) * removed expansion of env variables in provenance specs * added to CHANGELOG * corrected unit tests * added todos * working on expand_env_var function * fixed type bug * removed comments, prints, fixed style * added to CHANGELOG * tweaks * fixed output path * allowed study name to use env vars * fixed style * added to CHANGELOG, added support for 'restart' * Update CHANGELOG.md * added docs strings * added to docs * updated comment * fixed CHANGELOG * removed debug print * flux examples fix (#253) * Update flux example workflows to use new kvs query methods. * Update CHANGELOG with fix info. * Add coker fix comment. * Update CHANGELOG.md * added gitlab ci file * improved gitlab ci setup * renamed file * hotfix to allow distributed cli tests to run * redis update (#255) * Update redis to new version with TLS support. * Add changelog. * Bugfix for rediss cert_reqs keyword. (#256) * expose celery (#245) * added default app.yaml, still in progress... * added celery defaults * Update app.yaml * config override appears to be working * corrected value name * shifted some properties to be optional defaults * removed unneeded file * improvements * fixed style * fixed style * hid 'no override' message * tweaked var names * added 2 tests for celery app (#246) * fixed test * fixed workers bug * fixed unit tests for running offline * removed 2 unit tests * fixed CHANGELOG * added info print of all celery configurations being overridden * 1.7.0 * 1.7.0 * copy samplesfile to merlin_info (#259) * copy samplesfile to merlin_info * added to CHANGELOG * Bugfix/koning/process exceptions (#260) * Add bugfix log. * Add an exception catcher for the multiprocessing Process class. * fixed style * 1.7.1 * hotfix * fix * 1.7.2 Co-authored-by: Joe Koning Co-authored-by: Luc Peterson Co-authored-by: fixdocker <68359629+fixdocker@users.noreply.github.com> --- Makefile | 2 +- merlin/__init__.py | 4 ++-- merlin/ascii_art.py | 2 +- merlin/celery.py | 2 +- merlin/common/__init__.py | 2 +- merlin/common/abstracts/__init__.py | 2 +- merlin/common/abstracts/enums/__init__.py | 2 +- merlin/common/openfilelist.py | 2 +- merlin/common/opennpylib.py | 2 +- merlin/common/sample_index.py | 2 +- merlin/common/sample_index_factory.py | 2 +- merlin/common/security/__init__.py | 2 +- merlin/common/security/encrypt.py | 2 +- merlin/common/security/encrypt_backend_traffic.py | 2 +- merlin/common/tasks.py | 2 +- merlin/common/util_sampling.py | 2 +- merlin/config/__init__.py | 2 +- merlin/config/broker.py | 2 +- merlin/config/configfile.py | 2 +- merlin/config/results_backend.py | 2 +- merlin/data/celery/__init__.py | 2 +- merlin/display.py | 2 +- merlin/examples/__init__.py | 2 +- merlin/examples/examples.py | 2 +- merlin/examples/generator.py | 2 +- merlin/exceptions/__init__.py | 2 +- merlin/log_formatter.py | 2 +- merlin/main.py | 2 +- merlin/merlin_templates.py | 2 +- merlin/router.py | 2 +- merlin/spec/__init__.py | 2 +- merlin/spec/all_keys.py | 2 +- merlin/spec/defaults.py | 2 +- merlin/spec/expansion.py | 8 ++++++-- merlin/spec/specification.py | 2 +- merlin/study/__init__.py | 2 +- merlin/study/batch.py | 2 +- merlin/study/celeryadapter.py | 2 +- merlin/study/dag.py | 2 +- merlin/study/script_adapter.py | 2 +- merlin/study/step.py | 2 +- merlin/study/study.py | 11 ++++++++++- merlin/utils.py | 2 +- setup.py | 2 +- tests/integration/run_tests.py | 2 +- 45 files changed, 60 insertions(+), 47 deletions(-) diff --git a/Makefile b/Makefile index 31ef22647..27b3af359 100644 --- a/Makefile +++ b/Makefile @@ -6,7 +6,7 @@ # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.7.1. +# This file is part of Merlin, Version: 1.7.2. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/__init__.py b/merlin/__init__.py index 08e49456a..ea5c42e1f 100644 --- a/merlin/__init__.py +++ b/merlin/__init__.py @@ -6,7 +6,7 @@ # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.7.1. +# This file is part of Merlin, Version: 1.7.2. # # For details, see https://github.com/LLNL/merlin. # @@ -38,7 +38,7 @@ import sys -__version__ = "1.7.1" +__version__ = "1.7.2" VERSION = __version__ PATH_TO_PROJ = os.path.join(os.path.dirname(__file__), "") diff --git a/merlin/ascii_art.py b/merlin/ascii_art.py index 43c58d927..eadce2997 100644 --- a/merlin/ascii_art.py +++ b/merlin/ascii_art.py @@ -6,7 +6,7 @@ # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.7.1. +# This file is part of Merlin, Version: 1.7.2. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/celery.py b/merlin/celery.py index 21d6813f1..c1d4325b3 100644 --- a/merlin/celery.py +++ b/merlin/celery.py @@ -6,7 +6,7 @@ # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.7.1. +# This file is part of Merlin, Version: 1.7.2. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/common/__init__.py b/merlin/common/__init__.py index e55a9b5da..36f9a9979 100644 --- a/merlin/common/__init__.py +++ b/merlin/common/__init__.py @@ -6,7 +6,7 @@ # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.7.1. +# This file is part of Merlin, Version: 1.7.2. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/common/abstracts/__init__.py b/merlin/common/abstracts/__init__.py index e55a9b5da..36f9a9979 100644 --- a/merlin/common/abstracts/__init__.py +++ b/merlin/common/abstracts/__init__.py @@ -6,7 +6,7 @@ # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.7.1. +# This file is part of Merlin, Version: 1.7.2. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/common/abstracts/enums/__init__.py b/merlin/common/abstracts/enums/__init__.py index a4c5cba7b..7182fc78e 100644 --- a/merlin/common/abstracts/enums/__init__.py +++ b/merlin/common/abstracts/enums/__init__.py @@ -6,7 +6,7 @@ # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.7.1. +# This file is part of Merlin, Version: 1.7.2. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/common/openfilelist.py b/merlin/common/openfilelist.py index 2dcf54137..07ce27365 100644 --- a/merlin/common/openfilelist.py +++ b/merlin/common/openfilelist.py @@ -6,7 +6,7 @@ # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.7.1. +# This file is part of Merlin, Version: 1.7.2. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/common/opennpylib.py b/merlin/common/opennpylib.py index f9ccce90b..eac129a2f 100644 --- a/merlin/common/opennpylib.py +++ b/merlin/common/opennpylib.py @@ -6,7 +6,7 @@ # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.7.1. +# This file is part of Merlin, Version: 1.7.2. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/common/sample_index.py b/merlin/common/sample_index.py index 96ab3d975..52a738f45 100644 --- a/merlin/common/sample_index.py +++ b/merlin/common/sample_index.py @@ -6,7 +6,7 @@ # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.7.1. +# This file is part of Merlin, Version: 1.7.2. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/common/sample_index_factory.py b/merlin/common/sample_index_factory.py index 6ebaf3494..3f6f0b46f 100644 --- a/merlin/common/sample_index_factory.py +++ b/merlin/common/sample_index_factory.py @@ -6,7 +6,7 @@ # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.7.1. +# This file is part of Merlin, Version: 1.7.2. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/common/security/__init__.py b/merlin/common/security/__init__.py index e55a9b5da..36f9a9979 100644 --- a/merlin/common/security/__init__.py +++ b/merlin/common/security/__init__.py @@ -6,7 +6,7 @@ # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.7.1. +# This file is part of Merlin, Version: 1.7.2. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/common/security/encrypt.py b/merlin/common/security/encrypt.py index ac45452a3..00aa807b4 100644 --- a/merlin/common/security/encrypt.py +++ b/merlin/common/security/encrypt.py @@ -6,7 +6,7 @@ # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.7.1. +# This file is part of Merlin, Version: 1.7.2. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/common/security/encrypt_backend_traffic.py b/merlin/common/security/encrypt_backend_traffic.py index fbb040242..f289f2006 100644 --- a/merlin/common/security/encrypt_backend_traffic.py +++ b/merlin/common/security/encrypt_backend_traffic.py @@ -6,7 +6,7 @@ # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.7.1. +# This file is part of Merlin, Version: 1.7.2. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/common/tasks.py b/merlin/common/tasks.py index 8a56f153a..880378ca3 100644 --- a/merlin/common/tasks.py +++ b/merlin/common/tasks.py @@ -6,7 +6,7 @@ # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.7.1. +# This file is part of Merlin, Version: 1.7.2. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/common/util_sampling.py b/merlin/common/util_sampling.py index e5a600675..48c886e5d 100644 --- a/merlin/common/util_sampling.py +++ b/merlin/common/util_sampling.py @@ -6,7 +6,7 @@ # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.7.1. +# This file is part of Merlin, Version: 1.7.2. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/config/__init__.py b/merlin/config/__init__.py index 799cc9420..a380994cc 100644 --- a/merlin/config/__init__.py +++ b/merlin/config/__init__.py @@ -6,7 +6,7 @@ # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.7.1. +# This file is part of Merlin, Version: 1.7.2. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/config/broker.py b/merlin/config/broker.py index 963eeb1fa..6ce2a769b 100644 --- a/merlin/config/broker.py +++ b/merlin/config/broker.py @@ -6,7 +6,7 @@ # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.7.1. +# This file is part of Merlin, Version: 1.7.2. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/config/configfile.py b/merlin/config/configfile.py index 358116f74..a5f13caf3 100644 --- a/merlin/config/configfile.py +++ b/merlin/config/configfile.py @@ -6,7 +6,7 @@ # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.7.1. +# This file is part of Merlin, Version: 1.7.2. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/config/results_backend.py b/merlin/config/results_backend.py index 7fa4ec6c1..03cf0f381 100644 --- a/merlin/config/results_backend.py +++ b/merlin/config/results_backend.py @@ -6,7 +6,7 @@ # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.7.1. +# This file is part of Merlin, Version: 1.7.2. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/data/celery/__init__.py b/merlin/data/celery/__init__.py index e55a9b5da..36f9a9979 100644 --- a/merlin/data/celery/__init__.py +++ b/merlin/data/celery/__init__.py @@ -6,7 +6,7 @@ # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.7.1. +# This file is part of Merlin, Version: 1.7.2. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/display.py b/merlin/display.py index 3d2b6cd14..b63949f7e 100644 --- a/merlin/display.py +++ b/merlin/display.py @@ -6,7 +6,7 @@ # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.7.1. +# This file is part of Merlin, Version: 1.7.2. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/examples/__init__.py b/merlin/examples/__init__.py index e55a9b5da..36f9a9979 100644 --- a/merlin/examples/__init__.py +++ b/merlin/examples/__init__.py @@ -6,7 +6,7 @@ # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.7.1. +# This file is part of Merlin, Version: 1.7.2. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/examples/examples.py b/merlin/examples/examples.py index a12452734..0b8dbe792 100644 --- a/merlin/examples/examples.py +++ b/merlin/examples/examples.py @@ -6,7 +6,7 @@ # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.7.1. +# This file is part of Merlin, Version: 1.7.2. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/examples/generator.py b/merlin/examples/generator.py index ac35ebb1b..5b71bc828 100644 --- a/merlin/examples/generator.py +++ b/merlin/examples/generator.py @@ -6,7 +6,7 @@ # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.7.1. +# This file is part of Merlin, Version: 1.7.2. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/exceptions/__init__.py b/merlin/exceptions/__init__.py index 9d0cbc8fe..62ebefe60 100644 --- a/merlin/exceptions/__init__.py +++ b/merlin/exceptions/__init__.py @@ -6,7 +6,7 @@ # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.7.1. +# This file is part of Merlin, Version: 1.7.2. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/log_formatter.py b/merlin/log_formatter.py index d4249a239..53d5b8718 100644 --- a/merlin/log_formatter.py +++ b/merlin/log_formatter.py @@ -6,7 +6,7 @@ # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.7.1. +# This file is part of Merlin, Version: 1.7.2. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/main.py b/merlin/main.py index f1cb1a3a2..52fe0e0dd 100644 --- a/merlin/main.py +++ b/merlin/main.py @@ -6,7 +6,7 @@ # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.7.1. +# This file is part of Merlin, Version: 1.7.2. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/merlin_templates.py b/merlin/merlin_templates.py index 51be84eac..a687c6688 100644 --- a/merlin/merlin_templates.py +++ b/merlin/merlin_templates.py @@ -6,7 +6,7 @@ # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.7.1. +# This file is part of Merlin, Version: 1.7.2. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/router.py b/merlin/router.py index 0e18a384a..265dac5df 100644 --- a/merlin/router.py +++ b/merlin/router.py @@ -6,7 +6,7 @@ # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.7.1. +# This file is part of Merlin, Version: 1.7.2. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/spec/__init__.py b/merlin/spec/__init__.py index e55a9b5da..36f9a9979 100644 --- a/merlin/spec/__init__.py +++ b/merlin/spec/__init__.py @@ -6,7 +6,7 @@ # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.7.1. +# This file is part of Merlin, Version: 1.7.2. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/spec/all_keys.py b/merlin/spec/all_keys.py index 54c68cb28..88d6bcf53 100644 --- a/merlin/spec/all_keys.py +++ b/merlin/spec/all_keys.py @@ -6,7 +6,7 @@ # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.7.1. +# This file is part of Merlin, Version: 1.7.2. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/spec/defaults.py b/merlin/spec/defaults.py index b083e2770..da757c167 100644 --- a/merlin/spec/defaults.py +++ b/merlin/spec/defaults.py @@ -6,7 +6,7 @@ # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.7.1. +# This file is part of Merlin, Version: 1.7.2. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/spec/expansion.py b/merlin/spec/expansion.py index cb3ae1534..965554221 100644 --- a/merlin/spec/expansion.py +++ b/merlin/spec/expansion.py @@ -6,7 +6,7 @@ # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.7.1. +# This file is part of Merlin, Version: 1.7.2. # # For details, see https://github.com/LLNL/merlin. # @@ -155,7 +155,11 @@ def determine_user_variables(*user_var_dicts): {'TARGET': 'target_dir', 'PATH': '$(SPECROOT)/target_dir'} """ - all_var_dicts = dict(ChainMap(*user_var_dicts)) + # TODO move this logic to specification.py? + try: + all_var_dicts = dict(ChainMap(*user_var_dicts)) + except TypeError: + all_var_dicts = {} determined_results = {} for key, val in all_var_dicts.items(): if key in RESERVED: diff --git a/merlin/spec/specification.py b/merlin/spec/specification.py index dd606ecc8..072bee2c2 100644 --- a/merlin/spec/specification.py +++ b/merlin/spec/specification.py @@ -6,7 +6,7 @@ # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.7.1. +# This file is part of Merlin, Version: 1.7.2. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/study/__init__.py b/merlin/study/__init__.py index e55a9b5da..36f9a9979 100644 --- a/merlin/study/__init__.py +++ b/merlin/study/__init__.py @@ -6,7 +6,7 @@ # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.7.1. +# This file is part of Merlin, Version: 1.7.2. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/study/batch.py b/merlin/study/batch.py index 9d93ad4a9..e8b81d54e 100644 --- a/merlin/study/batch.py +++ b/merlin/study/batch.py @@ -6,7 +6,7 @@ # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.7.1. +# This file is part of Merlin, Version: 1.7.2. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/study/celeryadapter.py b/merlin/study/celeryadapter.py index 2db38f70b..65ac5cce1 100644 --- a/merlin/study/celeryadapter.py +++ b/merlin/study/celeryadapter.py @@ -6,7 +6,7 @@ # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.7.1. +# This file is part of Merlin, Version: 1.7.2. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/study/dag.py b/merlin/study/dag.py index 40e2348f4..f9dbffd02 100644 --- a/merlin/study/dag.py +++ b/merlin/study/dag.py @@ -6,7 +6,7 @@ # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.7.1. +# This file is part of Merlin, Version: 1.7.2. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/study/script_adapter.py b/merlin/study/script_adapter.py index 753c13955..66a1a77b5 100644 --- a/merlin/study/script_adapter.py +++ b/merlin/study/script_adapter.py @@ -6,7 +6,7 @@ # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.7.1. +# This file is part of Merlin, Version: 1.7.2. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/study/step.py b/merlin/study/step.py index aa40620cd..8e1be2d2a 100644 --- a/merlin/study/step.py +++ b/merlin/study/step.py @@ -6,7 +6,7 @@ # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.7.1. +# This file is part of Merlin, Version: 1.7.2. # # For details, see https://github.com/LLNL/merlin. # diff --git a/merlin/study/study.py b/merlin/study/study.py index 9951d73f3..19bf5fbcd 100644 --- a/merlin/study/study.py +++ b/merlin/study/study.py @@ -6,7 +6,7 @@ # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.7.1. +# This file is part of Merlin, Version: 1.7.2. # # For details, see https://github.com/LLNL/merlin. # @@ -495,9 +495,18 @@ def load_dag(self): Generates a dag (a directed acyclic execution graph). Assigns it to `self.dag`. """ + # TODO move this logic to specification.py + for key in ["variables", "labels", "sources", "dependencies"]: + if key not in self.expanded_spec.environment: + continue + if self.expanded_spec.environment[key] is None: + self.expanded_spec.environment[key] = {} environment = self.expanded_spec.get_study_environment() steps = self.expanded_spec.get_study_steps() + # TODO move this logic to specification.py + if self.expanded_spec.globals is None: + self.expanded_spec.globals = {} parameters = self.expanded_spec.get_parameters() # Setup the study. diff --git a/merlin/utils.py b/merlin/utils.py index 0f8963e76..266339cc2 100644 --- a/merlin/utils.py +++ b/merlin/utils.py @@ -6,7 +6,7 @@ # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.7.1. +# This file is part of Merlin, Version: 1.7.2. # # For details, see https://github.com/LLNL/merlin. # diff --git a/setup.py b/setup.py index a5c27d539..c805da67e 100644 --- a/setup.py +++ b/setup.py @@ -6,7 +6,7 @@ # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.7.1. +# This file is part of Merlin, Version: 1.7.2. # # For details, see https://github.com/LLNL/merlin. # diff --git a/tests/integration/run_tests.py b/tests/integration/run_tests.py index 998a081b7..c261407d5 100644 --- a/tests/integration/run_tests.py +++ b/tests/integration/run_tests.py @@ -6,7 +6,7 @@ # # LLNL-CODE-797170 # All rights reserved. -# This file is part of Merlin, Version: 1.7.1. +# This file is part of Merlin, Version: 1.7.2. # # For details, see https://github.com/LLNL/merlin. #