Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add !list and !dict UW YAML tags #676

Closed
3 tasks
maddenp-noaa opened this issue Dec 12, 2024 · 0 comments · Fixed by #682
Closed
3 tasks

Add !list and !dict UW YAML tags #676

maddenp-noaa opened this issue Dec 12, 2024 · 0 comments · Fixed by #682
Assignees

Comments

@maddenp-noaa
Copy link
Collaborator

maddenp-noaa commented Dec 12, 2024

Description

Add support for !list and !dict UW YAML tags to convert quoted values, e.g. Jinja2 expressions, into Python list and dict objects. Among other use cases, this may help avoid the use of wildcards in file paths in e.g. fs mode configs, while keeping them DRY.

For example config.yaml

as: !list "[{% for n in range(3) %} a{{ n }},{% endfor %} ]"
bs: !dict "{{ '{' }}{% for n in range(3) %} b{{ n }}: {{ n }},{% endfor %}{{ '}' }}"
cs: !dict "[{% for n in range(3) %} ['c{{ n }}',{{ n }}],{% endfor %} ]"

would currently be rendered as

$ uw template render --input-file config.yaml
as: !list "[ a0, a1, a2, ]"
bs: !dict "{ b0: 0, b1: 1, b2: 2,}"
cs: !dict "[ ['c0',0], ['c1',1], ['c2',2], ]"

Recall that, since config realize renders values in key-value configs on their own, this is a realistic view of a potential intermediate state of the config during dereferencing. After implementation of the new tags, these values would then be converted to primitive Python values like

>>> list(yaml.safe_load("[ a0, a1, a2, ]"))
['a0', 'a1', 'a2']
>>> dict(yaml.safe_load("{ b0: 0, b1: 1, b2: 2,}"))
{'b0': 0, 'b1': 1, 'b2': 2}
>>> dict(yaml.safe_load("[ ['c0',0], ['c1',1], ['c2',2], ]"))
{'c0': 0, 'c1': 1, 'c2': 2}

Note that the final values of bs and cs are equivalent, but the list-of-lists representation in the original Jinja2 expression was more concise, due to not having to escape braces.

AC

  • Implement tags.
  • Add/update unit tests.
  • Update documentation.
@maddenp-noaa maddenp-noaa self-assigned this Dec 12, 2024
@maddenp-noaa maddenp-noaa converted this from a draft issue Dec 12, 2024
@elcarpenterNOAA elcarpenterNOAA moved this from Todo to In Progress in Unified Workflow Kanban Dec 20, 2024
@elcarpenterNOAA elcarpenterNOAA moved this from In Progress to Peer Review in Unified Workflow Kanban Dec 27, 2024
@github-project-automation github-project-automation bot moved this from Peer Review to Done in Unified Workflow Kanban Jan 7, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Development

Successfully merging a pull request may close this issue.

2 participants