command line interface for pangeo-forge: here or in pangeo-smithy

[rabernat]

For submitting recipes to bakeries from scripts, github workflows, etc., it will be useful to have a command line interface (CLI) for pangeo-forge. The CLI would work something like this:

$ pangeo smithy recipe validate recipe.py
- foo_recipe [ok]
- bar_recipe [ok]
$ pangeo smithy recipe submit recipe.py
- foo_recipe [submitted] http://status_url/foo/123
- bar_recipe [submitted] http://status_url/bar/456

Some questions:

• should this live here in or https://github.com/rabernat/pangeo-smithy ?
• what's the best way to run python code from a file?
• how to we pass configuration like prefect secrets?

[rabernat]

Worth nothing that @TomAugspurger had a CLI already in the earlier version, which I removed in my big refactor:

https://github.com/pangeo-forge/pangeo-forge/blob/00306b38addaafa475c9f932c8eec58d4ffa047a/pangeo_forge/cli.py

[charlesbluca]

From my perspective, the CLI for running/validating recipes should belong in this repo, as this is where the majority of Recipe, Pipeline, etc. implementation will be hosted. If I'm understanding the function of pangeo-smithy correctly, a CLI for that project should handle:

• Generating a git repo from a YAML config file
• Notifying a user what environment variables need to be configured (authentication, other CLI settings, etc.)
• Reporting warnings or errors to help a user debug problems before/during making a PR to staged-recipes

I also feel like the processes of running recipes locally and submitting them to a bakery are distinct enough that they could be separate CLI tools, but I recognize why users would probably want to do both of those tasks within the context of this one project. In summary:

• pangeo-forge - running and validation of recipes locally
• pangeo-smithy - generation of recipes and validation of YAML configs
• pangeo-bakery - submission of recipes to bakeries (optional and could probably go in pangeo-forge)

[TomAugspurger]

I’m also +1 for just developing as much as possible in a single repo, even if we register separate console scripts for the different users.

…

On Jan 25, 2021, at 2:17 PM, Charles Blackmon-Luca ***@***.***> wrote: From my perspective, the CLI for running/validating recipes should belong in this repo, as this is where the majority of Recipe, Pipeline, etc. implementation will be hosted. If I'm understanding the function of pangeo-smithy correctly, a CLI for that project should handle: Generating a git repo from a YAML config file Notifying a user what environment variables need to be configured (authentication, other CLI settings, etc.) Reporting warnings or errors to help a user debug problems before/during making a PR to staged-recipes I also feel like the processes of running recipes locally and submitting them to a bakery are distinct enough that they could be separate CLI tools, but I recognize why users would probably want to do both of those tasks within the context of this one project. In summary: pangeo-forge - running and validation of recipes locally pangeo-smithy - generation of recipes and validation of YAML configs pangeo-bakery - submission of recipes to bakeries — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#43 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAKAOIUUUVD2H3DZT3U5273S3XGUFANCNFSM4WPH7PYA>.

[cisaacstern]

This now exists in https://github.com/pangeo-forge/pangeo-forge-runner, with ongoing discussion of modularization vs. monorepo ongoing in #514, so closing this. Thanks all!