clarity about output

[Lewiscowles1986]

Hello, thanks for producing and sharing this very cool take on a declarative workflow engine

While searching for answers about passing output between tasks, I came across

https://github.com/dagu-org/dagu/blob/main/examples/pass_output_to_next.yaml

Does this mean that STDOUT is what is captured, similar to running a command within VARNAME=$(/some/command args) for bash / zsh?

https://dagu.readthedocs.io/en/latest/schema.html#output

A variable name to store the command’s STDOUT contents. You can reuse this variable in subsequent steps.

Seems to suggest it is

If that is the case, do you have tips and tricks to remove noise from cli tools when using dagu?

Do you think this could make a useful addition to the user quick-start guide https://dagu.readthedocs.io/en/latest/quickstart.html

I was thinking that in the docs I didn't see output called out, besides the reference docs and example.

Last thing. Do you think that the DSL / language might ever seek to refine the output, by adding additional filtering or parsing as part of the process? I understand I could chain steps to achieve this now, with each step transforming output.

[yohamta]

Hey @Lewiscowles1986 👋, thank you so much for your interest in Dagu!

Does this mean that STDOUT is what is captured, similar to running a command within VARNAME=$(/some/command args) for bash / zsh?

Yes, that's correct.

If that is the case, do you have tips and tricks to remove noise from cli tools when using dagu?
Do you think that the DSL / language might ever seek to refine the output, by adding additional filtering or parsing as part of the process?

Currently, Dagu only capture stdout. If the captured output is JSON, you can reference specific object fields (e.g., echo ${OUT.someField}). And yes, that's a very interesting idea. If you have any suggestions for additional tool that could be handy, I'd love to hear them.

Do you think this could make a useful addition to the user quick-start guide?

Yes, that's a great idea. It's a very common use case, so adding it to the quick start guide makes a lot of sense.

[Lewiscowles1986]

I've added an example of something:

• not a network resource
• simple-enough for quick start that people can ignore if it doesn't speak to them
• includes messy output
• cleans it up
• links to examples (where hopefully more examples come)

I tested using the docs locally, under python 3.11.

Please let me know if you'd like any changes

[Lewiscowles1986]

[Lewiscowles1986]

One thing that could be useful in a quick-start guide might even be notes about running the docker-compose.yml. It certainly simplifies things for someone learning about this software

[yohamta]

Yes, that's a great idea! Should we add a docker-compose.yaml file to the root of the project so users can easily clone the repository and try it out, or would it be enough to include just a snippet in the README?

[Lewiscowles1986]

So I think maybe we point out that examples has a "ready-to-go" docker-compose.yaml?

Esentially there are competing trade-offs

1. Folks running this are likely to start small, and expand. Docker compose is good for the starting, but maybe not the expanding?
2. It may dissuade curiosity, or color the perception of the project. I don't know this, but it can be a concern of how turn-key things are. I've heard a bit of learning helps stickiness, so long as it's not insurmountable, needing to get to quickstart, or install and then seeing explicit instructions could maybe be better. Of course that is if we accept this proposed world-view.

I'm happy to follow up with that if you like.