Add ability to visualize DAG at CLI

[alexanderdean]

We should be able to:

• Visualize a DAG at the CLI when editing a DAG
• The failure output should visualize the DAG
• The success output should be able to optionally visualize the DAG, maybe with a --verbose setting

[alexanderdean]

Super important for @ihortom

[ninjabear]

It's the ASCII art part that's hard here (using backslashes or pipes for edges etc). This is mainly because of complications such as font width etc (th is shorter than W etc).

Printing a topologically sorted version of the file like:

                                      A
                                B          C
                                D          E

would probably complete the use-case, and be a bit less complicated to implement?

[alexanderdean]

I think here, let's not get stuck on the ASCII Art aspect. If we have to pipe the output from Factotum into another tool like dot2png, so be it.

[ninjabear]

This could also be like:

A -> B
C -> D

The important thing here is to see which tasks have dependencies set up - so you can catch it if one is missed.

[ninjabear]

We could store these images with the Factfile - as we do with makefiles currently

[ninjabear]

This is completed in 2536f98 which introduces a new command factotum dot job.factfile to generate a graphviz dot file representation of the DAG (which can then be plugged into a standard toolchain)

[alexanderdean]

Super cool!

[ihortom]

@ninjabear - below is the output I got when running one of the factiles:

    "check-lock"
    "emr-etl-runner"
    "storage-loader"
    "run-dedupe-sql"
    "check-lock" -> "emr-etl-runner"
    "emr-etl-runner" -> "storage-loader"
    "storage-loader" -> "run-dedupe-sql"

I thought the DOT syntax would be rather like

    1 [label="check-lock"]
    2 [label="emr-etl-runner"]
    3 [label="storage-loader"]
    4 [label="run-dedupe-sql"]
    1 -> 2
    2 -> 3
    3 -> 4

Otherwise, I see no point in the top section (extra noise)

    "check-lock"
    "emr-etl-runner"
    "storage-loader"
    "run-dedupe-sql"

Just the below will suffice

    "check-lock" -> "emr-etl-runner"
    "emr-etl-runner" -> "storage-loader"
    "storage-loader" -> "run-dedupe-sql"

[ninjabear]

Hey @ihortom, are you using a tool to read these? Check out http://www.webgraphviz.com/ for an example, there's also a command line utility called dot which can generate images of the DAGs.

[ihortom]

@ninjabear - ah, great! Thanks for that.