Run multiple jobs after each other

[sedrik]

Hi,

Thanks for such a great project!

I would love to always run the tests no matter if clippy or check finds any errors. I believe that in #42 they talk about something similar (always seeing the output of the test job).

Motivation: For me it is not uncommon to be working on something but still have some clippy/check issues (usually unused functions, especially when writing tests). It would be nice if I could instruct bacon to always run the tests even though I get some warnings from the other checks.

What I tried: I tried chaining the jobs together using the on_success = "job:clippy" syntax on my test definition but then when I make a change the tests are not re-run as bacon is now on the clippy job. As I might have a warning in clippy I cant add a on_success = "back" as it is not seen as a success. I did try setting allow_warnings = true and then back on success but it did not seem to work.

Being able to run both clippy/check and tests and show the output of both (or just the tests pass! header) always would be a nice feature.

Here is an example of what I have tried.

[jobs.clippy]
command = [
    "cargo", "clippy",
    "--all-targets",
    "--color", "always",
]
need_stdout = false
allow_warnings = true
on_success = "back"

[jobs.test]
command = [
    "cargo", "test", "--color", "always",
    "--", "--color", "always", # see https://github.com/Canop/bacon/issues/124
]
need_stdout = true
on_success = "job:clippy"

[leeola]

I came here looking for the exact same idea. One additional thing i'd like to add is the ability to make a target sticky. Ie lets say i want to run three jobs:

1. tests
2. check
3. clippy

If i cascade those three, it might be a fair bit of computation. Lets then say clippy finds an issue, so we go to fix it - it would be nice if the failing step (3 in this case) could be marked as sticky. Such that when we make a change on the FS it re-runs the previously failing step and only goes back to the full rotation after that issue is resolved.

Without this "sticky" feature, it would mean subsequent steps become more and more expensive to iterate on if an issue is found with them, as previous successful steps always have to run. Perhaps there are better ideas to this problem, but i imagine it would be an issue with multiple-jobs if we cannot make latter jobs sticky in some fashion. Ideally without intervention, optionally.

[sedrik]

Hmm, this stickiness concept is not something I would want. It seems that there are many workflows that people are looking to achieve :)

I have settled on running 2 instances of bacon for now, one that runs clippy and one that runs tests and this covers most things for me. Ideally I would not have to run two instances but still be able to always run clippy and tests (and see the output of both) but this will do for me.

[leeola]

Out of curiosity, how did you plan on handling the lag/delay that each additional step introduces? Ie are you not concerned about needing to run the Nth step repeatedly to rerun tests/etc?

[sedrik]

Not really no, my concern is that if I don't automatically re-run tests on each save I will miss a change that breaks them and then spend more of my time finding the error rather than spending the computers resources to make sure I don't mess up.

The thing I tried in my initial message was just an attempt at getting it to work :)

[Qix-]

From the screenshots I actually expected things to be TUI tabs at the top that allowed jobs to run in tandem.

What would be great is to have jobs run in parallel and then as the project updates they get re-run and the tabs have badges that show that an error or warning or whatever was found.

I'm doing kernel development so there are multiple ways to compile the same project, defined as aliases in .cargo/config.toml, and I typically have a shell script that runs them all one after another. Having Bacon run them all in parallel and showing which failed and which succeeded would be a HUGE time saver.

[teor2345]

From the screenshots I actually expected things to be TUI tabs at the top that allowed jobs to run in tandem.

I was expecting this as well!

[c-git]

Having been using bacon for a while in vscode, I've come to realize sometimes the solution is just to ignore the overhead and run multiple instances of bacon. For example I want a full screen (on the other monitor) to show the whole error message but in vscode when I click on the links to the code in that one it sometimes open on the other screen. So.... I just run it again in a small docked terminal in vscode and click on the link there. Running it twice probably isn't great but it's fast enough that tbh it's fine.

[net]

In my case I need to run wasm-pack build then pnpm jest to run Jest tests against the compiled wasm from wasm-pack build. I can do this with a script, but it would be nice if I could set this up directly in my bacon.toml.

[Canop]

@net Can't you define a pnpm-jest job, and a wasm-pack job having on_success = "job:pnpm-jest" ?

[net]

@Canop yes, but then the wasm-pack job doesn't re-run the second time files change.

[net]

If there was the concept of a root job, which would be the job that re-runs always start from, then that would work nicely.

[net]

Even better would be if this accounted for different watch configs, so

[jobs.wasm-pack]
command = [ "wasm-pack", "build"]
need_stdout = true
on_success = "job:jest"

[jobs.jest]
command = [ "pnpm", "jest"]
need_stdout = true
watch = [ "js" ]

would re-run just jest if file change was in js/, and would re-run starting from wasm-pack if the file change was in src/.

---

Edit:

Maybe supporting jobs that are sequences of other jobs would be a better way to support this than on_success. This way jobs can be re-used.

[jobs.test]
command = [ "cargo", "test", "--color", "always", "--", "--color", "always" ]
need_stdout = true

[jobs.wasm-pack]
command = [ "wasm-pack", "build"]
need_stdout = true

[jobs.jest]
command = [ "pnpm", "jest"]
need_stdout = true
watch = [ "js" ]

[jobs.test-all]
actions = [ "job:test", "job:wasm-pack", "job.jest" ]

Then the build-test job would only advance to the next action if the one before was successful, and would re-run starting at the earliest job that is watching the changed file.