[nix-local-build] new-run, new-test, new-bench, new-exec

[acfoltzer]

Summary by @ezyang. These should be easy to do, it's a copy-paste job of the existing run/test/bench code on top of invoking the new-build logic to build things.

---

These are very important quality-of-life features that make it impractical to use new-build for some of our larger projects. In general, we'd be happy with straightforward ports of the legacy commands, but a couple specific comments arose.

Avoiding unnecessary work

This is a lot more important when working with metapackages, since we don't want to have to build every test suite in the metapackage in order to build one particular test. The way the [COMPONENTS] argument currently works for new-build should be fine for this.

Easier argument passing for test and bench

cabal run [FLAGS] [EXECUTABLE] [-- EXECUTABLE_FLAGS] makes it easy to pass arguments to the executable by just putting them after the --. It would be nice to have the same for test and bench, rather than having to use --test-options or --benchmark-options flags.

[ezyang]

So, I guess, if you say cabal new-test test1 test2 -- --some-test-option this test option would be passed to both tests you are running?

[ezyang]

Note that however this gets implemented, make sure it works with --enable-coverage, c.f. #3682

[elliottt]

Would it be possible to add new-exec to this list? I prefer using cabal exec ghci to using cabal repl as it doesn't try to load any modules.

[dcoutts]

@elliottt on that topic, we should decide if that would be the better behaviour anyway and/or add a flag to repl to control it. I'm not sure if there's a ticket for it, but if not please open one and explain the advantages/disadvantages.

[elliottt]

@dcoutts a quick search turned up #2592 that does a fairly good job of framing the problem. My use case is similar: large projects with lots of modules make cabal repl unpleasant, as it doesn't allow me to start up the repl and then load only the modules I'm interested in.

[ezyang]

I spent a little time looking at how to implement new-run. There does seem to be some rejiggering needed:

1. We need a way of specifying a specific executable as the one we want to run. Not easily done because this is a callback from pre-build. Maybe we should do it repl style by marking the ElaboratedConfiguredPackage according to the command line (much how we mark the things we want to repl in a similar way.) But the way repl is run is bogus, see cabal new-repl target1 target2 shouldn't bring up GHCi prompt twice #3659, so maybe it should be done properly here. CC @dcoutts
2. Make sure you look at Distribution/Client/Run.hs, it's accumulated a bunch of edge case handling which should be preserved
3. Need a mode which assumes that the project is up to date and directly invokes the executable with as little overhead as possible. But do we still have to run the pre-build phase? It would definitely be convenient to do so, because that's the easiest way to read off the dist-dir. We could read out the cached value w/o checking if it's invalid? Or maybe we can figure out executable location from first principles?

[dcoutts]

@ezyang I don't understand why it's so difficult. Isn't it just a matter of doing what new-build does for an exe target, ie making sure it's up-to-date, and then executing it?

Sure we have to check that the target the user has given does correspond to an exe. Not so hard. Then that gives us an elaborated plan that's up to date for the exe, and we can read off from that where the exe lives in the file system, then we just run it.

If a "no rebuild" mode is needed, then we just make sure the plan is up to date, read off the location of the executable from the elaborated plan, checking the exe exists, and do the same invocation.

And yes, the logic for invoking needs to be ported from Run.hs.

What am I missing?

[dcoutts]

Ok, so the runProjectPreBuildPhase should return the resolved targets selected. That'd be most direct. I'm doing some refactoring in ProjectOrchestration so I can do this.

[phadej]

FWIW, there is bare bones new-test which simply builds and runs all tests matching build targets.

[acfoltzer]

#4304 has new-exec

[acfoltzer]

Per @dmwit, new-test is also now implemented in master

[hvr]

@dcoutts @ezyang @dmwit I notice with #4179 merged, there's now cabal new-run (courtesy of f2a8bdf), but as the commit message says:

So far they just build the exe bench and don't do the next steps, but they do at least select the right components to build.

So, this sounds like we're almost there? is the only thing left to do actually invoke the produced executable component now?

[dcoutts]

@hvr yes, new-run and new-bench CLI bits are in but they only build the things, they don't yet run.

[ezyang]

Let's track the new-run business at #4477

[23Skidoo]

So, I guess, if you say cabal new-test test1 test2 -- --some-test-option this test option would be passed to both tests you are running?

I should note that this syntax is ambiguous: does test2 name a target, or is it the beginning of the argument list for test1?

[fgaz]

Tracking the args issue at #4643

[hvr]

I should note that this syntax is ambiguous: does test2 name a target, or is it the beginning of the argument list for test1?

IMHO, everything left of -- is either a cabal-flag or an argument (i.e. the test-component names). While everything right of -- is to be treated as argument to the executed component (iirc this nicely works for +RTS too). This is how I interpret the original description of this ticket regarding easy parsing.

To keep things simple (and avoid clever heuristics), this would imply that the use of -- is mandatory as soon as you want to pass unquoted flags to an test-suite (w/o --test-options).

Some examples to clarify what I mean:

• cabal new-test test1 test2 -- --some-test-option: two testsuites test1 & test2; and one flag passed to both
• cabal new-test test1 test2: two testsuites test1 & test2
• cabal new-test test1 test2 --some-test-option: invalid, as --some-test-option would be assumed to be a cabal flag.
• cabal new-test test1 -- test2 --some-test-option: one testsuite test1, which is invoked with two arguments test2 --some-test-option.

We can still support per-test-component flags via --test-options flags, maybe

• cabal new-test test1 --test-options 'flags for test1' test2 --test-options 'flags for test2'

[23Skidoo]

Our cmdline parser currently doesn't provide the information about the relative position of -- in the extra args list, in both case 1 and case 4 above the client will receive ["test1", "test2", "--some-test-option"] as input.