Poll as to possible new features

[nathanross]

First off, I want to say great crate - in a case of what can only be convergent evolution, it's very similar to a crate I built a while back to fulfill a similar need and given that this crate has more buy-in (and let's be honest, has a better, more intuitive, name), I'll probably retire second_law if I find it's no longer needed.

Second_Law has a lot of features that I'm wondering if there are any you'd be interested in me making PRs for. I didn't want to spam you with feature requests, but let me know if opening a separate issue each is better.

1. Default to operation within a temporary folder
2. Support for specifying the executable relative or absolute path
3. Runs the executable in the temporary folder with a "clean slate" of environment variables by default, with the option of keeping the environment the user has set.
4. Support for a fixtures directory whose contents gets copied over to the temporary directory
5. Support for multicall binaries
6. Support for piped input
7. Support for evaluating that stdout or stderr is equal to the contents of a fixture

Note that second_law is not dual-licensed, so it would probably be best for these to be implemented fresh, not copied. However, if relicense+reuse is the easier path in your mind, you're welcome to set up a poll of me and the nine contributors to this file which second_law is derived from.

[killercup]

Thanks! That's quite the list!

I really want to keep assert_cli simple. So, while piping input might be a thing that I want to add (to allow testing CLI apps that read from stdin), dealing with temp dirs is probably out of scope of this crate. I've recently written these tests, and was wondering if some of the util functions could become their own crate (that can internally use assert_cli).

Support for evaluating that stdout or stderr is equal to the contents of a fixture

This is basically .prints_exactly(include_str!("test/fixtures/foo/stdout")), right?

[nathanross]

Good point on the fixture thing.

With piping and fixtures covered, I think if I were to pare down the rest of the list by stripping away the special "off the shelf use case" features, I guess what remains is whether you'd be interested in what I consider these two key unopinionated features:

• Supporting setting/clearing of the environment variables. Launching an executable is an important chokepoint for this. Setting them when launching a Command is familiar, idiomatic rust, and there is no other convenient, conventional way to do this in each test (the next best way is storing, clearing, then returning the environment). Testing against consistent environment variables across test environments is important not just when developing binaries that are sensitive to environment variables, but also if your tests involve calling other binaries that may be very sensitive to Locale environment variables, etc.

• Supporting setting of the command working directory. Like environment variables, some executables (whether due to legacy support or the nature of their function) are sensitive to the current working directory. This is another feature that can be set with Command, and is very inconvenient / has no convention of a way to replicate if not setting it within Command.

One could separately implement all the other features and they could still be used jointly with assert_cli still being a value add (at the least, as a rust community convention over asserts).

[killercup]

Great suggestions, I'll open issues to track those right now.

I'm a bit wary of implementing any new things before I have a concrete use case, though. coreutils seems like a good list of cli tools we could tests, but at the same time, many seem to be pretty special in their own way…

[nathanross]

coreutils seems like a good list of cli tools we could tests

I was thinking the same thing. The more I think about it, the more I appreciate the conceptually cleanliness of balkanizing custom assertions in their own crate. Coreutils currently uses the progenitor of second_law, but if there is a third party crate that is already established, has name and method recognition, and can accommodate the features (or most of them) in those three issues that we depend on a lot, switching to it as a convention could help eliminate a lot of custom code.

I have a lot of time this april when I will be catching up on coreutils; reducing the amount of custom test utility code will probably come to the front of the queue around late april.