x.py (bootstrap) does not accept -j, despite the docs and apparent logic to handle it.

[CleanCut]

While working on #40417 I discovered that x.py spits out the help string if you try to use the -j option. This will affect #40366 as well, which I ran across while looking to see if this issue had already been reported.

x.py is just a thin wrapper around the main() function of src/bootstrap/bootstrap.py, which mostly ensures the bootstrap binary (src/bootstrap/bin/main.rs) has been compiled and then passes most arguments (including -j) to the bootstrap binary.

It is the bootstrap binary itself which can't handle the -j argument -- the python script just passes the argument through to it. The bootstrap binary is placed at build/bootstrap/debug/bootstrap when it is built. You can see the buggy behavior by just doing this:

$ build/bootstrap/debug/bootstrap -j 10 build
unknown command: -j
Usage: x.py -j [options] [<args>...]

Options:
    -v, --verbose       use verbose output (-vv for very verbose)
    -i, --incremental   use incremental compilation
        --config FILE   TOML configuration file for build
        --build BUILD   build target of the stage0 compiler
        --host HOST     host targets to build
        --target TARGET target targets to build
        --on-fail CMD   command to run on failure
        --stage N       stage to build
        --keep-stage N  stage to keep without recompiling
        --src DIR       path to the root of the rust checkout
    -j, --jobs JOBS     number of jobs to run in parallel
    -h, --help          print this help message

Subcommands:
    build       Compile either the compiler or libraries
    test        Build and run some test suites
    bench       Build and run some benchmarks
    doc         Build documentation
    clean       Clean out build directories
    dist        Build and/or install distribution artifacts

To learn more about a subcommand, run `./x.py <command> -h`

If the bootstrap binary hasn't been built yet, you can build it by running x.py build first.

I think the problem is in the getopts logic in src/bootstrap/flags.rs. That's where I have gotten to debugging the program, anyway. I have never used the getopts module, so I paused to create this issue before diving into the docs.

I'll continue on and see if I can debug it and produce a patch, but I would be delighted if someone beat me to it. ;-)

[hanna-kruppe]

Odd, I seem to recall successfully passing -j before. Perhaps it's a recent regression? (The corresponding getopts call hasn't been touched in over a year, though: https://github.com/rust-lang/rust/blame/master/src/bootstrap/flags.rs#L89)

[petrochenkov]

x.py build -j 10 works, x.py -j 10 build doesn't.

[nagisa]

Duplicate of #38373

[retep998]

This is indeed a duplicate. The first parameter to x.py must be a subcommand with flags coming afterward.

[CleanCut]

Well, then, the bug is in the documentation. The help appears to state that -j is a global option, that -j must come before subcommands (if you make the leap that subcommands must be part of [<args>, ...] and not [options]), and doesn't even list subcommands as a possibility in the main usage line. On top of that, whatever option caused the problem is listed as part of the usage, which confuses things even more. (See how the -j is included in the usage line? Whatever option is misinterpreted as a subcommand will be placed there).

I would consider this mis-information as a separate bug from the inability to parse options before subcommands.

[CleanCut]

...and I'm putting my time where my mouth is -- working on a PR ;-)

[CleanCut]

It's coming along nicely. I have at least one more fix and a feature to do before I submit the PR. Work is on this branch, if anyone's interested.