Add --cfg and --rustc-cfg flags to output compiler configuration

[volks73]

This PR is my attempt to address #8923.

I have added the --cfg flag to the cargo rustc subcommand and the --rustc-cfg flag to the cargo build, cargo check, cargo test, and cargo bench subcommands, respectively. Both versions of the flag, --cfg and --rustc-cfg, do the same thing, but I thought it looked weird for the cargo rustc subcommand to be cargo rustc --rustc-cfg. The following example invocations are possible, once stabilized:

• cargo rustc --cfg
• cargo build --rustc-cfg
• cargo check --rustc-cfg
• cargo test --rustc-cfg
• cargo bench --rustc-cfg

In each case, compilation is aborted and only compiler configuration is emitted. All of the context creation and configuration is still executed, but execution of the compilation job is aborted. Similar to the --unit-graph implementation, the --cfg/--rustc-cfg flag is hidden, marked as a "unstable", and requires the -Z unstable-options flag at the moment. A complete example invocation with this PR would be:

$ cargo +nightly rustc -Z unstable-options --cfg

I am open to alternatives for the flag name. I have thought of --compiler-cfg, --compile-cfg, and --target-cfg, but I went with --rustc-cfg because it is the Rust compiler (rustc) configuration (cfg). The --target-cfg could be confusing because there are Cargo targets and Compiler targets. A lone --cfg for the build, check, test, and bench subcommands would also be ambiguous and configuration that is being displayed.

Originally, I was only going to add the --cfg flag to the cargo rustc subcommand, but build, check, test, and bench all have the --unit-graph flag, and I used that flag's implementation and existence as a template for this implementation. I am not opposed to having just the --cfg flag for the cargo rustc subcommand as originally proposed in #8923.

I discovered during my initial investigation to implement the feature and familiarization with the Cargo codebase, that as part of the build context creation and compilation process, Cargo internally calls the rustc --print cfg command for all targets and stores the output in the RustcTargetData type. It does this for the host and any explicitly defined targets for cross-compilation. Compilation features, such as +crt-static, added to the compilation process through Cargo environment variables or TOML configuration directives are added to the internal rustc --print cfg command call. So, the --cfg/--rustc-cfg just emits the contents of the RustcTagetData type as JSON. There is no need to call the rustc --print cfg command again. The implementation then becomes nearly identical to the --unit-graph implementation.

The output is only in JSON, similar to the --unit-graph feature, instead of the key-value and name style of the rustc --print cfg output because it easily resolves issues related to multi-targets, explicit target (--target <TRIPLE>) versus host configurations, and cross compilation. Subcommands and other projects can parse the JSON to recreate the key-value and name style if desired based on a specific target or the host. Here is an example of the JSON output (formatted for humans, the actual output is condensed), which is also available as a comment in the cargo::core::compiler::rustc_cfg module:

{
     "version": 1,
     "host": {
         "names": ["windows", "debug_assertions"],
         "arch":"x86_64",
         "endian":"little",
         "env":"msvc",
         "family":"windows",
         "features":["fxsr","sse","sse2"],
         "os":"windows",
         "pointer_width":"64",
         "vendor":"pc"
     },
     "targets": {
         "x86_64-unknown-linux-gnu": {
             "names": ["debug_assertions", "unix"],
             "arch":"x86_64",
             "endian":"little",
             "env":"gnu",
             "family":"unix",
             "features": ["fxsr","sse","sse2"],
             "os":"linux",
             "pointer_width":"64",
             "vendor":"unknown"
         },
         "i686-pc-windows-msvc": {
             "names": ["windows", "debug_assertions"],
             "arch":"x86",
             "endian":"little",
             "env":"msvc",
             "family":"windows",
             "features":["fxsr","sse","sse2"],
             "os":"windows",
             "pointer_width":"32",
             "vendor":"pc"
         }
     }
 }

I decided to remove the "target_" prefix from the relevant configurations to reduce "noise" in the output. Again, I am open to alternatives or suggestions on the JSON format.

[rust-highfive]

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @Eh2406 (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

Please see the contribution instructions for more information.

[ehuss]

Thanks for the PR! Just a heads up, with the holidays, it may take a while to get a proper look at this.

Just my initial reaction, I'm a little uneasy about adding so many flags. AFAIK, the cfg values only depend on the --target flag, is that correct? The reason --unit-graph was done that way is because it depends on the command and pretty much all the flags, so it couldn't really work as a separate command.

I would also avoid hard-coding keys and stripping prefixes, and just dump the contents of the Cfg map. Is there a particular reason to not display the whole thing?

[volks73]

Thanks for the PR! Just a heads up, with the holidays, it may take a while to get a proper look at this.

Your welcome, and I understand. Happy holidays!

Just my initial reaction, I'm a little uneasy about adding so many flags. AFAIK, the cfg values only depend on the --target flag, is >that correct? The reason --unit-graph was done that way is because it depends on the command and pretty much all the flags, so >it couldn't really work as a separate command.

Yes, I believe the cfg values are only dependent on the --target option(s) and the reasoning for the --unit-graph flag makes sense. The --cfg/--rustc-cfg flag should probably only be added for the cargo rustc subcommand. Obtaining the compiler configuration for a specific Cargo target, like for tests, could be obtained with cargo rustc --test --cfg instead of cargo test --rustc-cfg. My original thought was to only add the flag to the cargo rustc subcommand anyways. I will hold off on making this change until further confirmation is received.

I would also avoid hard-coding keys and stripping prefixes, and just dump the contents of the Cfg map. Is there a particular reason to not display the whole thing?

Up until commit 234089, the initial implementation was just a dump of the contents:

{
    "version": 1,
    "host": {
        "names": ["windows", "debug_assertions"],
        "key_pairs": [
            {"target_arch": "x86_64"},
            {"target_endian": "little"},
            {"target_env": "msvc"},
            {"target_family": "windows"},
            {"target_feature": "fxsr"},
            {"target_feature": "sse"},
            {"target_feature": "sse2"},
            {"target_os": "windows"},
            {"target_pointer_width": "64"},
            {"target_vendor": "pc"}
        ]
    },
    "targets": [
        "x86_64-unknown-linux-gnu": {
            "names": ["debug_assertions", "unix"],
            "key_pairs": [
                {"target_arch": "x86_64"},
                {"target_endian": "little"},
                {"target_env": "gnu"},
                {"target_family": "unix"},
                {"target_feature": "fxsr"},
                {"target_feature": "sse"},
                {"target_feature": "sse2"},
                {"target_os": "linux"},
                {"target_pointer_width": "64"},
                {"target_vendor": "unknown"}
            ]
        },
         "i686-pc-windows-msvc": {
            "names": ["windows", "debug_assertions"],
            "key_pairs": [
                {"target_arch":"x86"},
                {"target_endian":"little"},
                {"target_env":"msvc"},
                {"target_family":"windows"},
                {"target_feature":"fxsr"},
                {"target_feature":"sse"},
                {"target_feature":"sse2"},
                {"target_os":"windows"},
                {"target_pointer_width":"32"},
                {"target_vendor":"pc"}
         }
    ]
}

And looking at the output, I see a lot of "target_" being repeated. I use repeated prefixes as a "code smell" or indicator that something should be refactored/restructured. From the consumer side of the JSON output, like in a crate or third party subcommand, I would not have Cfg::target_os, Cfg::target_family, Cfg::target_env, etc. methods/fields, but Cfg::os, Cfg::family, Cfg::env, etc. or even Cfg::Target::os, Cfg::Target::family, Cfg::Target::env, etc. So, I thought the JSON should mirror this.

I recognize hard-coding and stripping prefixes is maybe not the most flexible or future-proof implementation. All of the output I tested with the rustc --print cfg --target <TRIPLE> command had the hard-coded fields, i.e. "target_env", "target_os", "target_endian", etc., albeit an empty string for targets without the value. This seemed to indicate that the "target_*" configurations were not going away and would always be present, but it is possible new configurations (prefixed with target_ or not) might be added or used and this is the concern? I thought the Version field would help in these situations.

I don't really have a strong reason for the proposed JSON format other than "look and feel", smaller number of bytes, and empirical evidence that certain fields are always available, which is not great because it is really for machine consumption, not human consumption, bandwidth/performance is not a concern, and I am not familiar enough with future compiler changes. Honestly, I struggled with the format and have mentally run around in circles on it. So, I decided I reached my limits and it was time to seek advice and review. I would greatly appreciate some direction, suggestions, and/or alternative organization of the output (when time and capacity permit).

An example of me mentally running around in a circle, a hybrid approach that condenses "target_*" configurations to "target" array field but maintains some extensibility because everything is an array where new configurations can just be added as new maps, but indicates that both the "names" and "target" configurations/fields are ultimately optional and may not exist:

{
    "version": 1,
    "host": [
        {"names": ["windows", "debug_assertions"]},
        {"target": [
            {"arch": "x86_64"},
            {"endian": "little"},
            {"env": "msvc"},
            {"family": "windows"},
            {"features": ["fxsr", "sse", "sse2"]},
            {"os": "windows"},
            {"pointer_width": "64"},
            {"vendor": "pc"}
        ]}
    ],
    "targets": [
        "i686-pc-windows-msvc": [
            {"names": ["windows", "debug_assertions"]},
            {"target": [
                {"arch":"x86"},
                {"endian":"little"},
                {"env":"msvc"},
                {"family":"windows"},
                {"features":["fxsr","sse","sse2"]},
                {"os":"windows"},
                {"pointer_width":"32"},
                {"vendor":"pc"}
            ]}
        ]
    ]
}

[alexcrichton]

I would agree with @ehuss that we probably don't want to munge the values and instead just report exactly what rustc is telling us. I think that a flag also may not be necessary for this, we can probably just add this json blob to the ones already emitted?

[volks73]

I think that a flag also may not be necessary for this, we can probably just add this json blob to the ones already emitted?

I am not aware of other flags that output JSON and skip compiling/building. Are you referring to the --unit-graph flag? If yes, then I am good with making this implementation, too. Would it just be:

{
    "rustc": ["windows", "debug_assertions", "target_arch='x86'", "target_endian='little'", ...]
}

somewhere within the --unit-graph JSON?

[alexcrichton]

Oh I thought that this was just a normal json message coming from Cargo during build? We can add it to --unit-graph as well but I think it's fine for us to just add other blobs coming out during a build, they should have an identifier on them and implementations should be skipping identifiers they don't recognize.

[volks73]

Oh I thought that this was just a normal json message coming from Cargo during build?

The goal was to get the compiler configurations with Cargo modifications, such as those defined in environment variables or Cargo configuration files, without building/compiling the project, similar to the rustc --print cfg command. I have not looked into the JSON messages that are emitted during the Cargo build process to see if the compiler configurations are part of the output, but I could see where this information would also be useful in that context as well.

I looked at the --unit-graph output. There is the platform field, which is null if compilation is for the host and a target triple if it is for cross-compilation. A cfg, configurations, rustc, compiler_cfg, compiler_configurations, rustc_configurations, or other similarly named field could be added after the platform field:

...
"platform": null,
"compiler_configurations":["debug_assertions","windows","target_arch='x86'","target_endian='little'", ...],
"mode": "build",
...

I would still like to consider a separate flag for the cargo rustc --cfg command and not include the configuration in the --unit-graph output for two reasons:

1. The --unit-graph feature appears to be further along and closer to stabilization. I do not want this feature to slow down the stabilization of the --unit-graph feature as the --unit-graph feature appears to be more broadly desired. The compiler configuration field could be added after stabilization as a Version 2 of the --unit-graph output since I could see the configuration information for each unit being useful as well.
2. A cargo rustc --cfg command would be a nice analog to the rustc --print cfg command, which prints the configuration but skips compilation. The compiler configuration would be buried in the --unit-graph output. The cargo rustc --cfg command is more "semantically meaningful" to me, i.e. "show me the compiler configuration for the cargo compilation process".

[alexcrichton]

Ah ok I understand now, thanks for helping me to clarify. I missed that this was a pre-compilation step rather than something learned during compilation.

To add to what @ehuss was mentioning earlier, this feels like the current implementation is a bit heavyweight relative to what it's doing. For example this is downloading the entire crate graph, which it probably doesn't need if it's just scraping config values. Unfortunately we don't really have a great analog for something similar to this and where it would go today, so I don't know where it would best go.

[volks73]

I have changed to using the exec function instead of the exec_with_output. Something must have been up with my system because when I originally tried to use the exec function, I was getting no output and the command appeared to hang and not work. Now it works!

Anyways, I will work on adding a test next.

[volks73]

I have added a couple of tests. These tests include using the --print cfg command with the -Z multitarget option, too.

[alexcrichton]

Looks good to me, thanks!

@bors: r+

[bors]

📌 Commit 2a5355f has been approved by alexcrichton

[bors]

⌛ Testing commit 2a5355f with merge 8d8459a7f81bbb335b38665f980e03ce5c5b9ce9...

[bors]

💔 Test failed - checks-actions

[ehuss]

@bors retry
macos: The runner has received a shutdown signal.

[bors]

⌛ Testing commit 2a5355f with merge 4ae4aad...

[bors]

☀️ Test successful - checks-actions
Approved by: alexcrichton
Pushing 4ae4aad to master...