implement "target" features

[Gankra]

• able to specify that an audit is only valid for a target(s)
• able to specify that a policy only applies to a target(s) (package has no reqs for targets that aren't included)

Targets are based on cfg syntax and include:

• any (or)
• all (and)
• not
• platform "atoms" like target_arch, target_os
• feature "atoms" like my_crate_feature
• target "families" like "unix" and "wasm" which overlap with platform atoms
• target triples like x86_64-gnu-unknown-linux which overlap with platform atoms

[Gankra]

Here are some "interesting" cfg's that show up in firefox's and wasm-time's cargo metadata graphs:

// Stuff in wasm-time:

// Simple
"target": "cfg(unix)",
"target": "cfg(windows)",
"target": "cfg(not(windows))",
"target": "cfg(target_os = \"windows\")",
"target": "cfg(not(target_os = \"wasi\"))",
"target": "cfg(target_arch = \"wasm32\")",
"target": "cfg(not(target_arch = \"wasm32\"))",

// Complex
"target": "cfg(any(target_os = \"macos\", target_os = \"ios\"))",
"target": "cfg(any(target_arch = \"x86_64\", target_arch = \"x86\"))",
"target": "cfg(all(target_arch = \"aarch64\", target_os = \"linux\"))",
"target": "cfg(not(any(windows, target_os = \"hermit\")))",
"target": "cfg(not(any(windows, target_arch = \"s390x\")))",
"target": "cfg(not(all(target_arch = \"arm\", target_os = \"none\")))",
"target": "cfg(not(any(target_os = \"linux\", target_os = \"android\", target_os = \"windows\", target_os = \"macos\", target_os = \"ios\", target_os = \"freebsd\", target_os = \"openbsd\", target_os = \"netbsd\", target_os = \"dragonfly\", target_os = \"solaris\", target_os = \"illumos\", target_os = \"fuchsia\", target_os = \"redox\", target_os = \"cloudabi\", target_os = \"haiku\", target_os = \"vxworks\", target_os = \"emscripten\", target_os = \"wasi\")))",
"target": "cfg(any(target_os = \"linux\", target_os = \"android\", target_os = \"windows\", target_os = \"macos\", target_os = \"ios\", target_os = \"freebsd\", target_os = \"openbsd\", target_os = \"netbsd\", target_os = \"dragonfly\", target_os = \"solaris\", target_os = \"illumos\", target_os = \"fuchsia\", target_os = \"redox\", target_os = \"cloudabi\", target_os = \"haiku\", target_os = \"vxworks\", target_os = \"emscripten\", target_os = \"wasi\"))",

// Evil
"target": "cfg(all(any(target_arch = \"x86_64\", target_arch = \"aarch64\"), target_os = \"hermit\"))",
"target": "cfg(all(not(rustix_use_libc), not(miri), target_os = \"linux\", any(target_arch = \"x86\", all(target_arch = \"x86_64\", not(target_pointer_width = \"32\")), target_arch = \"arm\", target_arch = \"aarch64\", target_arch = \"powerpc64\", target_arch = \"riscv64\", target_arch = \"mips\", target_arch = \"mips64\")))",
"target": "cfg(all(windows, not(target_vendor = \"uwp\")))",

// Triples
"target": "aarch64-apple-darwin",
"target": "aarch64-linux-android",
"target": "wasm32-unknown-unknown",
"target": "asmjs-unknown-emscripten",
"target": "wasm32-unknown-emscripten",

// Totally Custom
"target": "cfg(loom)",
"target": "cfg(tokio_unstable)",
"target": "cfg(tracing_unstable)",
"target": "cfg(fuzzing)",




// Extra Evil Stuff in firefox

"target": "cfg(all(target_arch = \"wasm32\", not(any(target_os = \"emscripten\", target_os = \"wasi\"))))",
"target": "cfg(all(target_arch = \"wasm32\", not(target_os = \"emscripten\")))",
"target": "cfg(any(target_os = \"android\", all(unix, not(target_os = \"macos\"))))",
"target": "cfg(all(not(target_arch = \"wasm32\"), any(target_os = \"ios\", target_os = \"macos\")))",
"target": "cfg(all(not(target_arch = \"wasm32\"), unix, not(target_os = \"ios\"), not(target_os = \"macos\")))",
"target": "cfg(all(not(target_arch = \"wasm32\"), windows))",

[Gankra]

In general a cfg expression is a boolean expression with a possible large number of input vars, and answering questions like "is this cfg <= another" is BOOL SAT and therefore NP-complete. Cargo and rustc don't need to answer such questions because the pick a specific target and just plug the inputs in and check if an expr is true or false.

We would be evaluating the exprs more abstractly, putting us more in BOOL SAT territory. The existence of "not" also introduces problematic law-of-the-excluded-middle problems. For instance, checking if an audit with "any(X, Y)" satisfies "NOT Z" requires you to know for certain that the universe is strictly "X, Y, Z". This is not generally possible to know, as rustc can always introduce new OSes/arches.

If you remove "all" and "not" and just support "any" this is relatively tractable because "any" just means "set all these bits" and you can have nice little bitvectors of length N (N=number of atoms) and do subset queries easily. If you introduce either AND or NOT then things quickly become BOOL SAT and you need to have bitvectors of length 2^N to represent the full truth table, in the general case.

"triples" and "families" are however still annoying because we potentially need to "understand" the contents of "unix" or "x86_64-unknown-linux-gnu" to map them to equivalent atoms (which again, may have excluded middle issues).

[bholley]

Still not quite sure what to do with this. I think we can work around the lack of this for now, and reevaluate when a strong use-case arises.

[conradludgate]

I have a use case for this:

My company only deploys code to linux servers and runs dev environments on linux/macos laptops.

Currently, windows-sys appears in my cargo vet suggest but this never even gets compiled in any or our environments. Ideally, I could have

[policy.'*']
targets = ["unix"]

and it would just ignore windows-sys

[bholley]

For sure. We definitely want some kind of affordance for this, we just have to figure out how to make it computationally tractable.

In the mean time, you can add suggest = false to that entry in your unaudited.toml to get it off the radar.

[Gankra]

cargo-deny seems to have figured out some tolerable approach to this, we should look into it

https://embarkstudios.github.io/cargo-deny/checks/cfg.html#the-targets-field-optional

(cc @Jake-Shadle who was interested in the two interoping)

[bholley]

So I think there are two broad categories of potential feature here.

The first is to exclude crates from the vet graph that don't get built on the platforms of interest. This seems like something we can just borrow cargo-deny's approach for (neat to see that rustc has a set of built-ins). The key point here is that audits don't have to be target-aware at all.

The second is to allow excluding target-specific code at sub-crate granularity, by augmenting audit records to record that the audit is only valid for a subset of the targets for which the crate builds. This is a much-harder problem per all the discussion above.

So it seems like we should perhaps do the first-thing, which has clear use-cases, and punt on the second thing, which may never be needed often enough to justify the complexity.

[repi]

would definitely appreciate, esp. as a start, the first and simple approach we have with cargo-deny of set of exact target triples to include in the vetting graph.

here is for example how our main application/server project's deny.toml looks like in the top (and could imagine something quite similar for supply-chain/config.toml:

targets = [
    { triple = "x86_64-unknown-linux-gnu" },
    { triple = "aarch64-apple-darwin" },
    { triple = "x86_64-apple-darwin" },
    { triple = "x86_64-pc-windows-msvc" },
    { triple = "aarch64-linux-android" },
    { triple = "x86_64-unknown-linux-musl" },
]

and a our Wasm-only projects correspondingly have just:

targets = [
    { triple = "wasm32-unknown-unknown" },
]

which drastically helps prune the dependency graph, and exclude the more exotic dependencies for platforms we are not even supporting. would be a good step towards to reduce set of exemptions and be able to eventually get to 0 in some projects.

[Jake-Shadle]

Just FYI, one current limitation is laid out here EmbarkStudios/cargo-deny#324, my plan was just do each requested target in parallel and collate diagnostics, but it's not a huge issue.

[bholley]

@Gankra @mystor Do y'all have a sense of how straightforward it would be to implement option (1)? Conceptually it seems straightforward to just loop N times when constructing the crate graph but that may or may not line up easily with what we've got.

[bholley]

It'd be extra nice if we could use the same target list for cargo-vet and cargo-vendor, which would allow us to fix rust-lang/cargo#6179. Right now we have some gross a-hoc workarounds to avoid vendoring certain crates into mozilla-central.

[hpenne]

In the mean time, you can add suggest = false to that entry in your unaudited.toml to get it off the radar.

Is this unaudited.toml file a feature that has been removed? The documentation does not seem to mention it. I'm looking for a way to configure a crate as a permanent excemption (any version), for cases where a crate is pulled in to support a platform that we will never building for.

[mystor]

unaudited has been renamed to exemptions since #228 - you can find the entry in your config.toml and add the suggest = false option to it there.

[bholley]

If you want a permanent exemption across all versions, you can also add a policy entry for the crate and set criteria = []