Add support for custom JSON targets when using build-std. #120232

c272 · 2024-01-22T11:42:08Z

Currently, when building with build-std, some library build scripts check properties of the target by inspecting the target triple at env::TARGET, which is simply set to the filename of the JSON file when using JSON target files.

This patch alters these build scripts to use env::CARGO_CFG_* to fetch target information instead, allowing JSON target files describing platforms without restricted_std to build correctly when using -Z build-std. There are some weak assertions here (for example, nintendo && newlib), however this seems at least a marginal improvement on the existing solution.

Fixes rust-lang/wg-cargo-std-aware#60.

rustbot · 2024-01-22T11:42:17Z

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @Mark-Simulacrum (or someone else) soon.

Please see the contribution instructions for more information. Namely, in order to ensure the minimum review times lag, PR authors and assigned reviewers should ensure that the review label (S-waiting-on-review and S-waiting-on-author) stays updated, invoking these commands when appropriate:

@rustbot author: the review is finished, PR author should check the comments and take action accordingly
@rustbot review: the author is ready for a review, this PR will be queued again in the reviewer's queue

Mark-Simulacrum · 2024-01-23T00:42:27Z

To make sure I understand the problem this is solving, you have custom target spec JSON not named after the triplet for the target? E.g., it looks to me like making the file x86_64-linux-special-gnu.json would probably also make all of these build scripts work?

(Doesn't make this change a bad one but want to make sure I understand the problem)

c272 · 2024-01-23T09:45:06Z

@Mark-Simulacrum Yes, renaming the JSON target file to include "linux" in the filename would allow the target to build (in the linked issue, the author is using mytarget.json, so none of the clauses match). The current behaviour seems a bit imprecise, for example you could currently name a target file "contour" or something similarly unrelated and catch the "nto" clause unknowingly, or have a valid target with a different filename that doesn't compile as above.

bors · 2024-01-23T12:57:10Z

☔ The latest upstream changes (presumably #117958) made this pull request unmergeable. Please resolve the merge conflicts.

Mark-Simulacrum · 2024-01-27T18:51:01Z

@bors r+ rollup

This looks good to me. Thanks!

bors · 2024-01-27T18:51:03Z

📌 Commit 797cf59 has been approved by Mark-Simulacrum

It is now in the queue for this repository.

Add support for custom JSON targets when using build-std. Currently, when building with `build-std`, some library build scripts check properties of the target by inspecting the target triple at `env::TARGET`, which is simply set to the filename of the JSON file when using JSON target files. This patch alters these build scripts to use `env::CARGO_CFG_*` to fetch target information instead, allowing JSON target files describing platforms without `restricted_std` to build correctly when using `-Z build-std`. There are some weak assertions here (for example, `nintendo && newlib`), however this seems at least a marginal improvement on the existing solution. Fixes rust-lang/wg-cargo-std-aware#60.

bjorn3 · 2024-01-27T22:52:31Z

library/std/build.rs

@@ -48,7 +53,7 @@ fn main() {
 // - mipsel-sony-psp
 // - nvptx64-nvidia-cuda
 // - arch=avr
- // - JSON targets
+ // - JSON targets not describing an excluded target above.


As I understand it restricted-std is meant to prevent being able to use libstd on stable for targets that we don't actually support libstd support for once -Zbuild-std is stabilized. When using custom json targets, IMO it shouldn't be stable as libstd changes may break support in the future even if it happens to work currently.

Hm, I think that seems reasonable. We have a long-standing issue about making custom target specs entirely unstable (#71009), which seems like the better fix to me. Or an explicit guard here if we want (not sure how feasible that is). I'd be ok cutting this specific change.

Giving `--target=file.json some unstable-machinery at the CLI level, to me, would clearly imply that std doesn't support JSON targets in the same way Rust doesn't.

At the moment I'm not sure how a build script would detect a JSON target without the old broken TARGET behaviour

Add support for custom JSON targets when using build-std. Currently, when building with `build-std`, some library build scripts check properties of the target by inspecting the target triple at `env::TARGET`, which is simply set to the filename of the JSON file when using JSON target files. This patch alters these build scripts to use `env::CARGO_CFG_*` to fetch target information instead, allowing JSON target files describing platforms without `restricted_std` to build correctly when using `-Z build-std`. There are some weak assertions here (for example, `nintendo && newlib`), however this seems at least a marginal improvement on the existing solution. Fixes rust-lang/wg-cargo-std-aware#60.

…iaskrgr Rollup of 8 pull requests Successful merges: - rust-lang#116677 (References refer to allocated objects) - rust-lang#120232 (Add support for custom JSON targets when using build-std.) - rust-lang#120266 (Improve documentation for [A]Rc::into_inner) - rust-lang#120358 (Bump Fuchsia, build tests, and use 8 core bots) - rust-lang#120373 (Adjust Behaviour of `read_dir` and `ReadDir` in Windows Implementation: Check Whether Path to Search In Exists) - rust-lang#120402 (Make the coroutine def id of an async closure the child of the closure def id) - rust-lang#120420 (Stop using derivative in rustc_pattern_analysis) - rust-lang#120425 (Remove unnecessary unit returns in query declarations) r? `@ghost` `@rustbot` modify labels: rollup

Add support for custom JSON targets when using build-std. Currently, when building with `build-std`, some library build scripts check properties of the target by inspecting the target triple at `env::TARGET`, which is simply set to the filename of the JSON file when using JSON target files. This patch alters these build scripts to use `env::CARGO_CFG_*` to fetch target information instead, allowing JSON target files describing platforms without `restricted_std` to build correctly when using `-Z build-std`. There are some weak assertions here (for example, `nintendo && newlib`), however this seems at least a marginal improvement on the existing solution. Fixes rust-lang/wg-cargo-std-aware#60.

…iaskrgr Rollup of 6 pull requests Successful merges: - rust-lang#116677 (References refer to allocated objects) - rust-lang#118533 (Suppress unhelpful diagnostics for unresolved top level attributes) - rust-lang#120232 (Add support for custom JSON targets when using build-std.) - rust-lang#120238 (Always check the result of `pthread_mutex_lock`) - rust-lang#120266 (Improve documentation for [A]Rc::into_inner) - rust-lang#120373 (Adjust Behaviour of `read_dir` and `ReadDir` in Windows Implementation: Check Whether Path to Search In Exists) r? `@ghost` `@rustbot` modify labels: rollup

Add support for custom JSON targets when using build-std. Currently, when building with `build-std`, some library build scripts check properties of the target by inspecting the target triple at `env::TARGET`, which is simply set to the filename of the JSON file when using JSON target files. This patch alters these build scripts to use `env::CARGO_CFG_*` to fetch target information instead, allowing JSON target files describing platforms without `restricted_std` to build correctly when using `-Z build-std`. There are some weak assertions here (for example, `nintendo && newlib`), however this seems at least a marginal improvement on the existing solution. Fixes rust-lang/wg-cargo-std-aware#60.

Mark-Simulacrum · 2024-01-28T14:59:47Z

@bors rollup=iffy

…iaskrgr Rollup of 8 pull requests Successful merges: - rust-lang#116677 (References refer to allocated objects) - rust-lang#118533 (Suppress unhelpful diagnostics for unresolved top level attributes) - rust-lang#119991 (Reject infinitely-sized reads from io::Repeat) - rust-lang#120232 (Add support for custom JSON targets when using build-std.) - rust-lang#120358 (Bump Fuchsia, build tests, and use 8 core bots) - rust-lang#120424 (raw pointer metadata API: data address -> data pointer) - rust-lang#120434 (Revert outdated version of "Add the wasm32-wasi-preview2 target") - rust-lang#120443 (Fixes footnote handling in rustdoc) r? `@ghost` `@rustbot` modify labels: rollup

library/std/build.rs

bors · 2024-01-29T11:30:27Z

💔 Test failed - checks-actions

c272 · 2024-01-29T14:58:16Z

Fixed the typo, and the new patch should also fix the test failures - all linux-android targets report their OS as android and not linux, which I overlooked.

library/std/build.rs

Currently, when building with `build-std`, some library build scripts check properties of the target by inspecting the target triple at `env::TARGET`, which is simply set to the filename of the JSON file when using JSON target files. This patch alters these build scripts to use `env::CARGO_CFG_*` to fetch target information instead, allowing JSON target files describing platforms without `restricted_std` to build correctly when using `-Z build-std`. Fixes wg-cargo-std-aware/rust-lang#60.

Mark-Simulacrum · 2024-02-11T01:54:40Z

@bors r+

bors · 2024-02-11T01:54:43Z

📌 Commit 1ecb084 has been approved by Mark-Simulacrum

It is now in the queue for this repository.

bors · 2024-02-11T02:10:20Z

⌛ Testing commit 1ecb084 with merge 0cbef48...

bors · 2024-02-11T04:23:34Z

☀️ Test successful - checks-actions
Approved by: Mark-Simulacrum
Pushing 0cbef48 to master...

rust-timer · 2024-02-11T07:09:21Z

Finished benchmarking commit (0cbef48): comparison URL.

Overall result: no relevant changes - no action needed

@rustbot label: -perf-regression

Instruction count

This benchmark run did not return any relevant results for this metric.

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	4.4%	[4.4%, 4.4%]	1
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-	-	0

Cycles

This benchmark run did not return any relevant results for this metric.

Binary size

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	0.1%	[0.1%, 0.1%]	1
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-	-	0

Bootstrap: 666.058s -> 668.04s (0.30%)
Artifact size: 308.32 MiB -> 308.26 MiB (-0.02%)

This PR aims to pin down exactly what restricted_std is meant to achieve and what it isn't. This commit fixes rust-lang/wg-cargo-std-aware#87 by explaining why the error appears and what the choices the user has. The error describes how std cannot function without knowing about some form of OS/platform support. Any features of std that work without an OS should be moved to core/alloc (see rust-lang#27242 rust-lang#103765). Note that the message says "platform" and "environment" because, since rust-lang#120232, libstd can be built for some JSON targets. This is still unsupported (all JSON targets probably should be unstable rust-lang/wg-cargo-std-aware#90), but a JSON target with the right configuration should hopefully have some partial libstd support. I propose closing rust-lang/wg-cargo-std-aware#69 as "Won't fix" since any support of std without properly configured os, vendor or env fields is very fragile considering future upgrades of Rust or dependencies. In addition there's no likely path to it being fixed long term (making std buildable for all targets being the only solution). This is distinct from tier 3 platforms with limited std support implemented (and as such aren't restricted_std) because these platforms can conceptually work in the future and std support should mainly improve over time. The alternative to closing rust-lang/wg-cargo-std-aware#69 is a new crate feature for std which escapes the restricted_std mechanism in build.rs. It could be used with the -Zbuild-std-features flag if we keep it permanently unstable, which I hope we can do anyway. A minor side-effect in this scenario is that std wouldn't be marked as unstable if documentation for it were generated with build-std.

…d-std, r=ehuss Document restricted_std This PR aims to pin down exactly what restricted_std is meant to achieve and what it isn't. This commit fixes rust-lang/wg-cargo-std-aware#87 by explaining why the error appears and what the choices the user has. The error describes how std cannot function without knowing about some form of OS/platform support. Any features of std that work without an OS should be moved to core/alloc (see rust-lang#27242 rust-lang#103765). Note that the message says "platform" and "environment" because, since rust-lang#120232, libstd can be built for some JSON targets. This is still unsupported (all JSON targets probably should be unstable rust-lang/wg-cargo-std-aware#90), but a JSON target with the right configuration should hopefully have some partial libstd support. I propose closing rust-lang/wg-cargo-std-aware#69 as "Won't fix" since any support of std without properly configured os, vendor or env fields is very fragile considering future upgrades of Rust or dependencies. In addition there's no likely path to it being fixed long term (making std buildable for all targets being the only solution). This is distinct from tier 3 platforms with limited std support implemented (and as such aren't restricted_std) because these platforms can conceptually work in the future and std support should mainly improve over time. The alternative to closing rust-lang/wg-cargo-std-aware#69 is a new crate feature for std which escapes the restricted_std mechanism in build.rs. It could be used with the -Zbuild-std-features flag if we keep it permanently unstable, which I hope we can do anyway. A minor side-effect in this scenario is that std wouldn't be marked as unstable if documentation for it were generated with build-std. cc `@ehuss`

…d-std, r=ehuss Document restricted_std This PR aims to pin down exactly what restricted_std is meant to achieve and what it isn't. This commit fixes rust-lang/wg-cargo-std-aware#87 by explaining why the error appears and what the choices the user has. The error describes how std cannot function without knowing about some form of OS/platform support. Any features of std that work without an OS should be moved to core/alloc (see rust-lang#27242 rust-lang#103765). Note that the message says "platform" and "environment" because, since rust-lang#120232, libstd can be built for some JSON targets. This is still unsupported (all JSON targets probably should be unstable rust-lang/wg-cargo-std-aware#90), but a JSON target with the right configuration should hopefully have some partial libstd support. I propose closing rust-lang/wg-cargo-std-aware#69 as "Won't fix" since any support of std without properly configured os, vendor or env fields is very fragile considering future upgrades of Rust or dependencies. In addition there's no likely path to it being fixed long term (making std buildable for all targets being the only solution). This is distinct from tier 3 platforms with limited std support implemented (and as such aren't restricted_std) because these platforms can conceptually work in the future and std support should mainly improve over time. The alternative to closing rust-lang/wg-cargo-std-aware#69 is a new crate feature for std which escapes the restricted_std mechanism in build.rs. It could be used with the -Zbuild-std-features flag if we keep it permanently unstable, which I hope we can do anyway. A minor side-effect in this scenario is that std wouldn't be marked as unstable if documentation for it were generated with build-std. cc ``@ehuss``

…d-std, r=ehuss Document restricted_std This PR aims to pin down exactly what restricted_std is meant to achieve and what it isn't. This commit fixes rust-lang/wg-cargo-std-aware#87 by explaining why the error appears and what the choices the user has. The error describes how std cannot function without knowing about some form of OS/platform support. Any features of std that work without an OS should be moved to core/alloc (see rust-lang#27242 rust-lang#103765). Note that the message says "platform" and "environment" because, since rust-lang#120232, libstd can be built for some JSON targets. This is still unsupported (all JSON targets probably should be unstable rust-lang/wg-cargo-std-aware#90), but a JSON target with the right configuration should hopefully have some partial libstd support. I propose closing rust-lang/wg-cargo-std-aware#69 as "Won't fix" since any support of std without properly configured os, vendor or env fields is very fragile considering future upgrades of Rust or dependencies. In addition there's no likely path to it being fixed long term (making std buildable for all targets being the only solution). This is distinct from tier 3 platforms with limited std support implemented (and as such aren't restricted_std) because these platforms can conceptually work in the future and std support should mainly improve over time. The alternative to closing rust-lang/wg-cargo-std-aware#69 is a new crate feature for std which escapes the restricted_std mechanism in build.rs. It could be used with the -Zbuild-std-features flag if we keep it permanently unstable, which I hope we can do anyway. A minor side-effect in this scenario is that std wouldn't be marked as unstable if documentation for it were generated with build-std. cc ```@ehuss```

Rollup merge of rust-lang#123360 - adamgemmell:dev/adagem01/restricted-std, r=ehuss Document restricted_std This PR aims to pin down exactly what restricted_std is meant to achieve and what it isn't. This commit fixes rust-lang/wg-cargo-std-aware#87 by explaining why the error appears and what the choices the user has. The error describes how std cannot function without knowing about some form of OS/platform support. Any features of std that work without an OS should be moved to core/alloc (see rust-lang#27242 rust-lang#103765). Note that the message says "platform" and "environment" because, since rust-lang#120232, libstd can be built for some JSON targets. This is still unsupported (all JSON targets probably should be unstable rust-lang/wg-cargo-std-aware#90), but a JSON target with the right configuration should hopefully have some partial libstd support. I propose closing rust-lang/wg-cargo-std-aware#69 as "Won't fix" since any support of std without properly configured os, vendor or env fields is very fragile considering future upgrades of Rust or dependencies. In addition there's no likely path to it being fixed long term (making std buildable for all targets being the only solution). This is distinct from tier 3 platforms with limited std support implemented (and as such aren't restricted_std) because these platforms can conceptually work in the future and std support should mainly improve over time. The alternative to closing rust-lang/wg-cargo-std-aware#69 is a new crate feature for std which escapes the restricted_std mechanism in build.rs. It could be used with the -Zbuild-std-features flag if we keep it permanently unstable, which I hope we can do anyway. A minor side-effect in this scenario is that std wouldn't be marked as unstable if documentation for it were generated with build-std. cc ```@ehuss```

rustbot assigned Mark-Simulacrum Jan 22, 2024

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-libs Relevant to the library team, which will review and decide on the PR/issue. labels Jan 22, 2024

c272 force-pushed the json-buildstd branch 2 times, most recently from 4146e40 to 797cf59 Compare January 23, 2024 14:56

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jan 27, 2024

matthiaskrgr mentioned this pull request Jan 27, 2024

Rollup of 10 pull requests #120431

Closed

bjorn3 reviewed Jan 27, 2024

View reviewed changes

matthiaskrgr mentioned this pull request Jan 27, 2024

Rollup of 8 pull requests #120432

Closed

matthiaskrgr mentioned this pull request Jan 28, 2024

Rollup of 7 pull requests #120436

Closed

matthiaskrgr mentioned this pull request Jan 28, 2024

Rollup of 6 pull requests #120441

Closed

matthiaskrgr mentioned this pull request Jan 28, 2024

Rollup of 8 pull requests #120448

Closed

Urgau mentioned this pull request Jan 28, 2024

Always check the result of pthread_mutex_lock #120238

Merged

kadiwa4 suggested changes Jan 29, 2024

View reviewed changes

library/std/build.rs Outdated Show resolved Hide resolved

bors added the S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. label Jan 29, 2024

This comment has been minimized.

Sign in to view

bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Jan 29, 2024

c272 force-pushed the json-buildstd branch from 797cf59 to a571647 Compare January 29, 2024 14:56

Mark-Simulacrum reviewed Feb 4, 2024

View reviewed changes

library/std/build.rs Outdated Show resolved Hide resolved

c272 force-pushed the json-buildstd branch from a571647 to 1ecb084 Compare February 5, 2024 14:11

Mark-Simulacrum mentioned this pull request Feb 11, 2024

restricted_std applicability to custom JSON targets rust-lang/wg-cargo-std-aware#90

Open

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Feb 11, 2024

bors added the merged-by-bors This PR was explicitly merged by bors. label Feb 11, 2024

bors merged commit 0cbef48 into rust-lang:master Feb 11, 2024
12 checks passed

rustbot added this to the 1.78.0 milestone Feb 11, 2024

bors mentioned this pull request Feb 11, 2024

Netbsd10 update #118738

Merged

c272 deleted the json-buildstd branch February 12, 2024 09:59

ehuss mentioned this pull request Feb 12, 2024

Can't build std if I specify target json file: std does not see networking rust-lang/wg-cargo-std-aware#60

Closed

adamgemmell mentioned this pull request Apr 2, 2024

Document restricted_std #123360

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for custom JSON targets when using build-std. #120232

Add support for custom JSON targets when using build-std. #120232

c272 commented Jan 22, 2024

rustbot commented Jan 22, 2024

Mark-Simulacrum commented Jan 23, 2024

c272 commented Jan 23, 2024 •

edited

Loading

bors commented Jan 23, 2024

Mark-Simulacrum commented Jan 27, 2024

bors commented Jan 27, 2024

bjorn3 Jan 27, 2024

Mark-Simulacrum Jan 27, 2024

adamgemmell Feb 2, 2024

Mark-Simulacrum commented Jan 28, 2024

This comment has been minimized.

bors commented Jan 29, 2024

c272 commented Jan 29, 2024

Mark-Simulacrum commented Feb 11, 2024

bors commented Feb 11, 2024

bors commented Feb 11, 2024

bors commented Feb 11, 2024

rust-timer commented Feb 11, 2024

Add support for custom JSON targets when using build-std. #120232

Add support for custom JSON targets when using build-std. #120232

Conversation

c272 commented Jan 22, 2024

rustbot commented Jan 22, 2024

Mark-Simulacrum commented Jan 23, 2024

c272 commented Jan 23, 2024 • edited Loading

bors commented Jan 23, 2024

Mark-Simulacrum commented Jan 27, 2024

bors commented Jan 27, 2024

bjorn3 Jan 27, 2024

Choose a reason for hiding this comment

Mark-Simulacrum Jan 27, 2024

Choose a reason for hiding this comment

adamgemmell Feb 2, 2024

Choose a reason for hiding this comment

Mark-Simulacrum commented Jan 28, 2024

This comment has been minimized.

bors commented Jan 29, 2024

c272 commented Jan 29, 2024

Mark-Simulacrum commented Feb 11, 2024

bors commented Feb 11, 2024

bors commented Feb 11, 2024

bors commented Feb 11, 2024

rust-timer commented Feb 11, 2024

Overall result: no relevant changes - no action needed

Instruction count

Max RSS (memory usage)

Cycles

Binary size

c272 commented Jan 23, 2024 •

edited

Loading