std::os::windows missing on http://doc.rust-lang.org/nightly/std/os/ #24658

SimonSapin · 2015-04-21T09:20:35Z

The builders are running rustdoc on Linux, so #[cfg(windows)] is false.

Maybe the platform rustdoc in running on should not be relevant for generating documentation. Could rustdoc be run with both --cfg unix and --cfg windows?

The text was updated successfully, but these errors were encountered:

steveklabnik · 2015-04-21T12:29:41Z

Could rustdoc be run with both --cfg unix and --cfg windows?

I would imagine this wouldn't work, as they have implementaitons that conflict and such.

nwin · 2015-04-21T12:29:47Z

This is actually not so trivial. What about things that exist on both Platforms but are different?

klutzy · 2015-04-21T12:30:59Z

cc #1998

SimonSapin · 2015-04-21T12:31:18Z

My suggested solution probably doesn’t work, but the issue remains that non-Linux things should still be documented.

SimonSapin · 2015-04-21T12:32:03Z

Yes, this sounds like a special case of #1998.

steveklabnik · 2015-04-21T13:01:26Z

I'm going to nominate this particular version, as it would be really unfortunate for the official docs to only have linux docs. Maybe we can have something bespoke for 1.0 at first until the underlying bug is fixed

nwin · 2015-04-21T13:06:41Z

The most straightforward solution is probably to compile all docs separately and add a little pop-up box on the top to switch between Windows <-> Linux <-> Mac. I suppose one needs some more metadata to avoid that google gets confused…

lambda · 2015-04-21T13:07:29Z

rust-lang/rfcs#915 is a postponed RFC that proposes one possible solution.

lambda · 2015-04-21T13:12:51Z

This is actually not so trivial. What about things that exist on both Platforms but are different?

I think that that case should generally be avoided, shouldn't it? It would be confusing to have the same functionality available on multiple platforms, but with conflicting implementations; instead, if functionality differs meaningfully, they should probably have different names. For example, in the standard library, there are std::os::unix and std::os::windows, so even if you happened to have some operation with the same name, they'd be in different modules and so could be documented separately.

Are there current cases in the standard library or widely used crates in which items with the same name but conflicting signatures exist on the different platforms?

nwin · 2015-04-21T13:19:18Z

Are there current cases in the standard library or widely used crates in which items with the same name but conflicting signatures exist on the different platforms?

I was thinking of libc::types::os::common::bsd44::sockaddr_in. That struct is defined differently on Darwin and Linux (probably also Windows). I’m pretty sure there are more surprises like this lurking around.

SimonSapin · 2015-04-21T13:50:57Z

The most straightforward solution is probably to compile all docs separately and add a little pop-up box on the top to switch between Windows <-> Linux <-> Mac.

It would be unfortunate to duplicate the entire set of documentation for a comparatively small set of differences.

lambda · 2015-04-21T15:00:11Z

I was thinking of libc::types::os::common::bsd44::sockaddr_in. That struct is defined differently on Darwin and Linux (probably also Windows). I’m pretty sure there are more surprises like this lurking around.

Ah, you're right. There are a bunch of those cases in libc. I guess that means that the cfg(doc) approach probably won't work.

It would be unfortunate to duplicate the entire set of documentation for a comparatively small set of differences.

Agreed, especially since you couldn't have just those three choices, you would need to be able to switch based on any cfg option that could affect the set of methods available.

Given that it looks like it will need to support displaying different signatures for different platforms, it seems like the best you could do is list each of them, with the appropriate cfg flags listed to disambiguate which one is for which platform. That might get a little cumbersome for libc, but shouldn't affect most other code since most code should be common that doesn't depend on the platform, or platform-specific and organized in a platform-specific module; cases like libc should be more often the exception rather than the rule.

SimonSapin · 2015-04-21T15:06:39Z

Perhaps std::os::* modules are enough of a special case that we should generate separate docs for them specifically? Can rustdoc be instructed to document only one module and its contents?

steveklabnik · 2015-04-22T20:28:13Z

Rustdoc operates on crates, just like everything else. It needs to be able to compile the crate to run doc tests.

pnkfelix · 2015-04-23T21:21:53Z

We should fix this to some degree for 1.0. There are both short-term solutions and long-term solutions.

pnkfelix · 2015-04-23T21:22:18Z

P-medium.

brson · 2015-04-23T21:23:00Z

As an immediate solution we might upload docs for all three tier-one platforms.

KostaCitrix · 2015-07-13T15:19:50Z

Please fix. It's really confusing to see missing/wrong documentation if working on non-linux. This is one more 'gotcha' that you need to know, so this makes it harder to start working with rust on non-linux platform.

Thank you for you consideration! :)

steveklabnik · 2015-08-01T20:38:04Z

I wonder if @edunham has any thoughts on this

retep998 · 2016-09-07T11:42:01Z

We really need to upload docs for more platforms already. It is honestly quite silly that there are no docs for libstd on Windows online, especially given the sheer number of cases where people couldn't find Windows specific functionality like OsStringExt. It's such a simple thing to do, and the fact that it hasn't been done is sending a message to Windows users that Rust doesn't care about them.

Diggsey · 2016-09-07T11:52:41Z

I agree, IMO even just the solution used by https://docs.rs/ would be fine.

eminence · 2016-09-30T18:35:22Z

This could be partly solved by having local crate docs link to the local stdlib docs, instead of linking to the unix-only docs on rust-lang.org.

See the second half of #35183

…ichton Expose all OS-specific modules in libstd doc. 1. Uses the special `--cfg dox` configuration passed by rustbuild when running `rustdoc`. Changes the `#[cfg(platform)]` into `#[cfg(any(dox, platform))]` so that platform-specific API are visible to rustdoc. 2. Since platform-specific implementations often won't compile correctly on other platforms, `rustdoc` is changed to apply `everybody_loops` to the functions during documentation and doc-test harness. 3. Since platform-specific code are documented on all platforms now, it could confuse users who found a useful API but is non-portable. Also, their examples will be doc-tested, so must be excluded when not testing on the native platform. An undocumented attribute `#[doc(cfg(...))]` is introduced to serve the above purposed. Fixes #24658 (Does _not_ fully implement #1998).

@eddyb

Don't run `everybody_loops` for rustdoc; instead ignore resolution errors r? @eddyb cc @petrochenkov, @GuillaumeGomez, @Manishearth, @ecstatic-morse, @marmeladema ~~Blocked on rust-lang#73743 Merged. ~~Blocked on crater run.~~ Crater popped up some ICEs ([now fixed](rust-lang#73566 (comment))). See [crater run](https://crater-reports.s3.amazonaws.com/pr-73566/index.html), [ICEs](rust-lang#73566 (comment)). ~~Blocked on rust-lang#74070 so that we don't make typeck_tables_of public when it shouldn't be.~~ Merged. Closes rust-lang#71820, closes rust-lang#71104, closes rust-lang#65863. ## What is the motivation for this change? As seen from a lengthy trail of PRs and issues (rust-lang#73532, rust-lang#73103, rust-lang#71820, rust-lang#71104), `everybody_loops` is causing bugs in rustdoc. The main issue is that it does not preserve the validity of the `DefId` tree, meaning that operations on DefIds may unexpectedly fail when called later. This is blocking intra-doc links (see rust-lang#73101). This PR starts by removing `everybody_loops`, fixing rust-lang#71104 and rust-lang#71820. However, that brings back the bugs seen originally in rust-lang#43348: Since libstd documents items for all platforms, the function bodies sometimes do not type check. Here are the errors from documenting `libstd` with `everybody_loops` disabled and no other changes: ```rust error[E0433]: failed to resolve: could not find `handle` in `sys` --> src/libstd/sys/windows/ext/process.rs:13:27 | 13 | let handle = sys::handle::Handle::new(handle as *mut _); | ^^^^^^ could not find `handle` in `sys` error[E0425]: cannot find function `symlink_inner` in module `sys::fs` --> src/libstd/sys/windows/ext/fs.rs:544:14 | 544 | sys::fs::symlink_inner(src.as_ref(), dst.as_ref(), false) | ^^^^^^^^^^^^^ not found in `sys::fs` error[E0425]: cannot find function `symlink_inner` in module `sys::fs` --> src/libstd/sys/windows/ext/fs.rs:564:14 | 564 | sys::fs::symlink_inner(src.as_ref(), dst.as_ref(), true) | ^^^^^^^^^^^^^ not found in `sys::fs` ``` ## Why does this need changes to `rustc_resolve`? Normally, this could be avoided by simply not calling the `typeck_item_bodies` pass. However, the errors above happen before type checking, in name resolution itself. Since name resolution is intermingled with macro expansion, and rustdoc needs expansion to happen before it knows all items to be documented, there needs to be someway to ignore _resolution_ errors in function bodies. An alternative solution suggested by @petrochenkov was to not run `everybody_loops` on anything containing a nested `DefId`. This would solve some of the immediate issues, but isn't bullet-proof: the following functions still could not be documented if the items in the body failed to resolve: - Functions containing a nested `DefId` (rust-lang#71104) - ~~Functions returning `impl Trait` (rust-lang#43878 These ended up not resolving anyway with this PR. - ~~`const fn`, because `loop {}` in `const fn` is unstable (rust-lang#43636 `const_loop` was just stabilized. This also isn't exactly what rustdoc wants, which is to avoid looking at function bodies in the first place. ## What changes were made? The hack implemented in this PR is to add an option to ignore all resolution errors in function bodies. This is enabled only for rustdoc. Since resolution errors are ignored, the MIR generated will be invalid, as can be seen in the following ICE: ```rust error: internal compiler error: broken MIR in DefId(0:11 ~ doc_cfg[8787]::uses_target_feature[0]) ("return type"): bad type [type error] --> /home/joshua/src/rust/src/test/rustdoc/doc-cfg.rs:51:1 | 51 | / pub unsafe fn uses_target_feature() { 52 | | content::should::be::irrelevant(); 53 | | } | |_^ ``` Fortunately, rustdoc does not need to access MIR in order to generate documentation. Therefore this also removes the call to `analyze()` in `rustdoc::run_core`. This has the side effect of not generating all lints by default. Most lints are safe to ignore (does rustdoc really need to run liveness analysis?) but `missing_docs` in particular is disabled when it should not be. Re-running `missing_docs` specifically does not help, because it causes the typechecking pass to be run, bringing back the errors from rust-lang#24658: ``` error[E0599]: no method named `into_handle` found for struct `sys::unix::pipe::AnonPipe` in the current scope --> src/libstd/sys/windows/ext/process.rs:71:27 | 71 | self.into_inner().into_handle().into_raw() as *mut _ | ^^^^^^^^^^^ method not found in `sys::unix::pipe::AnonPipe` | ``` Because of rust-lang#73743, we only run typeck on demand. So this only causes an issue for functions returning `impl Trait`, which were already special cased by `ReplaceFunctionWithBody`. However, it now considers `async fn f() -> T` to be considered `impl Future<Output = T>`, where before it was considered to have a concrete `T` type. ## How will this affect future changes to rustdoc? - Any new changes to rustdoc will not be able to perform type checking without bringing back resolution errors in function bodies. + As a corollary, any new lints cannot require or perform type checking. In some cases this may require refactoring other parts of the compiler to perform type-checking only on-demand, see for example rust-lang#73743. + As a corollary, rustdoc can never again call `tcx.analysis()` unless this PR is reverted altogether. ## Current status - ~~I am not yet sure how to bring back `missing_docs` without running typeck. @eddyb suggested allowing lints to opt-out of type-checking, which would probably be another rabbit hole.~~ The opt-out was implemented in rust-lang#73743. However, of the rustc lints, now _only_ missing_docs is run and no other lints: rust-lang#73566 (comment). We need a team decision on whether that's an acceptable tradeoff. Note that all rustdoc lints are still run (`intra_doc_link_resolution_failure`, etc). **UPDATE**: This was deemed acceptable in rust-lang#73566 (comment) - ~~The implementation of optional errors in `rustc_resolve` is very brute force, it should probably be moved from `LateResolver` to `Resolver` to avoid duplicating the logic in many places.~~ I'm mostly happy with it now. - This no longer allows errors in `async fn f() -> T`. This caused breakage in 50 crates out of a full crater run, all of which (that I looked at) didn't compile when run with rustc directly. In other words, it used to be that they could not be compiled but could still be documented; now they can't be documented either. This needs a decision from the rustdoc team on whether this is acceptable breakage. **UPDATE**: This was deemed acceptable in rust-lang#73566 (comment) - ~~This makes `fn typeck_tables_of` in `rustc_typeck` public. This is not desired behavior, but needs the changes from rust-lang#74070 in order to be fixed.~~ Reverted.

@eddyb

Don't run `everybody_loops` for rustdoc; instead ignore resolution errors r? @eddyb cc @petrochenkov, @GuillaumeGomez, @Manishearth, @ecstatic-morse, @marmeladema ~~Blocked on rust-lang#73743 Merged. ~~Blocked on crater run.~~ Crater popped up some ICEs ([now fixed](rust-lang#73566 (comment))). See [crater run](https://crater-reports.s3.amazonaws.com/pr-73566/index.html), [ICEs](rust-lang#73566 (comment)). ~~Blocked on rust-lang#74070 so that we don't make typeck_tables_of public when it shouldn't be.~~ Merged. Closes rust-lang#71820, closes rust-lang#71104, closes rust-lang#65863. ## What is the motivation for this change? As seen from a lengthy trail of PRs and issues (rust-lang#73532, rust-lang#73103, rust-lang#71820, rust-lang#71104), `everybody_loops` is causing bugs in rustdoc. The main issue is that it does not preserve the validity of the `DefId` tree, meaning that operations on DefIds may unexpectedly fail when called later. This is blocking intra-doc links (see rust-lang#73101). This PR starts by removing `everybody_loops`, fixing rust-lang#71104 and rust-lang#71820. However, that brings back the bugs seen originally in rust-lang#43348: Since libstd documents items for all platforms, the function bodies sometimes do not type check. Here are the errors from documenting `libstd` with `everybody_loops` disabled and no other changes: ```rust error[E0433]: failed to resolve: could not find `handle` in `sys` --> src/libstd/sys/windows/ext/process.rs:13:27 | 13 | let handle = sys::handle::Handle::new(handle as *mut _); | ^^^^^^ could not find `handle` in `sys` error[E0425]: cannot find function `symlink_inner` in module `sys::fs` --> src/libstd/sys/windows/ext/fs.rs:544:14 | 544 | sys::fs::symlink_inner(src.as_ref(), dst.as_ref(), false) | ^^^^^^^^^^^^^ not found in `sys::fs` error[E0425]: cannot find function `symlink_inner` in module `sys::fs` --> src/libstd/sys/windows/ext/fs.rs:564:14 | 564 | sys::fs::symlink_inner(src.as_ref(), dst.as_ref(), true) | ^^^^^^^^^^^^^ not found in `sys::fs` ``` ## Why does this need changes to `rustc_resolve`? Normally, this could be avoided by simply not calling the `typeck_item_bodies` pass. However, the errors above happen before type checking, in name resolution itself. Since name resolution is intermingled with macro expansion, and rustdoc needs expansion to happen before it knows all items to be documented, there needs to be someway to ignore _resolution_ errors in function bodies. An alternative solution suggested by @petrochenkov was to not run `everybody_loops` on anything containing a nested `DefId`. This would solve some of the immediate issues, but isn't bullet-proof: the following functions still could not be documented if the items in the body failed to resolve: - Functions containing a nested `DefId` (rust-lang#71104) - ~~Functions returning `impl Trait` (rust-lang#43878 These ended up not resolving anyway with this PR. - ~~`const fn`, because `loop {}` in `const fn` is unstable (rust-lang#43636 `const_loop` was just stabilized. This also isn't exactly what rustdoc wants, which is to avoid looking at function bodies in the first place. ## What changes were made? The hack implemented in this PR is to add an option to ignore all resolution errors in function bodies. This is enabled only for rustdoc. Since resolution errors are ignored, the MIR generated will be invalid, as can be seen in the following ICE: ```rust error: internal compiler error: broken MIR in DefId(0:11 ~ doc_cfg[8787]::uses_target_feature[0]) ("return type"): bad type [type error] --> /home/joshua/src/rust/src/test/rustdoc/doc-cfg.rs:51:1 | 51 | / pub unsafe fn uses_target_feature() { 52 | | content::should::be::irrelevant(); 53 | | } | |_^ ``` Fortunately, rustdoc does not need to access MIR in order to generate documentation. Therefore this also removes the call to `analyze()` in `rustdoc::run_core`. This has the side effect of not generating all lints by default. Most lints are safe to ignore (does rustdoc really need to run liveness analysis?) but `missing_docs` in particular is disabled when it should not be. Re-running `missing_docs` specifically does not help, because it causes the typechecking pass to be run, bringing back the errors from rust-lang#24658: ``` error[E0599]: no method named `into_handle` found for struct `sys::unix::pipe::AnonPipe` in the current scope --> src/libstd/sys/windows/ext/process.rs:71:27 | 71 | self.into_inner().into_handle().into_raw() as *mut _ | ^^^^^^^^^^^ method not found in `sys::unix::pipe::AnonPipe` | ``` Because of rust-lang#73743, we only run typeck on demand. So this only causes an issue for functions returning `impl Trait`, which were already special cased by `ReplaceFunctionWithBody`. However, it now considers `async fn f() -> T` to be considered `impl Future<Output = T>`, where before it was considered to have a concrete `T` type. ## How will this affect future changes to rustdoc? - Any new changes to rustdoc will not be able to perform type checking without bringing back resolution errors in function bodies. + As a corollary, any new lints cannot require or perform type checking. In some cases this may require refactoring other parts of the compiler to perform type-checking only on-demand, see for example rust-lang#73743. + As a corollary, rustdoc can never again call `tcx.analysis()` unless this PR is reverted altogether. ## Current status - ~~I am not yet sure how to bring back `missing_docs` without running typeck. @eddyb suggested allowing lints to opt-out of type-checking, which would probably be another rabbit hole.~~ The opt-out was implemented in rust-lang#73743. However, of the rustc lints, now _only_ missing_docs is run and no other lints: rust-lang#73566 (comment). We need a team decision on whether that's an acceptable tradeoff. Note that all rustdoc lints are still run (`intra_doc_link_resolution_failure`, etc). **UPDATE**: This was deemed acceptable in rust-lang#73566 (comment) - ~~The implementation of optional errors in `rustc_resolve` is very brute force, it should probably be moved from `LateResolver` to `Resolver` to avoid duplicating the logic in many places.~~ I'm mostly happy with it now. - This no longer allows errors in `async fn f() -> T`. This caused breakage in 50 crates out of a full crater run, all of which (that I looked at) didn't compile when run with rustc directly. In other words, it used to be that they could not be compiled but could still be documented; now they can't be documented either. This needs a decision from the rustdoc team on whether this is acceptable breakage. **UPDATE**: This was deemed acceptable in rust-lang#73566 (comment) - ~~This makes `fn typeck_tables_of` in `rustc_typeck` public. This is not desired behavior, but needs the changes from rust-lang#74070 in order to be fixed.~~ Reverted.

@eddyb

Don't run `everybody_loops` for rustdoc; instead ignore resolution errors r? @eddyb cc @petrochenkov, @GuillaumeGomez, @Manishearth, @ecstatic-morse, @marmeladema ~~Blocked on rust-lang#73743 Merged. ~~Blocked on crater run.~~ Crater popped up some ICEs ([now fixed](rust-lang#73566 (comment))). See [crater run](https://crater-reports.s3.amazonaws.com/pr-73566/index.html), [ICEs](rust-lang#73566 (comment)). ~~Blocked on rust-lang#74070 so that we don't make typeck_tables_of public when it shouldn't be.~~ Merged. Closes rust-lang#71820, closes rust-lang#71104, closes rust-lang#65863. ## What is the motivation for this change? As seen from a lengthy trail of PRs and issues (rust-lang#73532, rust-lang#73103, rust-lang#71820, rust-lang#71104), `everybody_loops` is causing bugs in rustdoc. The main issue is that it does not preserve the validity of the `DefId` tree, meaning that operations on DefIds may unexpectedly fail when called later. This is blocking intra-doc links (see rust-lang#73101). This PR starts by removing `everybody_loops`, fixing rust-lang#71104 and rust-lang#71820. However, that brings back the bugs seen originally in rust-lang#43348: Since libstd documents items for all platforms, the function bodies sometimes do not type check. Here are the errors from documenting `libstd` with `everybody_loops` disabled and no other changes: ```rust error[E0433]: failed to resolve: could not find `handle` in `sys` --> src/libstd/sys/windows/ext/process.rs:13:27 | 13 | let handle = sys::handle::Handle::new(handle as *mut _); | ^^^^^^ could not find `handle` in `sys` error[E0425]: cannot find function `symlink_inner` in module `sys::fs` --> src/libstd/sys/windows/ext/fs.rs:544:14 | 544 | sys::fs::symlink_inner(src.as_ref(), dst.as_ref(), false) | ^^^^^^^^^^^^^ not found in `sys::fs` error[E0425]: cannot find function `symlink_inner` in module `sys::fs` --> src/libstd/sys/windows/ext/fs.rs:564:14 | 564 | sys::fs::symlink_inner(src.as_ref(), dst.as_ref(), true) | ^^^^^^^^^^^^^ not found in `sys::fs` ``` ## Why does this need changes to `rustc_resolve`? Normally, this could be avoided by simply not calling the `typeck_item_bodies` pass. However, the errors above happen before type checking, in name resolution itself. Since name resolution is intermingled with macro expansion, and rustdoc needs expansion to happen before it knows all items to be documented, there needs to be someway to ignore _resolution_ errors in function bodies. An alternative solution suggested by @petrochenkov was to not run `everybody_loops` on anything containing a nested `DefId`. This would solve some of the immediate issues, but isn't bullet-proof: the following functions still could not be documented if the items in the body failed to resolve: - Functions containing a nested `DefId` (rust-lang#71104) - ~~Functions returning `impl Trait` (rust-lang#43878 These ended up not resolving anyway with this PR. - ~~`const fn`, because `loop {}` in `const fn` is unstable (rust-lang#43636 `const_loop` was just stabilized. This also isn't exactly what rustdoc wants, which is to avoid looking at function bodies in the first place. ## What changes were made? The hack implemented in this PR is to add an option to ignore all resolution errors in function bodies. This is enabled only for rustdoc. Since resolution errors are ignored, the MIR generated will be invalid, as can be seen in the following ICE: ```rust error: internal compiler error: broken MIR in DefId(0:11 ~ doc_cfg[8787]::uses_target_feature[0]) ("return type"): bad type [type error] --> /home/joshua/src/rust/src/test/rustdoc/doc-cfg.rs:51:1 | 51 | / pub unsafe fn uses_target_feature() { 52 | | content::should::be::irrelevant(); 53 | | } | |_^ ``` Fortunately, rustdoc does not need to access MIR in order to generate documentation. Therefore this also removes the call to `analyze()` in `rustdoc::run_core`. This has the side effect of not generating all lints by default. Most lints are safe to ignore (does rustdoc really need to run liveness analysis?) but `missing_docs` in particular is disabled when it should not be. Re-running `missing_docs` specifically does not help, because it causes the typechecking pass to be run, bringing back the errors from rust-lang#24658: ``` error[E0599]: no method named `into_handle` found for struct `sys::unix::pipe::AnonPipe` in the current scope --> src/libstd/sys/windows/ext/process.rs:71:27 | 71 | self.into_inner().into_handle().into_raw() as *mut _ | ^^^^^^^^^^^ method not found in `sys::unix::pipe::AnonPipe` | ``` Because of rust-lang#73743, we only run typeck on demand. So this only causes an issue for functions returning `impl Trait`, which were already special cased by `ReplaceFunctionWithBody`. However, it now considers `async fn f() -> T` to be considered `impl Future<Output = T>`, where before it was considered to have a concrete `T` type. ## How will this affect future changes to rustdoc? - Any new changes to rustdoc will not be able to perform type checking without bringing back resolution errors in function bodies. + As a corollary, any new lints cannot require or perform type checking. In some cases this may require refactoring other parts of the compiler to perform type-checking only on-demand, see for example rust-lang#73743. + As a corollary, rustdoc can never again call `tcx.analysis()` unless this PR is reverted altogether. ## Current status - ~~I am not yet sure how to bring back `missing_docs` without running typeck. @eddyb suggested allowing lints to opt-out of type-checking, which would probably be another rabbit hole.~~ The opt-out was implemented in rust-lang#73743. However, of the rustc lints, now _only_ missing_docs is run and no other lints: rust-lang#73566 (comment). We need a team decision on whether that's an acceptable tradeoff. Note that all rustdoc lints are still run (`intra_doc_link_resolution_failure`, etc). **UPDATE**: This was deemed acceptable in rust-lang#73566 (comment) - ~~The implementation of optional errors in `rustc_resolve` is very brute force, it should probably be moved from `LateResolver` to `Resolver` to avoid duplicating the logic in many places.~~ I'm mostly happy with it now. - This no longer allows errors in `async fn f() -> T`. This caused breakage in 50 crates out of a full crater run, all of which (that I looked at) didn't compile when run with rustc directly. In other words, it used to be that they could not be compiled but could still be documented; now they can't be documented either. This needs a decision from the rustdoc team on whether this is acceptable breakage. **UPDATE**: This was deemed acceptable in rust-lang#73566 (comment) - ~~This makes `fn typeck_tables_of` in `rustc_typeck` public. This is not desired behavior, but needs the changes from rust-lang#74070 in order to be fixed.~~ Reverted.

@eddyb

Don't run `everybody_loops` for rustdoc; instead ignore resolution errors r? @eddyb cc @petrochenkov, @GuillaumeGomez, @Manishearth, @ecstatic-morse, @marmeladema ~~Blocked on rust-lang#73743 Merged. ~~Blocked on crater run.~~ Crater popped up some ICEs ([now fixed](rust-lang#73566 (comment))). See [crater run](https://crater-reports.s3.amazonaws.com/pr-73566/index.html), [ICEs](rust-lang#73566 (comment)). ~~Blocked on rust-lang#74070 so that we don't make typeck_tables_of public when it shouldn't be.~~ Merged. Closes rust-lang#71820, closes rust-lang#71104, closes rust-lang#65863. ## What is the motivation for this change? As seen from a lengthy trail of PRs and issues (rust-lang#73532, rust-lang#73103, rust-lang#71820, rust-lang#71104), `everybody_loops` is causing bugs in rustdoc. The main issue is that it does not preserve the validity of the `DefId` tree, meaning that operations on DefIds may unexpectedly fail when called later. This is blocking intra-doc links (see rust-lang#73101). This PR starts by removing `everybody_loops`, fixing rust-lang#71104 and rust-lang#71820. However, that brings back the bugs seen originally in rust-lang#43348: Since libstd documents items for all platforms, the function bodies sometimes do not type check. Here are the errors from documenting `libstd` with `everybody_loops` disabled and no other changes: ```rust error[E0433]: failed to resolve: could not find `handle` in `sys` --> src/libstd/sys/windows/ext/process.rs:13:27 | 13 | let handle = sys::handle::Handle::new(handle as *mut _); | ^^^^^^ could not find `handle` in `sys` error[E0425]: cannot find function `symlink_inner` in module `sys::fs` --> src/libstd/sys/windows/ext/fs.rs:544:14 | 544 | sys::fs::symlink_inner(src.as_ref(), dst.as_ref(), false) | ^^^^^^^^^^^^^ not found in `sys::fs` error[E0425]: cannot find function `symlink_inner` in module `sys::fs` --> src/libstd/sys/windows/ext/fs.rs:564:14 | 564 | sys::fs::symlink_inner(src.as_ref(), dst.as_ref(), true) | ^^^^^^^^^^^^^ not found in `sys::fs` ``` ## Why does this need changes to `rustc_resolve`? Normally, this could be avoided by simply not calling the `typeck_item_bodies` pass. However, the errors above happen before type checking, in name resolution itself. Since name resolution is intermingled with macro expansion, and rustdoc needs expansion to happen before it knows all items to be documented, there needs to be someway to ignore _resolution_ errors in function bodies. An alternative solution suggested by @petrochenkov was to not run `everybody_loops` on anything containing a nested `DefId`. This would solve some of the immediate issues, but isn't bullet-proof: the following functions still could not be documented if the items in the body failed to resolve: - Functions containing a nested `DefId` (rust-lang#71104) - ~~Functions returning `impl Trait` (rust-lang#43878 These ended up not resolving anyway with this PR. - ~~`const fn`, because `loop {}` in `const fn` is unstable (rust-lang#43636 `const_loop` was just stabilized. This also isn't exactly what rustdoc wants, which is to avoid looking at function bodies in the first place. ## What changes were made? The hack implemented in this PR is to add an option to ignore all resolution errors in function bodies. This is enabled only for rustdoc. Since resolution errors are ignored, the MIR generated will be invalid, as can be seen in the following ICE: ```rust error: internal compiler error: broken MIR in DefId(0:11 ~ doc_cfg[8787]::uses_target_feature[0]) ("return type"): bad type [type error] --> /home/joshua/src/rust/src/test/rustdoc/doc-cfg.rs:51:1 | 51 | / pub unsafe fn uses_target_feature() { 52 | | content::should::be::irrelevant(); 53 | | } | |_^ ``` Fortunately, rustdoc does not need to access MIR in order to generate documentation. Therefore this also removes the call to `analyze()` in `rustdoc::run_core`. This has the side effect of not generating all lints by default. Most lints are safe to ignore (does rustdoc really need to run liveness analysis?) but `missing_docs` in particular is disabled when it should not be. Re-running `missing_docs` specifically does not help, because it causes the typechecking pass to be run, bringing back the errors from rust-lang#24658: ``` error[E0599]: no method named `into_handle` found for struct `sys::unix::pipe::AnonPipe` in the current scope --> src/libstd/sys/windows/ext/process.rs:71:27 | 71 | self.into_inner().into_handle().into_raw() as *mut _ | ^^^^^^^^^^^ method not found in `sys::unix::pipe::AnonPipe` | ``` Because of rust-lang#73743, we only run typeck on demand. So this only causes an issue for functions returning `impl Trait`, which were already special cased by `ReplaceFunctionWithBody`. However, it now considers `async fn f() -> T` to be considered `impl Future<Output = T>`, where before it was considered to have a concrete `T` type. ## How will this affect future changes to rustdoc? - Any new changes to rustdoc will not be able to perform type checking without bringing back resolution errors in function bodies. + As a corollary, any new lints cannot require or perform type checking. In some cases this may require refactoring other parts of the compiler to perform type-checking only on-demand, see for example rust-lang#73743. + As a corollary, rustdoc can never again call `tcx.analysis()` unless this PR is reverted altogether. ## Current status - ~~I am not yet sure how to bring back `missing_docs` without running typeck. @eddyb suggested allowing lints to opt-out of type-checking, which would probably be another rabbit hole.~~ The opt-out was implemented in rust-lang#73743. However, of the rustc lints, now _only_ missing_docs is run and no other lints: rust-lang#73566 (comment). We need a team decision on whether that's an acceptable tradeoff. Note that all rustdoc lints are still run (`intra_doc_link_resolution_failure`, etc). **UPDATE**: This was deemed acceptable in rust-lang#73566 (comment) - ~~The implementation of optional errors in `rustc_resolve` is very brute force, it should probably be moved from `LateResolver` to `Resolver` to avoid duplicating the logic in many places.~~ I'm mostly happy with it now. - This no longer allows errors in `async fn f() -> T`. This caused breakage in 50 crates out of a full crater run, all of which (that I looked at) didn't compile when run with rustc directly. In other words, it used to be that they could not be compiled but could still be documented; now they can't be documented either. This needs a decision from the rustdoc team on whether this is acceptable breakage. **UPDATE**: This was deemed acceptable in rust-lang#73566 (comment) - ~~This makes `fn typeck_tables_of` in `rustc_typeck` public. This is not desired behavior, but needs the changes from rust-lang#74070 in order to be fixed.~~ Reverted.

steveklabnik added the A-build label Apr 21, 2015

steveklabnik added the T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue. label Apr 21, 2015

steveklabnik added the I-nominated label Apr 21, 2015

pnkfelix added P-medium Medium priority and removed I-nominated labels Apr 23, 2015

pnkfelix assigned brson Apr 23, 2015

kamalmarhubi mentioned this issue Feb 19, 2016

Documentation lacks modules that depend on compile-time features. nix-rust/nix#206

Closed

brson removed their assignment Jul 11, 2016

retep998 mentioned this issue Sep 30, 2016

Consider hosting std docs rust-lang/docs.rs#56

Closed

retep998 mentioned this issue Oct 19, 2016

Rust, Windows, and MSVC rust-lang/rfcs#1061

Closed

47 tasks

steveklabnik added the T-tools label Mar 9, 2017

steveklabnik added T-dev-tools Relevant to the dev-tools subteam, which will review and decide on the PR/issue. and removed T-tools labels May 18, 2017

frewsxcv added the T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue. label Jun 2, 2017

kennytm mentioned this issue Jul 20, 2017

Expose all OS-specific modules in libstd doc. #43348

Merged

Mark-Simulacrum added C-feature-request Category: A feature request, i.e: not implemented / a PR. C-bug Category: This is a bug. and removed C-feature-request Category: A feature request, i.e: not implemented / a PR. labels Jul 22, 2017

bors closed this as completed in #43348 Aug 13, 2017

jyn514 mentioned this issue Jun 20, 2020

Don't run everybody_loops for rustdoc; instead ignore resolution errors #73566

Merged

jyn514 mentioned this issue Jun 29, 2020

Some NodeId/LocalDefId don't have a corresponding HirId #71104

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

std::os::windows missing on http://doc.rust-lang.org/nightly/std/os/ #24658

std::os::windows missing on http://doc.rust-lang.org/nightly/std/os/ #24658

SimonSapin commented Apr 21, 2015

steveklabnik commented Apr 21, 2015

nwin commented Apr 21, 2015

klutzy commented Apr 21, 2015

SimonSapin commented Apr 21, 2015

SimonSapin commented Apr 21, 2015

steveklabnik commented Apr 21, 2015

nwin commented Apr 21, 2015

lambda commented Apr 21, 2015

lambda commented Apr 21, 2015

nwin commented Apr 21, 2015

SimonSapin commented Apr 21, 2015

lambda commented Apr 21, 2015

SimonSapin commented Apr 21, 2015

steveklabnik commented Apr 22, 2015

pnkfelix commented Apr 23, 2015

pnkfelix commented Apr 23, 2015

brson commented Apr 23, 2015

KostaCitrix commented Jul 13, 2015

steveklabnik commented Aug 1, 2015

retep998 commented Sep 7, 2016

Diggsey commented Sep 7, 2016

eminence commented Sep 30, 2016

std::os::windows missing on http://doc.rust-lang.org/nightly/std/os/ #24658

std::os::windows missing on http://doc.rust-lang.org/nightly/std/os/ #24658

Comments

SimonSapin commented Apr 21, 2015

steveklabnik commented Apr 21, 2015

nwin commented Apr 21, 2015

klutzy commented Apr 21, 2015

SimonSapin commented Apr 21, 2015

SimonSapin commented Apr 21, 2015

steveklabnik commented Apr 21, 2015

nwin commented Apr 21, 2015

lambda commented Apr 21, 2015

lambda commented Apr 21, 2015

nwin commented Apr 21, 2015

SimonSapin commented Apr 21, 2015

lambda commented Apr 21, 2015

SimonSapin commented Apr 21, 2015

steveklabnik commented Apr 22, 2015

pnkfelix commented Apr 23, 2015

pnkfelix commented Apr 23, 2015

brson commented Apr 23, 2015

KostaCitrix commented Jul 13, 2015

steveklabnik commented Aug 1, 2015

retep998 commented Sep 7, 2016

Diggsey commented Sep 7, 2016

eminence commented Sep 30, 2016