Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scrape code examples from examples/ directory for Rustdoc #85833

Merged
merged 24 commits into from
Oct 23, 2021
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
4b3f82a
Add updated support for example-analyzer
willcrichton May 9, 2021
7831fee
Fix check issue
willcrichton May 30, 2021
2855bf0
Factor scraping and rendering into separate calls to rustdoc
willcrichton Jun 1, 2021
b6338e7
Generate example source files with corresponding links
willcrichton Jun 3, 2021
eea8f0a
Sort examples by size
willcrichton Aug 26, 2021
55bb517
Move highlighting logic from JS to Rust
willcrichton Aug 26, 2021
18edcf8
Reduce blur size, fix example width bug, add better error handling fo…
willcrichton Sep 14, 2021
a1cb194
Add styles for non-white themes
willcrichton Sep 14, 2021
829b1a9
Incorporate jyn's feedback
willcrichton Sep 17, 2021
5c05b3c
Add target crates as inputs to reduce size of intermediates
willcrichton Sep 20, 2021
df5e3a6
Change serialized format to use DefPathHash instead of custom String
willcrichton Sep 21, 2021
25323ec
Move JS into a standalone file
willcrichton Oct 1, 2021
55731bb
Fix lint error, change scrape-examples.js minify call
willcrichton Oct 1, 2021
ed8e12f
Unversioned -> InvocationSpecific
willcrichton Oct 1, 2021
5584c79
Update to latest rustc and rustdoc styles
willcrichton Oct 7, 2021
bb383ed
Move some expansion logic into generation-time, fix section header li…
willcrichton Oct 7, 2021
e22e858
Move more scrape-examples logic from JS to rust
willcrichton Oct 7, 2021
f10dcee
Change handling of spans in scrape examples, add test for highlight d…
willcrichton Oct 8, 2021
9e4958a
Add test for prev/back arrows + examples across multiple files
willcrichton Oct 9, 2021
b1616f3
Add test for ordering of examples, simplify with single scrape.mk file
willcrichton Oct 9, 2021
24a71cb
Fix local crate not being scraped
willcrichton Oct 13, 2021
8f80d86
Small scrape-example fixes
willcrichton Oct 20, 2021
d1c29c6
Revert def_id addition from clean::Function, add test for
willcrichton Oct 22, 2021
fd5d614
Move def_id logic into render_call_locations
willcrichton Oct 22, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 12 additions & 2 deletions src/doc/rustdoc/src/unstable-features.md
Original file line number Diff line number Diff line change
Expand Up @@ -458,7 +458,13 @@ Calculating code examples follows these rules:

### `--with-examples`: include examples of uses of items as documentation

This option, combined with `--scrape-examples-target-crate` and `--scrape-examples-output-path`, is used to implement the functionality in [RFC #3123](https://github.com/rust-lang/rfcs/pull/3123). Uses of an item (currently functions / call-sites) are found in a crate and its reverse-dependencies, and then the uses are included as documentation for that item. This feature is intended to be used via `cargo doc --scrape-examples`, but the rustdoc-only workflow looks like:
This option, combined with `--scrape-examples-target-crate` and
`--scrape-examples-output-path`, is used to implement the functionality in [RFC
#3123](https://github.com/rust-lang/rfcs/pull/3123). Uses of an item (currently
functions / call-sites) are found in a crate and its reverse-dependencies, and
then the uses are included as documentation for that item. This feature is
intended to be used via `cargo doc --scrape-examples`, but the rustdoc-only
workflow looks like:

```bash
$ rustdoc examples/ex.rs -Z unstable-options \
Expand All @@ -468,4 +474,8 @@ $ rustdoc examples/ex.rs -Z unstable-options \
$ rustdoc src/lib.rs -Z unstable-options --with-examples output.calls
```

First, the library must be checked to generate an `rmeta`. Then a reverse-dependency like `examples/ex.rs` is given to rustdoc with the target crate being documented (`foobar`) and a path to output the calls (`output.calls`). Then, the generated calls file can be passed via `--with-examples` to the subsequent documentation of `foobar`.
First, the library must be checked to generate an `rmeta`. Then a
reverse-dependency like `examples/ex.rs` is given to rustdoc with the target
crate being documented (`foobar`) and a path to output the calls
(`output.calls`). Then, the generated calls file can be passed via
`--with-examples` to the subsequent documentation of `foobar`.
1 change: 0 additions & 1 deletion src/librustdoc/clean/inline.rs
Original file line number Diff line number Diff line change
Expand Up @@ -235,7 +235,6 @@ fn build_external_function(cx: &mut DocContext<'_>, did: DefId) -> clean::Functi
decl,
generics,
header: hir::FnHeader { unsafety: sig.unsafety(), abi: sig.abi(), constness, asyncness },
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note to self: get rid of this later, it can be calculated on-demand from the DefId.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Open an issue to not forget it then. ;)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

def_id: did,
}
}

Expand Down
9 changes: 2 additions & 7 deletions src/librustdoc/clean/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -801,8 +801,7 @@ impl<'a> Clean<Function> for (&'a hir::FnSig<'a>, &'a hir::Generics<'a>, hir::Bo
fn clean(&self, cx: &mut DocContext<'_>) -> Function {
let (generics, decl) =
enter_impl_trait(cx, |cx| (self.1.clean(cx), (&*self.0.decl, self.2).clean(cx)));
let def_id = self.2.hir_id.owner.to_def_id();
Function { decl, generics, header: self.0.header, def_id }
Function { decl, generics, header: self.0.header }
}
}

Expand Down Expand Up @@ -934,8 +933,7 @@ impl Clean<Item> for hir::TraitItem<'_> {
let (generics, decl) = enter_impl_trait(cx, |cx| {
(self.generics.clean(cx), (&*sig.decl, &names[..]).clean(cx))
});
let def_id = self.def_id.to_def_id();
let mut t = Function { header: sig.header, decl, generics, def_id };
let mut t = Function { header: sig.header, decl, generics };
if t.header.constness == hir::Constness::Const
&& is_unstable_const_fn(cx.tcx, local_did).is_some()
{
Expand Down Expand Up @@ -1069,7 +1067,6 @@ impl Clean<Item> for ty::AssocItem {
constness,
asyncness,
},
def_id: self.def_id,
},
defaultness,
)
Expand All @@ -1083,7 +1080,6 @@ impl Clean<Item> for ty::AssocItem {
constness: hir::Constness::NotConst,
asyncness: hir::IsAsync::NotAsync,
},
def_id: self.def_id,
})
}
}
Expand Down Expand Up @@ -2103,7 +2099,6 @@ impl Clean<Item> for (&hir::ForeignItem<'_>, Option<Symbol>) {
constness: hir::Constness::NotConst,
asyncness: hir::IsAsync::NotAsync,
},
def_id,
})
}
hir::ForeignItemKind::Static(ref ty, mutability) => {
Expand Down
1 change: 0 additions & 1 deletion src/librustdoc/clean/types.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1254,7 +1254,6 @@ crate struct Function {
crate decl: FnDecl,
crate generics: Generics,
crate header: hir::FnHeader,
crate def_id: DefId,
}

#[derive(Clone, PartialEq, Eq, Debug, Hash)]
Expand Down
7 changes: 3 additions & 4 deletions src/librustdoc/html/render/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -597,11 +597,10 @@ fn document_full_inner(
clean::ItemKind::StrippedItem(box kind) | kind => kind,
};

match kind {
clean::ItemKind::FunctionItem(f) | clean::ItemKind::MethodItem(f, _) => {
render_call_locations(w, cx, f.def_id, item);
if let clean::ItemKind::FunctionItem(..) | clean::ItemKind::MethodItem(..) = kind {
if let Some(def_id) = item.def_id.as_def_id() {
willcrichton marked this conversation as resolved.
Show resolved Hide resolved
render_call_locations(w, cx, def_id, item);
}
_ => {}
}
}

Expand Down
4 changes: 2 additions & 2 deletions src/librustdoc/json/conversions.rs
Original file line number Diff line number Diff line change
Expand Up @@ -289,7 +289,7 @@ crate fn from_fn_header(header: &rustc_hir::FnHeader) -> HashSet<Qualifiers> {

impl FromWithTcx<clean::Function> for Function {
fn from_tcx(function: clean::Function, tcx: TyCtxt<'_>) -> Self {
let clean::Function { decl, generics, header, def_id: _ } = function;
let clean::Function { decl, generics, header } = function;
Function {
decl: decl.into_tcx(tcx),
generics: generics.into_tcx(tcx),
Expand Down Expand Up @@ -530,7 +530,7 @@ crate fn from_function_method(
has_body: bool,
tcx: TyCtxt<'_>,
) -> Method {
let clean::Function { header, decl, generics, def_id: _ } = function;
let clean::Function { header, decl, generics } = function;
Method {
decl: decl.into_tcx(tcx),
generics: generics.into_tcx(tcx),
Expand Down
1 change: 1 addition & 0 deletions src/test/rustdoc-ui/scrape-examples-wrong-options-1.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
// compile-flags: -Z unstable-options --scrape-examples-target-crate foobar
2 changes: 2 additions & 0 deletions src/test/rustdoc-ui/scrape-examples-wrong-options-1.stderr
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
error: must use --scrape-examples-output-path and --scrape-examples-target-crate together

1 change: 1 addition & 0 deletions src/test/rustdoc-ui/scrape-examples-wrong-options-2.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
// compile-flags: -Z unstable-options --scrape-examples-output-path ex.calls
2 changes: 2 additions & 0 deletions src/test/rustdoc-ui/scrape-examples-wrong-options-2.stderr
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
error: must use --scrape-examples-output-path and --scrape-examples-target-crate together

1 change: 0 additions & 1 deletion src/test/rustdoc-ui/scrape-examples-wrong-options.rs

This file was deleted.

2 changes: 0 additions & 2 deletions src/test/rustdoc-ui/scrape-examples-wrong-options.stderr

This file was deleted.