Add some timing info to rustdoc

[jyn514]

There are various improvements, but the main one is to time each pass
that rustdoc performs (rustdoc::passes).

Before, these were the top five timings for cargo doc on the cargo
repository:

+---------------------------------+-----------+-----------------+----------+------------+
| Item                            | Self time | % of total time | Time     | Item count |
+---------------------------------+-----------+-----------------+----------+------------+
| <unknown>                       | 854.70ms  | 20.888          | 2.47s    | 744823     |
+---------------------------------+-----------+-----------------+----------+------------+
| expand_crate                    | 795.29ms  | 19.436          | 848.00ms | 1          |
+---------------------------------+-----------+-----------------+----------+------------+
| metadata_decode_entry           | 256.73ms  | 6.274           | 279.49ms | 518344     |
+---------------------------------+-----------+-----------------+----------+------------+
| resolve_crate                   | 240.56ms  | 5.879           | 242.86ms | 1          |
+---------------------------------+-----------+-----------------+----------+------------+
| hir_lowering                    | 146.79ms  | 3.587           | 146.79ms | 1          |
+---------------------------------+-----------+-----------------+----------+------------+

Now the timings are:

+---------------------------------+-----------+-----------------+----------+------------+
| Item                            | Self time | % of total time | Time     | Item count |
+---------------------------------+-----------+-----------------+----------+------------+
| <unknown>                       | 1.40s     | 22.662          | 3.73s    | 771430     |
+---------------------------------+-----------+-----------------+----------+------------+
| collect-trait-impls             | 1.34s     | 21.672          | 2.87s    | 1          |
+---------------------------------+-----------+-----------------+----------+------------+
| expand_crate                    | 1.21s     | 19.577          | 1.28s    | 1          |
+---------------------------------+-----------+-----------------+----------+------------+
| build extern trait impl         | 704.66ms  | 11.427          | 1.07s    | 21893      |
+---------------------------------+-----------+-----------------+----------+------------+
| metadata_decode_entry           | 354.84ms  | 5.754           | 391.81ms | 544919     |
+---------------------------------+-----------+-----------------+----------+------------+

The goal is to help me debug regressions like #74518 (comment) (currently I have no idea what could have gone wrong).

r? @eddyb or @Mark-Simulacrum

[Mark-Simulacrum]

Do we have some idea where the unknown is coming from? The time spent in it seems to have ... gone up?

[jyn514]

@Mark-Simulacrum the overall time went up as well, I think because of one of the regressions in https://github.com/rust-lang/rustc-perf/blob/master/triage/2020-07-21.md. I would look at the percentages instead.

[jyn514]

I'm not sure where the <unknown> is coming from - I tried to wrap all of rustdoc::core in a timer but that didn't work because the main function doesn't have access to a compile session.

[bors]

☔ The latest upstream changes (presumably #75008) made this pull request unmergeable. Please resolve the merge conflicts.

[eddyb]

@jyn514 You don't need to wrap the entirety of rustdoc, just the outermost thing that has access to a timer/profiler.

r? @Mark-Simulacrum or @wesleywiser

[eddyb]

Note: not a bad change, but would not have been marked as <unknown> before (it would show up as part of typeck).

[eddyb]

At the very least, it looks like the entirety of global_ctxt.enter(|tcx| { could be wrapped in a named region (if not more).

[jyn514]

Addressed review comments. I separated the enormous closure into a separate function, so best to view with whitespace changes disabled. Building rustdoc now to see if that got rid of most of the <unknown>.

[jyn514]

Timing the global context didn't seem to help much.

+-------------------------------------------------+-----------+-----------------+----------+------------+
| Item                                            | Self time | % of total time | Time     | Item count |
+-------------------------------------------------+-----------+-----------------+----------+------------+
| collect-trait-impls                             | 1.46s     | 23.619          | 3.26s    | 1          |
+-------------------------------------------------+-----------+-----------------+----------+------------+
| expand_crate                                    | 1.20s     | 19.538          | 1.25s    | 1          |
+-------------------------------------------------+-----------+-----------------+----------+------------+
| <unknown>                                       | 1.09s     | 17.746          | 3.17s    | 753829     |
+-------------------------------------------------+-----------+-----------------+----------+------------+
| run_global_ctxt                                 | 3.34ms    | 0.054           | 4.24s    | 1          |
+-------------------------------------------------+-----------+-----------------+----------+------------+

[jyn514]

I did find this comment though:

rust/src/librustc_codegen_ssa/base.rs

Lines 782 to 792 in 62cf767

	fn finalize_tcx(tcx: TyCtxt<'_>) {
	tcx.sess.time("assert_dep_graph", || ::rustc_incremental::assert_dep_graph(tcx));
	tcx.sess.time("serialize_dep_graph", || ::rustc_incremental::save_dep_graph(tcx));
	// We assume that no queries are run past here. If there are new queries
	// after this point, they'll show up as "<unknown>" in self-profiling data.
	{
	let _prof_timer = tcx.prof.generic_activity("self_profile_alloc_query_strings");
	tcx.alloc_self_profile_query_strings();
	}
	}

[jyn514]

Aha, I think I found the issue: all of the html rendering isn't timed.

rust/src/librustdoc/lib.rs

Lines 525 to 533 in 1d69e3b

	let diag = core::new_handler(error_format, None, &debugging_options);
	match output_format {
	None | Some(config::OutputFormat::Html) => {
	run_renderer::<html::render::Context>(krate, renderopts, renderinfo, &diag, edition)
	}
	Some(config::OutputFormat::Json) => {
	run_renderer::<json::JsonRenderer>(krate, renderopts, renderinfo, &diag, edition)
	}
	}

[jyn514]

Well, that helped a little.

$ summarize summarize time-html/unknown-crate-3300.events 
+-------------------------------------------------+-----------+-----------------+----------+------------+
| Item                                            | Self time | % of total time | Time     | Item count |
+-------------------------------------------------+-----------+-----------------+----------+------------+
| collect-trait-impls                             | 1.46s     | 21.568          | 3.29s    | 1          |
+-------------------------------------------------+-----------+-----------------+----------+------------+
| expand_crate                                    | 1.22s     | 18.103          | 1.27s    | 1          |
+-------------------------------------------------+-----------+-----------------+----------+------------+
| <unknown>                                       | 1.12s     | 16.504          | 3.35s    | 753829     |
+-------------------------------------------------+-----------+-----------------+----------+------------+
| build extern trait impl                         | 822.75ms  | 12.162          | 1.34s    | 21968      |
+-------------------------------------------------+-----------+-----------------+----------+------------+
| render html                                     | 439.39ms  | 6.495           | 439.39ms | 1          |
+-------------------------------------------------+-----------+-----------------+----------+------------+

[jyn514]

It's coming from item_types_checking.

rustdoc.zip

[jyn514]

Anyone know why this might be getting the timing name reset?

    tcx.sess.time("item_types_checking", || {
        for &module in tcx.hir().krate().modules.keys() {
            tcx.sess.time("check_mod_item_types", || {
            tcx.ensure().check_mod_item_types(tcx.hir().local_def_id(module));
            });
        }
    });

check_mod_item_types works fine and shows almost exactly 0 time spent in it, instead there's half a second of <unknown>.
https://github.com/rust-lang/rust/pull/74590/files?diff=unified&w=1#diff-0de644786a2f1fd88c7d7f44bc3fa4bbR497

[Mark-Simulacrum]

I would personally like us to track down the problem with unknown timing before merging this in, though I don't have great ideas for doing that. Maybe @wesleywiser has suggestions?

[bors]

☔ The latest upstream changes (presumably #74489) made this pull request unmergeable. Please resolve the merge conflicts.

[jyn514]

@Mark-Simulacrum I don't expect to have time to debug this soon ... are you okay with merging the current changes and opening an issue for the <unknown>? I'd rather have some timing than having to guess.

[wesleywiser]

I've only skimmed the PR but I think we should not block merging this with resolving the unknown issue. We've seen that before in other profiles so it's probably another instance of the existing issue. I hope to have some time this week to look into it.

[Mark-Simulacrum]

Okay, knowing that we've seen this before is indeed helpful and that convinces me that it should be fine to move ahead here.

@bors r+

[bors]

📌 Commit 6e3e74d has been approved by Mark-Simulacrum

[bors]

⌛ Testing commit 6e3e74d with merge 0301700...

[bors]

☀️ Test successful - checks-actions, checks-azure
Approved by: Mark-Simulacrum
Pushing 0301700 to master...

[camelid]

You may already know this, but this seems to be the rustc code that produces the <unknown> name:

rust/compiler/rustc_codegen_llvm/src/debuginfo/metadata.rs

Line 795 in 192c7db

let file_name = file_name.as_deref().unwrap_or("<unknown>");

Not sure why it's happening though.

[jyn514]

That's under debuginfo, so I wouldn't expect it to be related.

I did notice a while back that rustdoc doesn't include the crate name in the .events file - I think that might be because doctree passes None for the crate name? I need to test it.
https://github.com/rust-lang/rust/blob/master/src/librustdoc/visit_ast.rs#L71

[jyn514]

I did notice a while back that rustdoc doesn't include the crate name in the .events file - I think that might be because doctree passes None for the crate name? I need to test it.

This was #79586.