Remove `prev_index_to_index` field from `CurrentDepGraph` #138824

Zoxc · 2025-03-22T05:36:01Z

The dep graph currently has 2 ways to map a previous index into a current index. The prev_index_to_index map stores the current index equivalent of a previous index. For indices which are marked green, we also store the same information in the DepNodeColorMap. We actually only need to known the mapping for green nodes however, so this PR removes prev_index_to_index and instead makes use of the DepNodeColorMap.

To avoid racing when promoting a node from the previous session, the encoder lock is now used to ensure only one thread encodes the promoted node. This was previously done by the lock in prev_index_to_index.

This also changes nodes_newly_allocated_in_current_session used to detect duplicate dep nodes to contain both new and previous nodes, which is simpler and can better catch duplicates.

The dep node index encoding used in DepNodeColorMap is tweak to avoid subtraction / addition to optimize accessing the current equivalent of a previous index.

r? @oli-obk

rustbot · 2025-03-22T05:36:07Z

Some changes occurred in compiler/rustc_codegen_ssa

cc @WaffleLapkin

oli-obk · 2025-03-24T08:43:00Z

@bors try @rust-timer queue

bors · 2025-03-24T08:48:27Z

⌛ Trying commit 59e27c8 with merge 17e10ad...

Remove `prev_index_to_index` field from `CurrentDepGraph` The dep graph currently has 2 ways to map a previous index into a current index. The `prev_index_to_index` map stores the current index equivalent of a previous index. For indices which are marked green, we also store the same information in the `DepNodeColorMap`. We actually only need to known the mapping for green nodes however, so this PR removes `prev_index_to_index` and instead makes use of the `DepNodeColorMap`. To avoid racing when promoting a node from the previous session, the encoder lock is now used to ensure only one thread encodes the promoted node. This was previously done by the lock in `prev_index_to_index`. This also changes `nodes_newly_allocated_in_current_session` used to detect duplicate dep nodes to contain both new and previous nodes, which is simpler and can better catch duplicates. The dep node index encoding used in `DepNodeColorMap` is tweak to avoid subtraction / addition to optimize accessing the current equivalent of a previous index. This is based on rust-lang#138629. r? `@oli-obk`

bors · 2025-03-24T10:56:07Z

☀️ Try build successful - checks-actions
Build commit: 17e10ad (17e10ad8da034ed4a42c29dcd36a2207ae7934e2)

rust-timer · 2025-03-24T12:49:43Z

Finished benchmarking commit (17e10ad): comparison URL.

Overall result: ❌✅ regressions and improvements - please read the text below

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please indicate this with @rustbot label: +perf-regression-triaged along with sufficient written justification. If you cannot justify the regressions please fix the regressions and do another perf run. If the next run shows neutral or positive results, the label will be automatically removed.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

This is the most reliable metric that we have; it was used to determine the overall result at the top of this comment. However, even this metric can sometimes exhibit noise.

	mean	range	count
Regressions ❌ (primary)	0.2%	[0.1%, 0.5%]	26
Regressions ❌ (secondary)	0.4%	[0.1%, 0.8%]	18
Improvements ✅ (primary)	-0.3%	[-0.5%, -0.1%]	66
Improvements ✅ (secondary)	-0.4%	[-0.7%, -0.2%]	20
All ❌✅ (primary)	-0.1%	[-0.5%, 0.5%]	92

Max RSS (memory usage)

Results (primary 5.6%, secondary 5.0%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	5.6%	[1.5%, 16.8%]	128
Regressions ❌ (secondary)	5.5%	[0.8%, 9.2%]	34
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-2.0%	[-2.0%, -2.0%]	2
All ❌✅ (primary)	5.6%	[1.5%, 16.8%]	128

Cycles

Results (primary 2.3%, secondary 4.1%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	2.3%	[1.5%, 4.1%]	27
Regressions ❌ (secondary)	4.1%	[2.3%, 7.5%]	12
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	2.3%	[1.5%, 4.1%]	27

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: missing data
Artifact size: 365.48 MiB -> 365.59 MiB (0.03%)

Zoxc · 2025-03-24T23:32:07Z

@bors try @rust-timer queue

bors · 2025-03-24T23:33:18Z

⌛ Trying commit 60e4a1b with merge c55e002...

Remove `prev_index_to_index` field from `CurrentDepGraph` The dep graph currently has 2 ways to map a previous index into a current index. The `prev_index_to_index` map stores the current index equivalent of a previous index. For indices which are marked green, we also store the same information in the `DepNodeColorMap`. We actually only need to known the mapping for green nodes however, so this PR removes `prev_index_to_index` and instead makes use of the `DepNodeColorMap`. To avoid racing when promoting a node from the previous session, the encoder lock is now used to ensure only one thread encodes the promoted node. This was previously done by the lock in `prev_index_to_index`. This also changes `nodes_newly_allocated_in_current_session` used to detect duplicate dep nodes to contain both new and previous nodes, which is simpler and can better catch duplicates. The dep node index encoding used in `DepNodeColorMap` is tweak to avoid subtraction / addition to optimize accessing the current equivalent of a previous index. r? `@oli-obk`

bors · 2025-03-25T01:43:03Z

☀️ Try build successful - checks-actions
Build commit: c55e002 (c55e002140fecce72db0a2841959882d767969be)

rust-timer · 2025-03-25T03:01:41Z

Finished benchmarking commit (c55e002): comparison URL.

Overall result: ❌✅ regressions and improvements - please read the text below

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please indicate this with @rustbot label: +perf-regression-triaged along with sufficient written justification. If you cannot justify the regressions please fix the regressions and do another perf run. If the next run shows neutral or positive results, the label will be automatically removed.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

This is the most reliable metric that we have; it was used to determine the overall result at the top of this comment. However, even this metric can sometimes exhibit noise.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	0.3%	[0.3%, 0.3%]	3
Improvements ✅ (primary)	-0.3%	[-0.5%, -0.2%]	55
Improvements ✅ (secondary)	-0.3%	[-0.6%, -0.2%]	16
All ❌✅ (primary)	-0.3%	[-0.5%, -0.2%]	55

Max RSS (memory usage)

Results (primary 5.5%, secondary 5.4%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	5.5%	[1.7%, 15.7%]	105
Regressions ❌ (secondary)	6.1%	[0.9%, 8.5%]	23
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-2.7%	[-3.5%, -1.9%]	2
All ❌✅ (primary)	5.5%	[1.7%, 15.7%]	105

Cycles

Results (primary 2.0%, secondary 4.5%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	2.0%	[1.1%, 3.6%]	13
Regressions ❌ (secondary)	4.5%	[2.3%, 7.1%]	15
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	2.0%	[1.1%, 3.6%]	13

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 777.151s -> 777.798s (0.08%)
Artifact size: 365.75 MiB -> 365.70 MiB (-0.01%)

Zoxc · 2025-03-25T04:18:41Z

I did some local benchmarks since incremental-verify-ich is made more expensive, throwing off perf.

Benchmark	Before	After		Before	After		Before	After
Benchmark	Time	Time	%	Physical Memory	Physical Memory	%	Committed Memory	Committed Memory	%
🟣 clap:check:unchanged	0.3296s	0.3271s	-0.76%	99.04 MiB	98.30 MiB	-0.75%	181.66 MiB	180.99 MiB	-0.37%
🟣 hyper:check:unchanged	0.1284s	0.1277s	-0.54%	62.31 MiB	62.03 MiB	-0.44%	135.99 MiB	135.74 MiB	-0.19%
🟣 regex:check:unchanged	0.2479s	0.2447s	💚 -1.26%	80.06 MiB	79.45 MiB	-0.76%	157.87 MiB	157.28 MiB	-0.37%
🟣 syn:check:unchanged	0.5364s	0.5338s	-0.50%	122.01 MiB	121.08 MiB	-0.76%	209.28 MiB	208.37 MiB	-0.43%
Total	1.2422s	1.2332s	-0.72%	363.42 MiB	360.86 MiB	-0.70%	684.80 MiB	682.38 MiB	-0.35%
Summary	1.0000s	0.9924s	-0.76%	1 byte	0.99 bytes	-0.68%	1 byte	1.00 bytes	-0.34%

Benchmark	Before	After		Before	After		Before	After
Benchmark	Time	Time	%	Physical Memory	Physical Memory	%	Committed Memory	Committed Memory	%
🟣 clap:check:initial	1.6306s	1.6188s	-0.73%	152.29 MiB	152.18 MiB	-0.08%	214.56 MiB	214.45 MiB	-0.05%
🟣 hyper:check:initial	0.2810s	0.2786s	-0.87%	83.06 MiB	83.00 MiB	-0.08%	142.27 MiB	142.24 MiB	-0.03%
🟣 regex:check:initial	0.9599s	0.9484s	💚 -1.20%	112.44 MiB	112.52 MiB	0.07%	168.46 MiB	168.57 MiB	0.07%
🟣 syn:check:initial	1.6135s	1.5986s	-0.92%	151.46 MiB	151.46 MiB	0.00%	212.65 MiB	212.65 MiB	-0.00%
Total	4.4850s	4.4443s	-0.91%	499.25 MiB	499.16 MiB	-0.02%	737.94 MiB	737.91 MiB	-0.00%
Summary	1.0000s	0.9907s	-0.93%	1 byte	1.00 bytes	-0.02%	1 byte	1.00 bytes	-0.00%

oli-obk

oh whoops, forgot to approve this yesterday

oli-obk · 2025-03-25T11:02:31Z

compiler/rustc_query_system/src/dep_graph/graph.rs

-const COMPRESSED_NONE: u32 = 0;
-const COMPRESSED_RED: u32 = 1;
-const COMPRESSED_FIRST_GREEN: u32 = 2;
+const COMPRESSED_NONE: u32 = u32::MAX;
+const COMPRESSED_RED: u32 = u32::MAX - 1;


Please put such drive-by changes into separate commits in the future

oli-obk · 2025-03-26T08:27:03Z

@bors r+

bors · 2025-03-26T08:27:06Z

📌 Commit 60e4a1b has been approved by oli-obk

It is now in the queue for this repository.

bors · 2025-03-26T17:36:56Z

⌛ Testing commit 60e4a1b with merge a2e6356...

bors · 2025-03-26T20:46:28Z

☀️ Test successful - checks-actions
Approved by: oli-obk
Pushing a2e6356 to master...

github-actions · 2025-03-26T20:48:03Z

What is this?

This is an experimental post-merge analysis report that shows differences in test outcomes between the merged PR and its parent PR.

Comparing 19cab6b (parent) -> a2e6356 (this PR)

Test differences

No test diffs found

rust-timer · 2025-03-26T23:03:56Z

Finished benchmarking commit (a2e6356): comparison URL.

Overall result: ❌✅ regressions and improvements - please read the text below

Our benchmarks found a performance regression caused by this PR.
This might be an actual regression, but it can also be just noise.

Next Steps:

If the regression was expected or you think it can be justified,
please write a comment with sufficient written justification, and add
@rustbot label: +perf-regression-triaged to it, to mark the regression as triaged.
If you think that you know of a way to resolve the regression, try to create
a new PR with a fix for the regression.
If you do not understand the regression or you think that it is just noise,
you can ask the @rust-lang/wg-compiler-performance working group for help (members of this group
were already notified of this PR).

@rustbot label: +perf-regression
cc @rust-lang/wg-compiler-performance

Instruction count

This is the most reliable metric that we have; it was used to determine the overall result at the top of this comment. However, even this metric can sometimes exhibit noise.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	0.3%	[0.3%, 0.4%]	3
Improvements ✅ (primary)	-0.3%	[-0.4%, -0.2%]	30
Improvements ✅ (secondary)	-0.3%	[-0.3%, -0.3%]	3
All ❌✅ (primary)	-0.3%	[-0.4%, -0.2%]	30

Max RSS (memory usage)

Results (primary 5.2%, secondary 0.1%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	5.5%	[1.7%, 15.8%]	105
Regressions ❌ (secondary)	4.6%	[0.9%, 7.6%]	23
Improvements ✅ (primary)	-3.5%	[-4.6%, -2.9%]	4
Improvements ✅ (secondary)	-2.6%	[-4.0%, -0.9%]	39
All ❌✅ (primary)	5.2%	[-4.6%, 15.8%]	109

Cycles

Results (primary 2.0%, secondary 3.7%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	2.0%	[1.0%, 3.3%]	23
Regressions ❌ (secondary)	4.3%	[2.1%, 7.2%]	9
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-2.0%	[-2.0%, -2.0%]	1
All ❌✅ (primary)	2.0%	[1.0%, 3.3%]	23

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 777.548s -> 778.152s (0.08%)
Artifact size: 365.81 MiB -> 365.74 MiB (-0.02%)

rustbot assigned oli-obk Mar 22, 2025

This comment has been minimized.

Sign in to view

rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Mar 24, 2025

This comment has been minimized.

Sign in to view

rustbot added perf-regression Performance regression. and removed S-waiting-on-perf Status: Waiting on a perf run to be completed. labels Mar 24, 2025

Remove prev_index_to_index field from CurrentDepGraph

60e4a1b

Zoxc force-pushed the dep-graph-no-prev-map branch from 59e27c8 to 60e4a1b Compare March 24, 2025 18:58

This comment has been minimized.

Sign in to view

rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Mar 24, 2025

This comment has been minimized.

Sign in to view

rustbot removed the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Mar 25, 2025

oli-obk approved these changes Mar 26, 2025

View reviewed changes

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Mar 26, 2025

bors added the merged-by-bors This PR was explicitly merged by bors. label Mar 26, 2025

bors merged commit a2e6356 into rust-lang:master Mar 26, 2025
7 checks passed

rustbot added this to the 1.87.0 milestone Mar 26, 2025

Zoxc deleted the dep-graph-no-prev-map branch March 26, 2025 20:49

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Remove `prev_index_to_index` field from `CurrentDepGraph` #138824

Remove `prev_index_to_index` field from `CurrentDepGraph` #138824

Zoxc commented Mar 22, 2025 •

edited

Loading

rustbot commented Mar 22, 2025

oli-obk commented Mar 24, 2025

This comment has been minimized.

bors commented Mar 24, 2025

bors commented Mar 24, 2025

This comment has been minimized.

rust-timer commented Mar 24, 2025

Zoxc commented Mar 24, 2025

This comment has been minimized.

bors commented Mar 24, 2025

bors commented Mar 25, 2025

This comment has been minimized.

rust-timer commented Mar 25, 2025

Zoxc commented Mar 25, 2025

oli-obk left a comment

oli-obk Mar 25, 2025

oli-obk commented Mar 26, 2025

bors commented Mar 26, 2025

bors commented Mar 26, 2025

bors commented Mar 26, 2025

github-actions bot commented Mar 26, 2025

rust-timer commented Mar 26, 2025

Remove prev_index_to_index field from CurrentDepGraph #138824

Remove prev_index_to_index field from CurrentDepGraph #138824

Conversation

Zoxc commented Mar 22, 2025 • edited Loading

rustbot commented Mar 22, 2025

oli-obk commented Mar 24, 2025

This comment has been minimized.

bors commented Mar 24, 2025

bors commented Mar 24, 2025

This comment has been minimized.

rust-timer commented Mar 24, 2025

Overall result: ❌✅ regressions and improvements - please read the text below

Zoxc commented Mar 24, 2025

This comment has been minimized.

bors commented Mar 24, 2025

bors commented Mar 25, 2025

This comment has been minimized.

rust-timer commented Mar 25, 2025

Overall result: ❌✅ regressions and improvements - please read the text below

Zoxc commented Mar 25, 2025

oli-obk left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

oli-obk commented Mar 26, 2025

bors commented Mar 26, 2025

bors commented Mar 26, 2025

bors commented Mar 26, 2025

github-actions bot commented Mar 26, 2025

Test differences

rust-timer commented Mar 26, 2025

Overall result: ❌✅ regressions and improvements - please read the text below

Remove `prev_index_to_index` field from `CurrentDepGraph` #138824

Remove `prev_index_to_index` field from `CurrentDepGraph` #138824

Zoxc commented Mar 22, 2025 •

edited

Loading