[do not merge] CI experiments #112049

Kobzol · 2023-05-28T21:07:18Z

Various CI experiments for try/dist builds.

Kobzol · 2023-05-28T21:07:27Z

bors · 2023-05-28T21:07:35Z

⌛ Trying commit c42e33cf36e728937b9e7d9bbb16c17d0b364e23 with merge c60ecf63ab7a2a0a260b0a8ee02adff4f4dbfe87...

bors · 2023-05-28T22:44:38Z

☀️ Try build successful - checks-actions
Build commit: c60ecf63ab7a2a0a260b0a8ee02adff4f4dbfe87 (c60ecf63ab7a2a0a260b0a8ee02adff4f4dbfe87)

rust-timer · 2023-05-29T03:49:49Z

Finished benchmarking commit (c60ecf63ab7a2a0a260b0a8ee02adff4f4dbfe87): comparison URL.

Overall result: ❌✅ regressions and improvements - ACTION NEEDED

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please indicate this with @rustbot label: +perf-regression-triaged along with sufficient written justification. If you cannot justify the regressions please fix the regressions and do another perf run. If the next run shows neutral or positive results, the label will be automatically removed.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	0.3%	[0.3%, 0.3%]	2
Regressions ❌ (secondary)	0.3%	[0.3%, 0.3%]	2
Improvements ✅ (primary)	-0.3%	[-0.3%, -0.3%]	1
Improvements ✅ (secondary)	-0.4%	[-0.7%, -0.2%]	9
All ❌✅ (primary)	0.1%	[-0.3%, 0.3%]	3

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	3.4%	[3.4%, 3.4%]	1
Regressions ❌ (secondary)	1.8%	[1.8%, 1.8%]	1
Improvements ✅ (primary)	-3.5%	[-3.5%, -3.5%]	1
Improvements ✅ (secondary)	-3.4%	[-3.7%, -3.1%]	2
All ❌✅ (primary)	-0.1%	[-3.5%, 3.4%]	2

Cycles

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-2.0%	[-2.2%, -1.7%]	2
All ❌✅ (primary)	-	-	0

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 646.44s -> 646.666s (0.03%)

Kobzol · 2023-05-29T06:14:36Z

Looks like removing libcore from the profile set doesn't hurt much. Now I'll try to also completely remove rustc-perf gathering when gathering LLVM PGO profiles.

Looks like after the initial stage 2 build with an instrumented LLVM, there is already 6 GiB of profiles (after benchmarks it's usually 20 GiB). So instrumenting stage 2 build provides a lot of data! I'll try what happens if we delete the profiles.

@bors try @rust-timer queue

bors · 2023-05-29T06:14:48Z

⌛ Trying commit 31c917d813e3461b6babe1220a1ac15fc66eae53 with merge 3004a6cee0725e571674b0efdefcd9b82b1e11d6...

bors · 2023-05-29T08:17:30Z

☀️ Try build successful - checks-actions
Build commit: 3004a6cee0725e571674b0efdefcd9b82b1e11d6 (3004a6cee0725e571674b0efdefcd9b82b1e11d6)

Kobzol · 2023-05-29T08:19:19Z

@bors try

bors · 2023-05-29T08:19:28Z

⌛ Trying commit 042e02535bb5a95c8d6d77fc5954d762b73079e0 with merge ed48ba9307a0a1b523e3519499c3918be2dba056...

rust-timer · 2023-05-29T09:33:21Z

Finished benchmarking commit (3004a6cee0725e571674b0efdefcd9b82b1e11d6): comparison URL.

Overall result: ❌ regressions - ACTION NEEDED

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please indicate this with @rustbot label: +perf-regression-triaged along with sufficient written justification. If you cannot justify the regressions please fix the regressions and do another perf run. If the next run shows neutral or positive results, the label will be automatically removed.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	3.2%	[0.3%, 7.3%]	51
Regressions ❌ (secondary)	3.1%	[0.4%, 5.9%]	24
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	3.2%	[0.3%, 7.3%]	51

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	2.5%	[2.5%, 2.5%]	1
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-	-	0

Cycles

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	2.6%	[0.8%, 4.1%]	32
Regressions ❌ (secondary)	2.8%	[1.8%, 3.6%]	12
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	2.6%	[0.8%, 4.1%]	32

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 642.999s -> 642.878s (-0.02%)

bors · 2023-05-29T10:34:36Z

☀️ Try build successful - checks-actions
Build commit: ed48ba9307a0a1b523e3519499c3918be2dba056 (ed48ba9307a0a1b523e3519499c3918be2dba056)

Kobzol · 2023-05-29T10:56:16Z

@rust-timer build ed48ba9307a0a1b523e3519499c3918be2dba056

rust-timer · 2023-05-29T12:33:22Z

Finished benchmarking commit (ed48ba9307a0a1b523e3519499c3918be2dba056): comparison URL.

Overall result: ✅ improvements - no action needed

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

@bors rollup=never
@rustbot label: -S-waiting-on-perf -perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-1.0%	[-2.0%, -0.3%]	33
Improvements ✅ (secondary)	-1.1%	[-1.5%, -0.4%]	12
All ❌✅ (primary)	-1.0%	[-2.0%, -0.3%]	33

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	2.5%	[2.5%, 2.5%]	1
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-	-	0

Cycles

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-2.8%	[-2.8%, -2.8%]	1
All ❌✅ (primary)	-	-	0

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 642.999s -> 644.35s (0.21%)

bors · 2025-01-04T20:30:15Z

⌛ Trying commit 3e4e4c7 with merge 6e853be3113fd014c37a3d055139a8310cf0a85f...

[do not merge] CI experiments Various CI experiments for try/dist builds. r? `@ghost`

bors · 2025-01-04T22:22:24Z

☀️ Try build successful - checks-actions
Build commit: 6e853be (6e853be3113fd014c37a3d055139a8310cf0a85f)

rust-timer · 2025-01-04T23:42:23Z

Finished benchmarking commit (6e853be): comparison URL.

Overall result: no relevant changes - no action needed

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

@bors rollup=never
@rustbot label: -S-waiting-on-perf -perf-regression

Instruction count

This benchmark run did not return any relevant results for this metric.

Max RSS (memory usage)

Results (primary 6.1%, secondary 18.3%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	6.1%	[1.9%, 27.4%]	93
Regressions ❌ (secondary)	18.3%	[2.4%, 47.8%]	57
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	6.1%	[1.9%, 27.4%]	93

Cycles

Results (secondary -2.7%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	2.0%	[2.0%, 2.0%]	1
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-5.1%	[-5.2%, -5.0%]	2
All ❌✅ (primary)	-	-	0

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 762.575s -> 762.46s (-0.02%)
Artifact size: 325.64 MiB -> 325.67 MiB (0.01%)

Kobzol · 2025-01-05T09:48:49Z

@bors try @rust-timer queue

[do not merge] CI experiments Various CI experiments for try/dist builds. r? `@ghost`

bors · 2025-01-05T09:51:09Z

⌛ Trying commit 62a660f with merge 018d612...

bors · 2025-01-05T11:37:47Z

☀️ Try build successful - checks-actions
Build commit: 018d612 (018d612521bd4cc3f7c758118155848b804742b6)

rust-timer · 2025-01-05T12:53:31Z

Finished benchmarking commit (018d612): comparison URL.

Overall result: ✅ improvements - no action needed

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

@bors rollup=never
@rustbot label: -S-waiting-on-perf -perf-regression

Instruction count

This is the most reliable metric that we have; it was used to determine the overall result at the top of this comment. However, even this metric can sometimes exhibit noise.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-0.3%	[-0.4%, -0.2%]	6
All ❌✅ (primary)	-	-	0

Max RSS (memory usage)

Results (primary 10.1%, secondary 3.1%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	10.6%	[0.9%, 33.6%]	59
Regressions ❌ (secondary)	3.5%	[0.9%, 11.2%]	22
Improvements ✅ (primary)	-6.5%	[-9.6%, -3.4%]	2
Improvements ✅ (secondary)	-1.5%	[-2.1%, -0.9%]	2
All ❌✅ (primary)	10.1%	[-9.6%, 33.6%]	61

Cycles

Results (primary -0.7%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	2.3%	[2.3%, 2.3%]	1
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-3.7%	[-3.7%, -3.7%]	1
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-0.7%	[-3.7%, 2.3%]	2

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 763.064s -> 762.666s (-0.05%)
Artifact size: 325.65 MiB -> 325.61 MiB (-0.01%)

Kobzol · 2025-01-05T15:39:21Z

@bors try @rust-timer queue

[do not merge] CI experiments Various CI experiments for try/dist builds. r? `@ghost`

bors · 2025-01-05T15:40:32Z

⌛ Trying commit 295c30d with merge 7aa173f...

bors · 2025-01-05T17:32:14Z

☀️ Try build successful - checks-actions
Build commit: 7aa173f (7aa173f82efe550a8152c9d8b0f296fe5716d5ac)

rust-timer · 2025-01-05T18:52:24Z

Finished benchmarking commit (7aa173f): comparison URL.

Overall result: ❌ regressions - please read the text below

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please indicate this with @rustbot label: +perf-regression-triaged along with sufficient written justification. If you cannot justify the regressions please fix the regressions and do another perf run. If the next run shows neutral or positive results, the label will be automatically removed.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

This is the most reliable metric that we have; it was used to determine the overall result at the top of this comment. However, even this metric can sometimes exhibit noise.

	mean	range	count
Regressions ❌ (primary)	0.5%	[0.1%, 1.6%]	258
Regressions ❌ (secondary)	0.7%	[0.1%, 2.3%]	214
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	0.5%	[0.1%, 1.6%]	258

Max RSS (memory usage)

Results (primary -5.0%, secondary -5.7%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	3.2%	[2.3%, 4.9%]	4
Regressions ❌ (secondary)	2.8%	[1.8%, 5.5%]	20
Improvements ✅ (primary)	-5.2%	[-30.7%, -1.1%]	180
Improvements ✅ (secondary)	-7.2%	[-21.1%, -1.2%]	114
All ❌✅ (primary)	-5.0%	[-30.7%, 4.9%]	184

Cycles

Results (primary 2.8%, secondary 3.9%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	2.8%	[0.8%, 7.9%]	171
Regressions ❌ (secondary)	3.9%	[1.1%, 10.5%]	108
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	2.8%	[0.8%, 7.9%]	171

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 764.981s -> 821.934s (7.45%)
Artifact size: 325.62 MiB -> 325.85 MiB (0.07%)

rustbot added A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue. labels May 28, 2023

This comment has been minimized.

Sign in to view

rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label May 28, 2023

This comment has been minimized.

Sign in to view

rustbot added perf-regression Performance regression. and removed S-waiting-on-perf Status: Waiting on a perf run to be completed. labels May 29, 2023

This comment has been minimized.

Sign in to view

rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label May 29, 2023

This comment has been minimized.

Sign in to view

rustbot removed the S-waiting-on-perf Status: Waiting on a perf run to be completed. label May 29, 2023

This comment has been minimized.

Sign in to view

rustbot removed the perf-regression Performance regression. label May 29, 2023

This comment has been minimized.

Sign in to view

rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Jan 4, 2025

bors added a commit to rust-lang-ci/rust that referenced this pull request Jan 4, 2025

Auto merge of rust-lang#112049 - Kobzol:pgo-omit-benchmarks, r=<try>

6e853be

[do not merge] CI experiments Various CI experiments for try/dist builds. r? `@ghost`

This comment has been minimized.

Sign in to view

rustbot removed S-waiting-on-perf Status: Waiting on a perf run to be completed. perf-regression Performance regression. labels Jan 4, 2025

Enable jemalloc background threads

62a660f

This comment has been minimized.

Sign in to view

rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Jan 5, 2025

bors added a commit to rust-lang-ci/rust that referenced this pull request Jan 5, 2025

Auto merge of rust-lang#112049 - Kobzol:pgo-omit-benchmarks, r=<try>

018d612

[do not merge] CI experiments Various CI experiments for try/dist builds. r? `@ghost`

This comment has been minimized.

Sign in to view

rustbot removed the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Jan 5, 2025

Immediate page purging plus background threads

295c30d

This comment has been minimized.

Sign in to view

rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Jan 5, 2025

bors added a commit to rust-lang-ci/rust that referenced this pull request Jan 5, 2025

Auto merge of rust-lang#112049 - Kobzol:pgo-omit-benchmarks, r=<try>

7aa173f

[do not merge] CI experiments Various CI experiments for try/dist builds. r? `@ghost`

This comment has been minimized.

Sign in to view

rustbot added perf-regression Performance regression. and removed S-waiting-on-perf Status: Waiting on a perf run to be completed. labels Jan 5, 2025

[do not merge] CI experiments #112049

Are you sure you want to change the base?

[do not merge] CI experiments #112049

Conversation

Kobzol commented May 28, 2023 • edited Loading

Kobzol commented May 28, 2023

This comment has been minimized.

bors commented May 28, 2023

bors commented May 28, 2023

This comment has been minimized.

rust-timer commented May 29, 2023

Overall result: ❌✅ regressions and improvements - ACTION NEEDED

Kobzol commented May 29, 2023

This comment has been minimized.

bors commented May 29, 2023

bors commented May 29, 2023

This comment has been minimized.

Kobzol commented May 29, 2023

bors commented May 29, 2023

rust-timer commented May 29, 2023

Overall result: ❌ regressions - ACTION NEEDED

bors commented May 29, 2023

Kobzol commented May 29, 2023

This comment has been minimized.

rust-timer commented May 29, 2023

Overall result: ✅ improvements - no action needed

This comment has been minimized.

bors commented Jan 4, 2025

bors commented Jan 4, 2025

This comment has been minimized.

rust-timer commented Jan 4, 2025

Overall result: no relevant changes - no action needed

Kobzol commented Jan 5, 2025

This comment has been minimized.

bors commented Jan 5, 2025

bors commented Jan 5, 2025

This comment has been minimized.

rust-timer commented Jan 5, 2025

Overall result: ✅ improvements - no action needed

Kobzol commented Jan 5, 2025

This comment has been minimized.

bors commented Jan 5, 2025

bors commented Jan 5, 2025

This comment has been minimized.

rust-timer commented Jan 5, 2025

Overall result: ❌ regressions - please read the text below

Kobzol commented May 28, 2023 •

edited

Loading