Don't include ASCII characters in Unicode tables #146173

Kmeakin · 2025-09-03T19:44:46Z

Split off from #145219

rustbot · 2025-09-03T19:44:51Z

rustbot has assigned @jhpratt.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

rustbot · 2025-09-03T19:44:53Z

library/core/src/unicode/unicode_data.rs is generated by the src/tools/unicode-table-generator tool.

If you want to modify unicode_data.rs, please modify the tool then regenerate the library source file via ./x run src/tools/unicode-table-generator instead of editing unicode_data.rs manually.

jhpratt · 2025-09-04T01:18:05Z

The change itself looks fine, but let's check perf just in case. Reviewing the other discussion, it seems that the team would like to see the impact before potentially merging this.

@bors2 try @rust-timer queue

Don't include ASCII characters in Unicode tables

rust-bors · 2025-09-04T03:34:46Z

☀️ Try build successful (CI)
Build commit: cdb994f (cdb994f1ff4d7f9776a14f9b741a104e6119f9a0, parent: a1208bf765ba783ee4ebdc4c29ab0a0c215806ef)

rust-timer · 2025-09-04T05:57:07Z

Finished benchmarking commit (cdb994f): comparison URL.

Overall result: ❌✅ regressions and improvements - no action needed

Benchmarking this pull request means it may be perf-sensitive – we'll automatically label it not fit for rolling up. You can override this, but we strongly advise not to, due to possible changes in compiler perf.

@bors rollup=never
@rustbot label: -S-waiting-on-perf -perf-regression

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

	mean	range	count
Regressions ❌ (primary)	0.2%	[0.2%, 0.2%]	1
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.4%	[-0.4%, -0.4%]	1
Improvements ✅ (secondary)	-0.1%	[-0.1%, -0.1%]	1
All ❌✅ (primary)	-0.1%	[-0.4%, 0.2%]	2

Max RSS (memory usage)

Results (primary -10.5%, secondary 1.9%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	1.9%	[1.4%, 2.4%]	2
Improvements ✅ (primary)	-10.5%	[-10.5%, -10.5%]	1
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-10.5%	[-10.5%, -10.5%]	1

Cycles

Results (primary -2.6%, secondary -3.0%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-2.6%	[-2.6%, -2.6%]	1
Improvements ✅ (secondary)	-3.0%	[-3.0%, -3.0%]	1
All ❌✅ (primary)	-2.6%	[-2.6%, -2.6%]	1

Binary size

Results (primary 0.1%, secondary -0.0%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	0.1%	[0.1%, 0.2%]	4
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.0%	[-0.0%, -0.0%]	1
Improvements ✅ (secondary)	-0.0%	[-0.0%, -0.0%]	1
All ❌✅ (primary)	0.1%	[-0.0%, 0.2%]	5

Bootstrap: 466.26s -> 465.424s (-0.18%)
Artifact size: 388.41 MiB -> 388.34 MiB (-0.02%)

bors · 2025-09-06T01:20:32Z

☔ The latest upstream changes (presumably #146255) made this pull request unmergeable. Please resolve the merge conflicts.

jhpratt · 2025-09-06T03:46:31Z

Given the nearly zero perf impact, this LGTM. As the team already indicated they're willing to accept changes like this, I'm fine merging this as is.

Let me know when this is rebased to avoid a conflict.

@rustbot author

rustbot · 2025-09-06T03:46:35Z

Reminder, once the PR becomes ready for a review, use @rustbot ready.

The ASCII subset of Unicode is fixed and will never change, so we don't need to generate tables for it with every new Unicode version. This saves a few bytes of static data and speeds up `char::is_control` and `char::is_grapheme_extended` on ASCII inputs. Since the table lookup functions exported from the `unicode` module will give nonsensical errors on ASCII input (and in fact will panic in debug mode), I had to add some private wrapper methods to `char` which check for ASCII-ness first.

rustbot · 2025-09-07T13:21:37Z

This PR was rebased onto a different master commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

Kmeakin · 2025-09-07T13:21:49Z

@rustbot ready

jhpratt · 2025-09-07T21:57:46Z

@bors r+

bors · 2025-09-07T21:57:48Z

📌 Commit a8c6694 has been approved by jhpratt

It is now in the queue for this repository.

bors · 2025-09-08T07:25:10Z

⌛ Testing commit a8c6694 with merge beeb8e3...

bors · 2025-09-08T10:34:07Z

☀️ Test successful - checks-actions
Approved by: jhpratt
Pushing beeb8e3 to master...

github-actions · 2025-09-08T10:37:25Z

What is this?

This is an experimental post-merge analysis report that shows differences in test outcomes between the merged PR and its parent PR.

Comparing a09fbe2 (parent) -> beeb8e3 (this PR)

Test differences

Show 148 test diffs

148 doctest diffs were found. These are ignored, as they are noisy.

Test dashboard

Run

cargo run --manifest-path src/ci/citool/Cargo.toml -- \
    test-dashboard beeb8e3af54295ba494c250e84ecda4c2c5d85ff --output-dir test-dashboard

And then open test-dashboard/index.html in your browser to see an overview of all executed tests.

Job duration changes

dist-aarch64-linux: 8613.2s -> 6263.2s (-27.3%)
dist-aarch64-apple: 7224.9s -> 6139.4s (-15.0%)
dist-x86_64-apple: 7786.3s -> 6734.4s (-13.5%)
aarch64-gnu-llvm-19-2: 2453.5s -> 2145.4s (-12.6%)
pr-check-2: 2440.5s -> 2143.8s (-12.2%)
x86_64-gnu-llvm-19: 2728.7s -> 2417.6s (-11.4%)
pr-check-1: 1538.3s -> 1372.2s (-10.8%)
i686-gnu-2: 6080.5s -> 5497.0s (-9.6%)
x86_64-gnu-llvm-19-1: 3537.6s -> 3212.8s (-9.2%)
x86_64-gnu-miri: 4980.7s -> 4542.9s (-8.8%)

How to interpret the job duration changes?

Job durations can vary a lot, based on the actual runner instance
that executed the job, system noise, invalidated caches, etc. The table above is provided
mostly for t-infra members, for simpler debugging of potential CI slow-downs.

rust-timer · 2025-09-08T11:44:09Z

Finished benchmarking commit (beeb8e3): comparison URL.

Overall result: ❌✅ regressions and improvements - no action needed

@rustbot label: -perf-regression

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	0.0%	[0.0%, 0.1%]	3
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-0.3%	[-0.3%, -0.3%]	1
All ❌✅ (primary)	-	-	0

Max RSS (memory usage)

Results (primary -2.7%, secondary -3.2%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	2.4%	[2.3%, 2.6%]	2
Improvements ✅ (primary)	-2.7%	[-2.7%, -2.7%]	1
Improvements ✅ (secondary)	-4.4%	[-6.7%, -2.2%]	10
All ❌✅ (primary)	-2.7%	[-2.7%, -2.7%]	1

Cycles

Results (secondary -3.3%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-3.3%	[-5.2%, -2.1%]	6
All ❌✅ (primary)	-	-	0

Binary size

Results (primary 0.1%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	0.2%	[0.2%, 0.2%]	4
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.0%	[-0.0%, -0.0%]	1
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	0.1%	[-0.0%, 0.2%]	5

Bootstrap: 468.082s -> 466.876s (-0.26%)
Artifact size: 387.72 MiB -> 387.41 MiB (-0.08%)

rustbot assigned jhpratt Sep 3, 2025

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-libs Relevant to the library team, which will review and decide on the PR/issue. labels Sep 3, 2025

This comment has been minimized.

Sign in to view

rust-bors bot added a commit that referenced this pull request Sep 4, 2025

Auto merge of #146173 - Kmeakin:km/unicode-data/no-ascii, r=<try>

cdb994f

Don't include ASCII characters in Unicode tables

This comment has been minimized.

Sign in to view

rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Sep 4, 2025

This comment has been minimized.

Sign in to view

rustbot removed the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Sep 4, 2025

rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Sep 6, 2025

Kmeakin force-pushed the km/unicode-data/no-ascii branch from f35f48c to a8c6694 Compare September 7, 2025 13:21

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Sep 7, 2025

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Sep 7, 2025

bors added the merged-by-bors This PR was explicitly merged by bors. label Sep 8, 2025

bors merged commit beeb8e3 into rust-lang:master Sep 8, 2025
11 checks passed

rustbot added this to the 1.91.0 milestone Sep 8, 2025

Kmeakin deleted the km/unicode-data/no-ascii branch September 8, 2025 10:51

bors mentioned this pull request Sep 8, 2025

Remove Cased Unicode table #146180

Open

Don't include ASCII characters in Unicode tables #146173

Don't include ASCII characters in Unicode tables #146173

Conversation

Kmeakin commented Sep 3, 2025

Uh oh!

rustbot commented Sep 3, 2025

Uh oh!

rustbot commented Sep 3, 2025

Uh oh!

jhpratt commented Sep 4, 2025

Uh oh!

This comment has been minimized.

This comment has been minimized.

rust-bors bot commented Sep 4, 2025

Uh oh!

This comment has been minimized.

rust-timer commented Sep 4, 2025

Overall result: ❌✅ regressions and improvements - no action needed

Uh oh!

bors commented Sep 6, 2025

Uh oh!

jhpratt commented Sep 6, 2025

Uh oh!

rustbot commented Sep 6, 2025

Uh oh!

rustbot commented Sep 7, 2025

Uh oh!

Kmeakin commented Sep 7, 2025

Uh oh!

jhpratt commented Sep 7, 2025

Uh oh!

bors commented Sep 7, 2025

Uh oh!

bors commented Sep 8, 2025

Uh oh!

bors commented Sep 8, 2025

Uh oh!

Uh oh!

github-actions bot commented Sep 8, 2025

Test differences

Job duration changes

Uh oh!

rust-timer commented Sep 8, 2025

Overall result: ❌✅ regressions and improvements - no action needed

Uh oh!

Uh oh!