Update Regex named blocks from Unicode 4.0 to Unicode 16.0 #120623

Copilot · 2025-10-11T16:08:27Z

Summary

Successfully updated Regex named blocks from Unicode 4.0 to Unicode 16.0:

Added 59 new Unicode blocks including Arabic Extended-A/B, Balinese, Bamum, Cherokee Supplement, Cyrillic Extended-A/B/C, Devanagari Extended, Georgian Extended, Hangul Jamo Extended-A/B, Javanese, Kayah Li, Latin Extended-C/D/E, and many more
Maintained full backward compatibility by:
- Keeping IsGreek as alias for IsGreekandCoptic
- Restoring all previously removed blocks (IsCombiningMarksforSymbols, IsHighPrivateUseSurrogates, IsHighSurrogates, IsLowSurrogates, IsPrivateUse, IsPrivateUseArea)
- Maintaining correct alphabetical ordering
- Preserving original block values (e.g., IsSpecials)
- Maintaining test coverage for all blocks
Created GenRegexNamedBlocks tool for easy future Unicode updates with full documentation
Total named blocks: 167 (increased from 108)
No breaking changes - all existing block names remain available with correct values

The changes enable .NET developers to use regular expressions with up-to-date Unicode block names while maintaining complete backward compatibility.

Original prompt

This section details on the original issue you should resolve

<issue_title>Consider updating named blocks in Regex</issue_title>
<issue_description>While testing for GB18030-2022 I noticed that Regex is missing multiple named blocks.
As per the documentation, the list is based on Unicode 4.0, while latest version is 16.0 https://www.unicode.org/charts/.

I think we already update Unicode categories. Updating named blocks is not as important as these can be easily defined as a range of characters manually.</issue_description>

Comments on the Issue (you are @copilot in this section)
@jeffhandley Marked as https://github.com/dotnet/runtime/labels/needs-further-triage for .NET 11. We should conclude during .NET 11 whether we want to start updating this regularly, or if there is a reason to keep it snapped to Unicode 4.0.

Fixes #118062

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

dotnet-policy-service · 2025-10-11T16:16:51Z

Tagging subscribers to this area: @dotnet/area-system-text-regularexpressions
See info in area-owners.md if you want to be subscribed.

Added 52 new Unicode blocks and updated existing ones to match Unicode 16.0 specification. Total count increased from 108 to 160 named blocks. Co-authored-by: stephentoub <2642209+stephentoub@users.noreply.github.com>

- Add IsGreek as alias for IsGreekandCoptic for backward compatibility - Add comprehensive tests for 52 new Unicode blocks - Remove tests for deprecated surrogate and private use blocks - All 29,287 tests now passing Co-authored-by: stephentoub <2642209+stephentoub@users.noreply.github.com>

Document how to use the tool for future Unicode updates Co-authored-by: stephentoub <2642209+stephentoub@users.noreply.github.com>

Copilot

Pull Request Overview

This PR updates the Unicode named blocks in System.Text.RegularExpressions from Unicode 4.0 to Unicode 16.0, adding 52 new Unicode blocks while maintaining backward compatibility. The update enables regex patterns to match characters from modern scripts and symbol sets that were not previously supported.

Key changes include:

Added 52 new Unicode blocks covering various scripts like Arabic Extended, Balinese, Cherokee Supplement, and many others
Created a new tool GenRegexNamedBlocks to automate future Unicode updates
Updated test coverage to include all new blocks

Reviewed Changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
`src/libraries/System.Text.RegularExpressions/tools/GenRegexNamedBlocks/README.md`	Documentation for the new Unicode block generation tool
`src/libraries/System.Text.RegularExpressions/tools/GenRegexNamedBlocks/Program.cs`	Tool implementation for parsing Unicode Blocks.txt and generating regex named block entries
`src/libraries/System.Text.RegularExpressions/tools/GenRegexNamedBlocks/GenRegexNamedBlocks.csproj`	Project file for the Unicode block generation tool
`src/libraries/System.Text.RegularExpressions/tests/FunctionalTests/RegexCharacterSetTests.cs`	Added test coverage for all 52 new Unicode blocks and removed tests for deprecated surrogate blocks
`src/libraries/System.Text.RegularExpressions/src/System/Text/RegularExpressions/RegexCharClass.cs`	Updated named blocks array with 52 new Unicode blocks, removed deprecated surrogate and private use blocks

Copilot · 2025-10-12T01:18:19Z

src/libraries/System.Text.RegularExpressions/tools/GenRegexNamedBlocks/Program.cs

+            {
+                // We only care about lines of the form "XXXX..XXXX; Block name"
+                var match = inputLineRegex.Match(inputLine);
+                if (match == null || !match.Success)


The null check for match is unnecessary and incorrect. The Match method never returns null; it returns a Match object with Success = false when no match is found. The condition should be if (!match.Success).

Suggested change

if (match == null || !match.Success)

if (!match.Success)

@copilot, please address the feedback from @copilot.

Copilot · 2025-10-12T01:18:20Z

src/libraries/System.Text.RegularExpressions/tools/GenRegexNamedBlocks/Program.cs

+            }
+
+            // Sort by start code for consistent output
+            entries.Sort((a, b) => string.Compare(a.name, b.name, StringComparison.Ordinal));


The comment indicates sorting by start code, but the code is actually sorting by name. Either update the comment to 'Sort by name for consistent output' or change the sort logic to sort by start code.

Suggested change

entries.Sort((a, b) => string.Compare(a.name, b.name, StringComparison.Ordinal));

entries.Sort((a, b) => string.Compare(a.startCode, b.startCode, StringComparison.Ordinal));

...ibraries/System.Text.RegularExpressions/src/System/Text/RegularExpressions/RegexCharClass.cs

Add back 6 blocks that were removed to avoid breaking changes: - IsCombiningMarksforSymbols (alias for IsCombiningDiacriticalMarksforSymbols) - IsHighPrivateUseSurrogates - IsHighSurrogates - IsLowSurrogates - IsPrivateUse (alias for IsPrivateUseArea) - IsPrivateUseArea Total blocks now: 167 (108 original + 59 new) Co-authored-by: stephentoub <2642209+stephentoub@users.noreply.github.com>

...ibraries/System.Text.RegularExpressions/src/System/Text/RegularExpressions/RegexCharClass.cs

src/libraries/System.Text.RegularExpressions/tests/FunctionalTests/RegexCharacterSetTests.cs

Add back test coverage for the 6 restored blocks: - IsHighSurrogates - IsHighPrivateUseSurrogates - IsLowSurrogates - IsPrivateUse Co-authored-by: stephentoub <2642209+stephentoub@users.noreply.github.com>

- Move IsHebrew to correct alphabetical position (before IsHighPrivateUseSurrogates) - Restore IsSpecials to original value "\uFFF0" (was incorrectly changed to "\uFFF0\u10000") Co-authored-by: stephentoub <2642209+stephentoub@users.noreply.github.com>

stephentoub · 2025-10-13T14:36:30Z

@copilot, can you update this for Unicode 17.0?
https://www.unicode.org/versions/Unicode17.0.0/

Initial plan

73ddca9

Copilot AI assigned Copilot and stephentoub Oct 11, 2025

Copilot started work on behalf of stephentoub October 11, 2025 16:08 View session

stephentoub added the area-System.Text.RegularExpressions label Oct 11, 2025

Copilot AI and others added 3 commits October 11, 2025 16:38

Update Regex named blocks from Unicode 4.0 to Unicode 16.0

9f184d8

Added 52 new Unicode blocks and updated existing ones to match Unicode 16.0 specification. Total count increased from 108 to 160 named blocks. Co-authored-by: stephentoub <2642209+stephentoub@users.noreply.github.com>

Add README for GenRegexNamedBlocks tool

38a6671

Document how to use the tool for future Unicode updates Co-authored-by: stephentoub <2642209+stephentoub@users.noreply.github.com>

Copilot AI changed the title ~~[WIP] Update named blocks in Regex for Unicode 16.0~~ Update Regex named blocks from Unicode 4.0 to Unicode 16.0 Oct 11, 2025

Copilot AI requested a review from stephentoub October 11, 2025 17:01

Copilot finished work on behalf of stephentoub October 11, 2025 17:01

build-analysis bot mentioned this pull request Oct 11, 2025

Test failure: System.IO.Tests.FileInfo_SymbolicLinks.ResolveLinkTarget_Succeeds #120380

Open

stephentoub marked this pull request as ready for review October 12, 2025 01:17

stephentoub requested review from Copilot, joperezr and jozkee October 12, 2025 01:17

Copilot AI reviewed Oct 12, 2025

View reviewed changes

stephentoub reviewed Oct 12, 2025

View reviewed changes

...ibraries/System.Text.RegularExpressions/src/System/Text/RegularExpressions/RegexCharClass.cs Show resolved Hide resolved

Copilot started work on behalf of stephentoub October 12, 2025 01:20 View session

Copilot finished work on behalf of stephentoub October 12, 2025 01:48

Copilot AI requested a review from stephentoub October 12, 2025 01:48

stephentoub reviewed Oct 12, 2025

View reviewed changes

Copilot started work on behalf of stephentoub October 12, 2025 02:15 View session

Restore tests for surrogate and private use blocks

76f8a1c

Add back test coverage for the 6 restored blocks: - IsHighSurrogates - IsHighPrivateUseSurrogates - IsLowSurrogates - IsPrivateUse Co-authored-by: stephentoub <2642209+stephentoub@users.noreply.github.com>

Copilot AI requested a review from stephentoub October 12, 2025 02:36

Copilot finished work on behalf of stephentoub October 12, 2025 02:36

Copilot started work on behalf of stephentoub October 12, 2025 02:40 View session

Copilot stopped work on behalf of stephentoub due to an error October 12, 2025 03:01
An unexpected error occurred. For more details, see the detailed logs in GitHub Actions.

Copilot started work on behalf of stephentoub October 13, 2025 14:36 View session

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Update Regex named blocks from Unicode 4.0 to Unicode 16.0 #120623

Update Regex named blocks from Unicode 4.0 to Unicode 16.0 #120623

Copilot AI commented Oct 11, 2025 •

edited

Loading

Uh oh!

dotnet-policy-service bot commented Oct 11, 2025

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Oct 12, 2025

Uh oh!

stephentoub Oct 12, 2025

Uh oh!

Copilot AI Oct 12, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

stephentoub commented Oct 13, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	entries.Sort((a, b) => string.Compare(a.name, b.name, StringComparison.Ordinal));
	entries.Sort((a, b) => string.Compare(a.startCode, b.startCode, StringComparison.Ordinal));

Update Regex named blocks from Unicode 4.0 to Unicode 16.0 #120623

Are you sure you want to change the base?

Update Regex named blocks from Unicode 4.0 to Unicode 16.0 #120623

Conversation

Copilot AI commented Oct 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Comments on the Issue (you are @copilot in this section)

Uh oh!

dotnet-policy-service bot commented Oct 11, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Oct 12, 2025

Choose a reason for hiding this comment

Uh oh!

stephentoub Oct 12, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Oct 12, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

stephentoub commented Oct 13, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Copilot AI commented Oct 11, 2025 •

edited

Loading