Upgrade ICU to 72.1.0.4 #686

ilonatommy · 2025-10-29T16:27:28Z

This PR:

Merges changes from the upstream repository.
Removes UCARules exclusion from filter files. This change affects only non-hybrid ICU filter, hybrid filters for mobiles never excluded collation rules.
Refreshes the prebuilts.

Previous attempt: #340.
Runtime PR, related to the last revert of the upgrade: dotnet/runtime#93756.

Upgrade size tracking

Before

-rw-rw-rw-  1 vscode root  956416 Oct 29 12:10 icudt_CJK.dat
-rw-rw-rw-  1 vscode root 1526128 Oct 29 12:10 icudt.dat
-rw-rw-rw-  1 vscode root  550832 Oct 29 12:10 icudt_EFIGS.dat
-rw-rw-rw-  1 vscode root 1107168 Oct 29 12:10 icudt_no_CJK.dat

-rw-rw-rw-  1 vscode vscode  248840 Oct 29 12:10 icudt_CJK.dat.br
-rw-rw-rw-  1 vscode vscode  329770 Oct 29 12:10 icudt.dat.br
-rw-rw-rw-  1 vscode vscode  143983 Oct 29 12:10 icudt_EFIGS.dat.br
-rw-rw-rw-  1 vscode vscode  222250 Oct 29 12:10 icudt_no_CJK.dat.br

After

-rw-rw-rw-  1 vscode vscode 1245712 Oct 29 16:02 icudt_CJK.dat
-rw-rw-rw-  1 vscode vscode 1856624 Oct 29 15:59 icudt.dat
-rw-rw-rw-  1 vscode vscode  831504 Oct 29 16:01 icudt_EFIGS.dat
-rw-rw-rw-  1 vscode vscode 1431696 Oct 29 16:00 icudt_no_CJK.dat

-rw-rw-rw-  1 vscode vscode  301244 Oct 29 16:02 icudt_CJK.dat.br
-rw-rw-rw-  1 vscode vscode  390211 Oct 29 15:59 icudt.dat.br
-rw-rw-rw-  1 vscode vscode  195542 Oct 29 16:01 icudt_EFIGS.dat.br
-rw-rw-rw-  1 vscode vscode  279858 Oct 29 16:00 icudt_no_CJK.dat.br

…7 tag (#112) * port library to work with ICU clean and update fix leaks remove caching and unneeded code handle memoryu allocation error change to uprv malloc change method usage change UChar to char use CharStrings and remove unused apis address feedback address feedback fix mistakes fix mistakes fix mistakes fix mistakes fix mistakes debugging return calendar that was failing address feedback and add calendar remove unneeded method create RAII objects for wchar t add tests for library test only on windows address feedback fix casting failures remove two check pattern and fix tests fix tests fix ci builds fix ci builds address feedback address feedback fix quotes I missed use char instead of strings add test file to objects fix typo sigh... add fork for previous behaviour and add test fix name of test Add variable to uconfig improve test case * address feedback * address feedback again * address feedback pt 3 * fix macros * refactor macro * address minor nits * fix brackets

) I'm adding a patch file to be able to keep the changes done in #112 This added the uprefs preference override library that will allow you to get the default locale as a BCP 47 tag.

…anch

…n.txt and cgmanifest.json

data. This changes modifies the ICU tests to pass with the modified CLDR-MS data. The modifications are generally for the following reasons: - Some of data is different due to internal requirements. - Some are due to the extra locales that we pick up from CLDR Seed. These Seed locales have data quality issues, which causes the ICU tests to fail as they don't meet the ICU expectations. - Removal of the yue-* locales.

This adds special marker comments to the ICU headers in order to omit things we don't want or can't expose in the Windows public SDK headers.

,add uprv_u_cleanup

…dows.

For the Windows OS build of ICU, we only have one data file and we don't use the extended data at all. We make this function a no-op in order to save a few cycles for perf, but more importantly so that we don't try to load a versioned data file (ex: icudt68l.dat) after already loading the non-versioned common data file.

Modify `make dist` paths for the MS-ICU GitHub path location, plus omit the Doxygen docs as we don't currently build the docs.

011-MSFT-Patch_change-tests-data-to-not-include-blocked-regions.patch

…he UCRT This modifies the .vcxproj files for the "common" (uc) and "i18n" (in) libraries in order to statically link the VCRuntime (libvcruntime.lib), VCStartup (libcmt.lib), and STL/MSVCP (libcpmt.lib), but dynamically link to the UCRT (ucrt.lib). The UCRT (C Runtime library: ucrtbase.dll) is included in Windows 10 and above as an OS component. (See https://docs.microsoft.com/cpp/c-runtime-library/crt-library-features ). On previous versions of Windows it is available via a Windows Update package. (See the KB article here: https://support.microsoft.com/kb/2999226 ). However, the vcruntime*.dll and msvcp*.dll files are only available via the Visual C++ Redistributable (VC Redist), and the Redist must be installed manually. This change avoids the need to install the VC Redist on Windows 10, and previous Windows versions with the KB update for the UCRT, in order to use the two icuuc*.dll and icuin*.dll libraries.

Modify the .vcxproj files to add the ICU major version number to the PDB filename in order to match the DLL filenames.

Windows only searches for .exe executables by default, so we can't just call "ant" and instead need to launch a cmd prompt to run it. The """ changes are needed in order double-quote the arguments that get passed to the JVM. Without this hack the arguments don't get quoted at all, which causes build failures as some of the variables which get replaced have embedded spaces in them.

This changes modifies the ICU tests to pass with extra locales from CLDR-MS.

…CLDR-MS

BCP47 tag (#112) Currently, for many processes and tasks, ICU gets the default locale and caches it. This means that when needed, ICU will get something like "en-US" and that will not change even if you were to change your language or region in your device. Furthermore, ICU has currently no way of getting other globalization settings such as currency, calendar, hour cycle, first day of week, sorting method and measurement system. We have decided to add a way to solve these two problems. By adding the uprefs library (only to the Windows implementation of uprv_getDefaultLocaleID()), we are adding the Uprefs_getBCP47Tag() internal API, which obtains a full, canonical and valid BCP47Tag containing all of the settings. This means we also change the way we get the default locale. We go from getting only the locale and region, to getting the full thing.

* Fix CI check for version number match * Normalize the ICU version fro all scripts to use * Move version normalization before syncing ICU version with other scripts * Moving sanity check on ICU version number before normalization

…locales

Bumps [guava](https://github.com/google/guava) from 30.0-jre to 32.0.0-jre. - [Release notes](https://github.com/google/guava/releases) - [Commits](https://github.com/google/guava/commits) --- updated-dependencies: - dependency-name: com.google.guava:guava dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com>

…ales (#134) * Replace NNBSP with regular space for date-time formats on English locales and fix test cases

… fields (#136)

* Disable the dynamic plug-in loading and update to v72.1.0.3 * Update changelog.md Co-authored-by: Jeff Genovy <29107334+jefgen@users.noreply.github.com> * Update changelog.md Co-authored-by: Jeff Genovy <29107334+jefgen@users.noreply.github.com> --------- Co-authored-by: Jeff Genovy <29107334+jefgen@users.noreply.github.com>

* Fixed CodeQL warnings: Too few arguments for formatting function * fix CodeQL Warnings: Comparison of narrow type with wide type in loop condition * Fix CodeQL Warnings: Important Severity, mixed issues * Addressed review comments * Addressed review comments on C Style casting * Addressed Review Comments * Trigger CLA

* Update ms-icu to support Unicode 15.1 character data * Update changelog and version to 72.1.0.4

…ebuilds with new filters.

ilonatommy · 2025-10-29T17:02:27Z

Testing with sample app revealed some issues, ICU call ulocdata_getCLDRVersion failed with error #2 'U_MISSING_RESOURCE_ERROR'., checking that.

ilonatommy · 2025-10-30T08:56:15Z

Testing with sample app revealed some issues, ICU call ulocdata_getCLDRVersion failed with error #2 'U_MISSING_RESOURCE_ERROR'., checking that.

It was caused by my testing methodology, I used <BlazorIcuDataFileName>icudt_72_no_CJK.dat</BlazorIcuDataFileName>, without updating libs files (they were still on 68).

After updating the whole runtime pack contents with newly created artifacts, we still see a problem: dotnet.native.wasm binary was compiled with ICU 68 symbols hardcoded (icudt68l-*).

I tried to repackage ICU 72 date files with the p68-prefix, to match the hardcoded symbols but it's not enough - 72 version data layout is different than 68 and we're hitting unknown data format 45.6d.6f.6a ("Emoj"). It's because ICU 72 improved emoji support by introducing new data format.

@akoeplinger, we have to merge this before having it fully tested.

akoeplinger

LGTM apart from one comment, great job, thank you!

We should also get rid of icudt_hybrid.dat (it's no longer used), but that can be done in a separate PR.

icu/tools/cldr/cldr-to-icu/pom.xml

erik0686 and others added 30 commits October 6, 2021 15:06

Create patch file for recently merged preferences override library (#118

fc2d980

) I'm adding a patch file to be able to keep the changes done in #112 This added the uprefs preference override library that will allow you to get the default locale as a BCP 47 tag.

Bump version number to 68.2.0.10 and update changelog (#119)

f869c21

Initial commit of unmodified ICU source code from maint-72 branch

b4140ea

Initial commit of unmodified cldr-to-icu build tools from maint-72 br…

c501d19

…anch

[ICU 72.1 Update] Update ICU version number and commit hash in versio…

c488372

…n.txt and cgmanifest.json

[ICU 72.1 Update] Remove obsolete upstream ICU patches

724c5e7

[ICU 72.1] Apply Patch 0 for versionless filenames

5eeb1de

[ICU 72.1] Apply patch 1 for hiding API in header

5829707

MSFT-PATCH: Changes to ICU headers for OS version.

ddc0250

This adds special marker comments to the ICU headers in order to omit things we don't want or can't expose in the Windows public SDK headers.

MSFT-PATCH: Make u_cleanup a no-op for OS version

2135c05

,add uprv_u_cleanup

MSFT-PATCH: Changes to the CLDR-to-ICU data build tool for MS-ICU/Win…

2c2f2cd

…dows.

MSFT-PATCH: Modify make dist for MS-ICU GitHub paths.

d127a5f

Modify `make dist` paths for the MS-ICU GitHub path location, plus omit the Doxygen docs as we don't currently build the docs.

MSFT-PATCH: Apply changes from:

9b9f246

011-MSFT-Patch_change-tests-data-to-not-include-blocked-regions.patch

MSFT-PATCH: Include the ICU major version number in the PDB filename.

05b92e7

Modify the .vcxproj files to add the ICU major version number to the PDB filename in order to match the DLL filenames.

MSFT-PATCH: Modify ICU tests to pass with extra locales from CLDR-MS.

cacf8f8

This changes modifies the ICU tests to pass with extra locales from CLDR-MS.

MSFT-PATCH: Increase the STRING_STORE_SIZE due to extra locales from …

56a35a2

…CLDR-MS

Add missing parameters to MockGetLocaleInfoEx

860c2ea

Fix azure version check (#124)

5d0df0c

* Fix CI check for version number match * Normalize the ICU version fro all scripts to use * Move version normalization before syncing ICU version with other scripts * Moving sanity check on ICU version number before normalization

Update tzdata 2022g and windows mappings for mexico changes

008919b

[ICU 72.1 Update] Hand carry MS-ICU v68 locale config

bdbdcf7

ICU 72 Update: Update build config to use NLS locale aliases for new …

e7bfb5d

…locales

ICU 72 Update: Rebuild ICU txt files with rebuilt CLDR-MS data

5d4be43

ICU 72 Update: Remove blocked locales from the txt files

918cd66

ICU 72 Update: Update ICU txt files to IANA tzdb 2022g

c0fc0aa

rp9-next and others added 17 commits July 19, 2023 12:17

Update Nuget CI to use ubuntu-latest

2083f53

Update changelog and version number to 72.1.0.1

52d621a

Update guava dependency to 32.1.1 (latest) to fix vulnerabilitites

2d8cc70

Replace NNBSP with regular space for date-time formats on English loc…

0351e5d

…ales (#134) * Replace NNBSP with regular space for date-time formats on English locales and fix test cases

Revert treatment of '@' as ALetter for word break (#135)

dcfed5f

restore strict parsing length tolerance for non-abutting numeric date…

46bc1a0

… fields (#136)

Update version number to 72.1.0.2 (#137)

51048b4

Update ms-icu to support Unicode 15.1 character data (#154)

2e2156e

* Update ms-icu to support Unicode 15.1 character data * Update changelog and version to 72.1.0.4

Workaround for issue with VS 17.12 in github/azure VMs (#155)

97b8365

Fix Prefast Compiler issues (#156)

8520d60

Address CodeQL issues affecting S360 items (#157)

5f127ca

Merge with upstream.

d997f26

Fix merge errors.

0265f10

Remove collation rules exclusions to avoid stack overflow + update pr…

58eaab0

…ebuilds with new filters.

ilonatommy requested a review from akoeplinger October 29, 2025 16:27

ilonatommy self-assigned this Oct 29, 2025

ilonatommy marked this pull request as draft October 29, 2025 17:01

ilonatommy marked this pull request as ready for review October 30, 2025 08:56

akoeplinger approved these changes Oct 30, 2025

View reviewed changes

icu/tools/cldr/cldr-to-icu/pom.xml Show resolved Hide resolved

Feedback - revert changes to pluginRepositories and repositories.

db94d6a

ilonatommy enabled auto-merge October 30, 2025 10:26

Align whitespaces.

dabce61

ilonatommy merged commit 060989b into dotnet/main Oct 30, 2025
7 checks passed

This was referenced Oct 30, 2025

Upgrade to ICU 72.1.0.4 dotnet/runtime#121202

Open

ICU 72 - prevent stack overflow in collator #691

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Upgrade ICU to 72.1.0.4 #686

Upgrade ICU to 72.1.0.4 #686

Uh oh!

ilonatommy commented Oct 29, 2025 •

edited

Loading

Uh oh!

ilonatommy commented Oct 29, 2025

Uh oh!

ilonatommy commented Oct 30, 2025

Uh oh!

akoeplinger left a comment •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Upgrade ICU to 72.1.0.4 #686

Upgrade ICU to 72.1.0.4 #686

Uh oh!

Conversation

ilonatommy commented Oct 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Upgrade size tracking

Before

After

Uh oh!

ilonatommy commented Oct 29, 2025

Uh oh!

ilonatommy commented Oct 30, 2025

Uh oh!

akoeplinger left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

ilonatommy commented Oct 29, 2025 •

edited

Loading

akoeplinger left a comment •

edited

Loading