[release/6.0-staging] Fix Ordinal Ignore Case string compare

[github-actions]

Backport of #71022 to release/6.0-staging

/cc @tarekgh

Customer Impact

A customer has reported a regression in .NET 6 where sorting with ordinal ignore case produces different results compared to .NET 5.0 and .NET Framework. The issue only arises when dealing with non-ASCII strings that contain characters with support casing. This can be challenging for the customer to troubleshoot and may manifest in various scenarios, such as deserialization across frameworks or user applications that rely on algorithms like binary trees or sorting.
Example of the broken behavior, when sorting Ѱa and ѰB, ѰB will come before Ѱa

The fix for this issue was already included in .NET 7.0. While we previously tried to backport the fix through a pull request PR, it was recommended that we wait and observe if more users experience the same problem. However, as we have received reports and requests from users to bring the fix to .NET 6.0, we are currently considering this option. More information can be found at #84251.

Testing

Have done manual testing to ensure we are getting the right behavior. I have added more tests to ensure I catch this problem if it happens again in the future. The change has passed all regression tests.

Risk

The risk associated with this change is low, as it is specific to the problematic scenario and is intended to correct the behavior. I do not anticipate any negative impact or interference with other processes. Additionally, since we have already implemented this fix successfully in .NET 7.0, the risk level is further reduced, given that there were no adverse effects reported following the implementation.

[ghost]

Tagging subscribers to this area: @dotnet/area-system-globalization
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Backport of #71022 to release/6.0-staging

/cc @tarekgh

Customer Impact

Testing

Risk

IMPORTANT: If this backport is for a servicing release, please verify that:

• The PR target branch is release/X.0-staging, not release/X.0.

• If the change touches code that ships in a NuGet package, you have added the necessary package authoring and gotten it explicitly reviewed.

Author:	github-actions[bot]
Assignees:	-
Labels:	area-System.Globalization
Milestone:	-

[carlossanlop]

@tarekgh - The new check-service-labels CI action should start passing after the Servicing-approved label is added, which is done after Tactics approves this PR.

[tarekgh]

This is approved by email.

[carlossanlop]

Requirements for merging:

• Approved by Tactics: added the Servicing-approved label and the check-service-labels CI action passing.
• Signed-off by at least one area owner.
• Confirm that the CI failures are unrelated to this change.
• Confirm if these changes require OOB package authoring changes or not.

[tarekgh]

The change here is in corelib which doesn't need OOB authoring. Additionally, the CI failures are not related to the change and are known issues.

[tarekgh]

The change here is in corelib which doesn't need OOB authoring. additionally, the CI failures are not related to the change and are known issues.