ToUpper not working as intended with Turkish Culture #77689

sencagri · 2022-10-31T08:38:01Z

Description

In our tests ToUpper() method returns false characters. The lowercase "i" character should be "İ" but we are getting "I" only in .NET MAUI apps. Other project types are working fine.

Steps to Reproduce

Create a new .NET MAUI project (.Net 6 or preview both are same)
We have added CodePages nuget (System.Text.Encoding.CodePages)
Add the following code at MauiProgram.cs

            Encoding.RegisterProvider(CodePagesEncodingProvider.Instance); // don't know if this helps, just tried...
            CultureInfo tr = new CultureInfo("tr-TR", false);
            string test = "i";
            var test2 = test.ToUpper(tr);
            var testResult = test2 == "İ"; // result is false because test2 is "I" instead of "İ"

Link to public reproduction project repository

https://github.com/sencagri/MauiAppToUpper

Version with bug

7.0 Release Candidate 2

Last version that worked well

Unknown/Other

Affected platforms

Android

Affected platform versions

All android versions

Did you find any workaround?

No response

Relevant log output

No response

Eilon · 2022-10-31T16:04:06Z

Hi @sencagri , did you try any other platforms besides Android, and does it work on those?

hartez · 2022-10-31T17:27:34Z

Related: dotnet/android#5425

hartez · 2022-10-31T17:28:14Z

Reproduced this on Android; tried it on a Windows MAUI project and it works fine.

hartez · 2022-10-31T17:33:14Z

@grendello @jpobst

ghost · 2022-10-31T17:33:28Z

We've moved this issue to the Backlog milestone. This means that it is not going to be worked on for the coming release. We will reassess the backlog following the current release and consider this item at that time. To learn more about our issue management process and to have better expectation regarding different types of issues you can read our Triage Process.

jpobst · 2022-10-31T17:44:31Z

You can probably move this to dotnet/runtime since it's likely in their part of the stack.

dotnet-issue-labeler · 2022-10-31T18:37:02Z

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

sencagri · 2022-10-31T19:00:55Z

Windows works fine

sencagri · 2022-11-01T10:55:09Z

Related: xamarin/xamarin-android#5425

Is this really related? I have tested both Xamarin.Forms and Xamarin.Android project, they are working fine.

ghost · 2022-11-01T15:18:50Z

Tagging subscribers to this area: @dotnet/area-system-globalization
See info in area-owners.md if you want to be subscribed.

Issue Details

Description

In our tests ToUpper() method returns false characters. The lowercase "i" character should be "İ" but we are getting "I" only in .NET MAUI apps. Other project types are working fine.

Steps to Reproduce

Create a new .NET MAUI project (.Net 6 or preview both are same)
We have added CodePages nuget (System.Text.Encoding.CodePages)
Add the following code at MauiProgram.cs

            Encoding.RegisterProvider(CodePagesEncodingProvider.Instance); // don't know if this helps, just tried...
            CultureInfo tr = new CultureInfo("tr-TR", false);
            string test = "i";
            var test2 = test.ToUpper(tr);
            var testResult = test2 == "İ"; // result is false because test2 is "I" instead of "İ"

Link to public reproduction project repository

https://github.com/sencagri/MauiAppToUpper

Version with bug

7.0 Release Candidate 2

Last version that worked well

Unknown/Other

Affected platforms

Android

Affected platform versions

All android versions

Did you find any workaround?

No response

Relevant log output

No response

Author:	sencagri
Assignees:	-
Labels:	`area-System.Globalization`, `untriaged`, `runtime-mono`
Milestone:	-

tarekgh · 2022-11-01T21:04:06Z

@marek-safar could you get someone look at this as it is mono runtime?

tarekgh · 2022-11-01T23:24:28Z

@sencagri just fyi, upper and lower casing strings have nothing to do with the Encoding.

doing

Encoding.RegisterProvider(CodePagesEncodingProvider.Instance); // don't know if this helps, just tried...

is not needed anyway.

ghost · 2022-11-02T07:26:23Z

Tagging subscribers to 'arch-android': @steveisok, @akoeplinger
See info in area-owners.md if you want to be subscribed.

Issue Details

Description

In our tests ToUpper() method returns false characters. The lowercase "i" character should be "İ" but we are getting "I" only in .NET MAUI apps. Other project types are working fine.

Steps to Reproduce

Create a new .NET MAUI project (.Net 6 or preview both are same)
We have added CodePages nuget (System.Text.Encoding.CodePages)
Add the following code at MauiProgram.cs

            Encoding.RegisterProvider(CodePagesEncodingProvider.Instance); // don't know if this helps, just tried...
            CultureInfo tr = new CultureInfo("tr-TR", false);
            string test = "i";
            var test2 = test.ToUpper(tr);
            var testResult = test2 == "İ"; // result is false because test2 is "I" instead of "İ"

Link to public reproduction project repository

https://github.com/sencagri/MauiAppToUpper

Version with bug

7.0 Release Candidate 2

Last version that worked well

Unknown/Other

Affected platforms

Android

Affected platform versions

All android versions

Did you find any workaround?

No response

Relevant log output

No response

Author:	sencagri
Assignees:	-
Labels:	`area-System.Globalization`, `os-android`, `untriaged`, `runtime-mono`
Milestone:	-

marek-safar · 2022-11-02T07:26:45Z

@steveisok please prioritize

steveisok · 2022-11-02T14:17:36Z

I'm not sure there's a whole lot we can do here as we are fully relying on the version of ICU that is packaged with Android.

We are going to experiment on a few ideas and see if they make a difference.

NightOwl888 · 2022-11-03T18:32:25Z

Related: xamarin/xamarin-android#5425

Is this really related? I have tested both Xamarin.Forms and Xamarin.Android project, they are working fine.

What version of Xamarin.Android is it succeeding in? We can't get a single test run to pass testing

netstandard2.1 library with MonoAndroid100
netstandard2.0 library with MonoAndroid90

https://dev.azure.com/NightOwl888/J2N/_build/results?buildId=249&view=ms.vss-test-web.build-test-results-tab&runId=18330&resultId=134521&paneView=debug

sencagri · 2022-11-04T13:21:51Z

Related: xamarin/xamarin-android#5425

Is this really related? I have tested both Xamarin.Forms and Xamarin.Android project, they are working fine.

What version of Xamarin.Android is it succeeding in? We can't get a single test run to pass testing

netstandard2.1 library with MonoAndroid100

netstandard2.0 library with MonoAndroid90

https://dev.azure.com/NightOwl888/J2N/_build/results?buildId=249&view=ms.vss-test-web.build-test-results-tab&runId=18330&resultId=134521&paneView=debug

Hi,
I just created a Xamarin.Android app now and put the code inside MainActivity.cs. If you can tell me how to check Xamarin.Android version I can check it.

steveisok · 2023-07-13T20:55:53Z

There's unfortunately not a lot we can do to control this behavior since we are relying on Android ICU. We had originally wanted to ship our own like we do for iOS, but we didn't want to take on a few dependencies.

hartez · 2023-07-18T00:05:55Z

relying on Android ICU

Relying on it how? What's the mechanism?

steveisok · 2023-07-18T00:36:21Z

Relying on it how? What's the mechanism?

The entire globalization interop calls into ICU functions. In the case of Android, it's the one they ship.

sencagri · 2023-07-18T06:57:54Z

Any kind of workaround?

Clockwork-Muse · 2023-07-18T17:24:27Z

You'd have to include your own copies of the ICU libraries, essentially (as @steveisok mentioned).

There are some relatively simple instructions for including them for your application. They only specifically list Windows/Linux (ie, intending it to be server/desktop applications), though, so I don't know how it would go for Android.

Keep in mind that if you go that route, you will have to make sure to update and distribute your application regularly to account for ICU updates, which is multiple times a year.

That said, in at least some cases some of the Turkish characters might be created via combining characters. Have you checked the actual length of the string and characters actually present?

hartez · 2023-07-19T02:19:54Z

The entire globalization interop calls into ICU functions. In the case of Android, it's the one they ship.

So presumably there's a bug in the functions they ship, and at some point in the future they'll fix it and ToUpper() will work correctly again?

Is it a known bug? Is it being tracked?

Clockwork-Muse · 2023-07-19T02:48:08Z

So presumably there's a bug in the functions they ship

More likely there's something up with their data tables (there's two parts to ICU - a set of functions, and a set of data tables)

Is it a known bug? Is it being tracked?

I doubt it.
The reason being that there are multiple updates per year, and you're normally just supposed to include the update when you can.

ghost added the untriaged New issue has not been triaged by the area owner label Oct 31, 2022

Eilon transferred this issue from dotnet/maui Oct 31, 2022

hartez added the runtime-mono specific to the Mono runtime label Oct 31, 2022

jeffschwMSFT added the area-System.Globalization label Nov 1, 2022

marek-safar added the os-android label Nov 2, 2022

marek-safar removed untriaged New issue has not been triaged by the area owner runtime-mono specific to the Mono runtime labels Nov 2, 2022

marek-safar added this to the 8.0.0 milestone Nov 2, 2022

steveisok assigned mkhamoyan Nov 2, 2022

steveisok closed this as completed Jul 13, 2023

ghost locked as resolved and limited conversation to collaborators Aug 18, 2023

ToUpper not working as intended with Turkish Culture #77689

ToUpper not working as intended with Turkish Culture #77689

Comments

sencagri commented Oct 31, 2022

Description

Steps to Reproduce

Link to public reproduction project repository

Version with bug

Last version that worked well

Affected platforms

Affected platform versions

Did you find any workaround?

Relevant log output

Eilon commented Oct 31, 2022

hartez commented Oct 31, 2022

hartez commented Oct 31, 2022

hartez commented Oct 31, 2022

ghost commented Oct 31, 2022

jpobst commented Oct 31, 2022

dotnet-issue-labeler bot commented Oct 31, 2022

sencagri commented Oct 31, 2022

sencagri commented Nov 1, 2022

ghost commented Nov 1, 2022

Description

Steps to Reproduce

Link to public reproduction project repository

Version with bug

Last version that worked well

Affected platforms

Affected platform versions

Did you find any workaround?

Relevant log output

tarekgh commented Nov 1, 2022

tarekgh commented Nov 1, 2022

ghost commented Nov 2, 2022

Description

Steps to Reproduce

Link to public reproduction project repository

Version with bug

Last version that worked well

Affected platforms

Affected platform versions

Did you find any workaround?

Relevant log output

marek-safar commented Nov 2, 2022

steveisok commented Nov 2, 2022 • edited Loading

NightOwl888 commented Nov 3, 2022

sencagri commented Nov 4, 2022

steveisok commented Jul 13, 2023

hartez commented Jul 18, 2023

steveisok commented Jul 18, 2023

sencagri commented Jul 18, 2023

Clockwork-Muse commented Jul 18, 2023

hartez commented Jul 19, 2023 • edited Loading

Clockwork-Muse commented Jul 19, 2023

steveisok commented Nov 2, 2022 •

edited

Loading

hartez commented Jul 19, 2023 •

edited

Loading