-
Notifications
You must be signed in to change notification settings - Fork 975
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix System.Windows.Forms.InputLanguage.FromCulture() for languages without LANGID value #8573
Conversation
There are a couple of test failures. Great job in tracking this down! |
I tried to use CultureInfo constructor with LANGID parameter but it turns out that the LCIDToLocaleName API, which is used inside CultureInfo, may return incorrect language tags for transient language identifiers. For example, it returns "nqo-GN" and "jv-Java-ID" instead of the "nqo" and "jv-Java"(as seen in the Get-WinUserLanguageList PowerShell cmdlet). I had to use undocumented Bcp47FromHkl API that returns valid values in these cases. I still need to add some proper code to test these locales with transient LCIDs. |
This comment was marked as resolved.
This comment was marked as resolved.
src/System.Windows.Forms.Primitives/src/Windows/Win32/PInvoke.Bcp47FromHkl.cs
Outdated
Show resolved
Hide resolved
src/System.Windows.Forms/src/System/Windows/Forms/InputLanguage.cs
Outdated
Show resolved
Hide resolved
@merriemcgaw I think @JeremyKuhne will need to review this because of the addition of a DLL Import? Does Microsoft have some documentation around As far as I can tell @mikebattista please correct me if I way off track here and it isn't a kernel DLL and just missing from Win32Metadata. |
@dotnet-policy-service agree |
This comment was marked as resolved.
This comment was marked as resolved.
Thanks for the heads up. We had something similar in other PRs. @dmitrii-drobotov there is another ARM test failure in this issue. Is it related to #8672 (comment)? |
src/System.Windows.Forms.Primitives/src/Interop/Interop.Libraries.cs
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great work! I imagine the failures are flaky tests. But the team will be able to tell you after reviewing logs.
src/System.Windows.Forms.Primitives/src/Windows/Win32/PInvoke.Bcp47FromHkl.cs
Outdated
Show resolved
Hide resolved
Yes, it looks like a flaky test not related to this PR. Stack trace
It may be worth rebasing on the latest main to get the fix from #8695. |
I found another way to convert the transient LANGID into a language tag (without a bug for transient LANGID language tags), which does not depend on the undocumented int langId = PARAM.LOWORD(_handle);
using RegistryKey? key = Registry.CurrentUser.OpenSubKey(@"Control Panel\International\User Profile");
if (key is not null && key.GetValue("Languages") is string[] languages)
{
foreach (string language in languages)
{
using RegistryKey? subKey = key.OpenSubKey(language);
if (subKey is not null && subKey.GetValue("TransientLangId") is int transientLangId &&
transientLangId == langId)
{
return language;
}
}
}
return CultureInfo.GetCultureInfo(langId).Name; I have tested this locally and everything looks good. |
Anything undocumented is probably a no-go. Since I believe Microsoft wouldn't consider it a "public API". We will have to wait for the response from someone in Microsoft working on Windows to let us know what is possible. It might just mean |
@elachlan yes, we cannot use undocumented Windows APIs in the shipping binaries. There are long standing legal reasons, so there isn't a lot of flexibility. We'll try to get things documented so we can close this gap properly. This sort of thing does come up from time to time, but it is infrequent enough that it is always a little rocky. Thanks for your patience on this everyone. |
@JeremyKuhne updated PR. please take a look |
src/System.Windows.Forms/tests/UnitTests/System/Windows/Forms/InputLanguageTests.cs
Outdated
Show resolved
Hide resolved
Many supported Windows languages do not have a unique pre-assigned LANGID to identify them and instead use a value 0x1000 (LOCALE_CUSTOM_UNSPECIFIED) since Windows 8. Windows tries to allocate and provide a unique transient value (from a pool four values: 0x2000, 0x2400, 0x2800, 0x2C00) in these cases in the HKL lowerword to distinguish between different languages via the old Win32 APIs, but these values can't be used to identify the keyboard layout directly, and in result we can't get the InputLanguage correctly in the FromCulture method. Aquire corresponging IETF BCP 47 language tag with a call to a proper API and compare locale names instead of LANGID to fix these corner cases. I tried to use CultureInfo constructor with LANGID parameter but it turns out that the LCIDToLocaleName API, which is used inside CultureInfo, may return incorrect language tags for transient language identifiers. For example, it returns "nqo-GN" and "jv-Java-ID" instead of the "nqo" and "jv-Java"(as seen in the Get-WinUserLanguageList PowerShell cmdlet). I had to use undocumented registry keys to to extract proper language tag from a LANGID. This workaround was approved by a Windows team. dotnet#8573 (comment)
@JeremyKuhne @dreddy-work any updates on this? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry about the delays in responses, been very busy. Just a few more comments. Please tag me when you've responded.
@@ -180,6 +172,62 @@ internal string LayoutId | |||
} | |||
} | |||
|
|||
private static readonly int[] s_transientLangIds = | |||
{ | |||
0x2000, // LOCALE_TRANSIENT_KEYBOARD1 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These should be added to the NativeMethods.txt file in System.Windows.Forms.Primitives.
0x2C00 // LOCALE_TRANSIENT_KEYBOARD4 | ||
}; | ||
|
||
private static string s_userProfileRegistryPath => @"Control Panel\International\User Profile"; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should be a constant, not a static.
src/System.Windows.Forms/src/System/Windows/Forms/InputLanguage.cs
Outdated
Show resolved
Hide resolved
Many supported Windows languages do not have a unique pre-assigned LANGID to identify them and instead use a value 0x1000 (LOCALE_CUSTOM_UNSPECIFIED) since Windows 8. Windows tries to allocate and provide a unique transient value (from a pool four values: 0x2000, 0x2400, 0x2800, 0x2C00) in these cases in the HKL lowerword to distinguish between different languages via the old Win32 APIs, but these values can't be used to identify the keyboard layout directly, and in result we can't get the InputLanguage correctly in the FromCulture method. Aquire corresponging IETF BCP 47 language tag with a call to a proper API and compare locale names instead of LANGID to fix these corner cases. It turns out that the LCIDToLocaleName API, which is used inside CultureInfo, may return incorrect language tags for transient language identifiers. For example, it returns "nqo-GN" and "jv-Java-ID" instead of the "nqo" and "jv-Java" (as seen in the Get-WinUserLanguageList PowerShell cmdlet). I had to use undocumented registry keys to to extract proper language tag from a LANGID. This workaround was approved by a Windows team. dotnet#8573 (comment)
@JeremyKuhne please take another look at this. I've made corrections. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for your patience!
src/System.Windows.Forms/src/System/Windows/Forms/InputLanguage.cs
Outdated
Show resolved
Hide resolved
@JeremyKuhne any updates? |
Great work @DJm00n! |
Verified it in the Release/8.0-Preview6 branch from this repo, result of running InputLanguage_FromCulture_Roundtrip_Success test is failed as following. Is there some wrong from my setting? @DJm00n |
Hi @Olina-Zhang! |
@DJm00n Here is Get-WinUserLanguageList PowerShell cmdlet output: |
@Olina-Zhang interesting...
This looks like some kind of configuration issue on your PC. If it is an RDP connection then it could be related too (since it can install keyboard layouts from the remote system): You can also check these registry paths for this suspicious "00070C00" keyboard layout:
Or attach files generated from these commands in PowerShell:
|
@DJm00n Thanks for your comment! The testing machine I used is connected by remoting. It may be related some machine setting. Today re-tested that test case: |
Since Windows 8 many locale languages do not have a unique pre-assigned LANGID/LCID to identify them and in these cases we can have transient keyboard locale identifier values in the HKL lowerword:
0x2000
- LOCALE_TRANSIENT_KEYBOARD10x2400
- LOCALE_TRANSIENT_KEYBOARD20x2800
- LOCALE_TRANSIENT_KEYBOARD30x2C00
- LOCALE_TRANSIENT_KEYBOARD4This is done to distingush up to 4
0x1000
(LOCALE_CUSTOM_UNSPECIFIED) or0x0C00
(LOCALE_CUSTOM_DEFAULT) languages via old Win32 APIs.But these values can't be used to identify the input language directly (we must convert it to BCP 47 language tag as soon as possible and use language tags where possible), and in result we can't get the InputLanguage correctly in the FromCulture method and may have wrong Culture in some cases..
More info about transient lang ids.
PS: Some background info on CultureInfo.KeyboardLayoutId that was used in FromCulture() method before my changes. Long story short - it should be avoided. :)
Proposed changes
Customer Impact
Regression?
Risk
Screenshots
Before
Get-WinUserLanguageList
PowerShell cmdlet output:After
Test methodology
Autotesting:
Manually:
InputLanguage_FromCulture_Roundtrip_Success
test and see the resultsAccessibility testing
Test environment(s)
Related issue in Runtime: dotnet/runtime#81633