WndProc/AddInputCharacter issue in Slovak layout.

[sneakyevil]

Version/Branch of Dear ImGui:

Version: 1.89.8
Branch: master

Back-end/Renderer/Compiler/OS

Back-ends: imgui_impl_win32.cpp + imgui_impl_dx11.cpp
Compiler: MSVC (Multi-byte Character Set)
Operating System: Windows 10

My Issue/Question:

I got some issue with WndProc and I couldn't find anything at google or anyone having similar issue besides the russian input posted here as issue like year ago, but this issue is kinda weird. So first at all I have loaded correct font with glyph range that can display the characters I'm trying to type and I even tried to manually insert the utf-8 encoded characters that I tried to type and they display correctly. The issue is when I press "4" on top of my keyboard which translates to "č" in Slovak layout it is passed in WndProc as wparam of 232 which instead displaying as "č" it display as "è" even after MultiByteToWideChar conversion. When I add primitive if check before io.AddInputCharacter call to check if wch is 232 and change it to 269 it will always properly display the "č" character without any issue.

Here are some characters typed on the Slovak layout on top keyboard from "1, 2, ... 0":

• In ImGui: "+¾šè�žýáíé"
• Outside ImGui: "+ľščťžýáíé"

I have no idea how to even solve this issue besides manually doing conversion to it and I still have no clue if this issue is present only on my system or anyone else.

[ocornut]

č is U+010D == 269 so that value is right.
https://www.compart.com/en/unicode/U+010D

Your issue has to do with Win32 API supports of Unicode and UTF+8.

I reckon you have the same issue as #5725.
As you are compiling in MBCS mode, you need to be calling CreateWindowW() when creating your window. CreateWindow() will call CreateWindowA().

See our solution
a229a7f
and 0a7054c

Compiler: MSVC (Multi-byte Character Set)

Why not switching to Unicode?

[sneakyevil]

Well this does indeed fix the issue by using CreateWindowW even in MBCS mode, but is there any solution for Ascii?
The reason why I can't switch to unicode is due to that I working on modification for game that uses CreateWindowExA.

[sneakyevil]

I think there could be some implementation done in ImGui for the ascii version, but I have no idea how to properly get this thing to even work. Changing the codepage to 1250 for MultiByteToWideChar will properly fix the issue of conversion, but at the same time this is hardcoded. Calling GetACP returns for me 1252 and same value I get when using GetLocaleInfoA for some odd reason. Using that codepage will break the conversion same way as before. There is wndproc msg WM_INPUTLANGCHANGE that could be used to somehow implement the codepage fetch properly and change it for the conversion.

[ocornut]

I don't know, you will need to dig (also check linked issues and other issues linked to it and to its commit).
If you find a solution that's not too complex I'm open to including it in imgui_impl_win32.

[sneakyevil]

No idea if this is proper way of getting code page, but it seems to work for me.

The code for getting codepage:

HKL m_KeyboardLayout = GetKeyboardLayout(0);

LCID m_KeyboardLCID = MAKELCID(HIWORD(m_KeyboardLayout), SORT_DEFAULT);
    
unsigned int m_CodePage = CP_ACP;
GetLocaleInfoA(m_KeyboardLCID, LOCALE_RETURN_NUMBER | LOCALE_IDEFAULTANSICODEPAGE, (LPTSTR)&m_CodePage, sizeof(m_CodePage));

Using this with wndproc WM_INPUTLANGCHANGE will return proper codepage when switching layout.
When switching from Slovak to English layout the code page is 1252, switching from English to Slovak the code page is 1250.

[ocornut]

Closed by #6785

[PathogenDavid]

I've had in my back of my head that I wanted to look into this more since #6785 didn't quite pass the sniff test for me (for reasons similar to Omar's concerns raised on the PR.)

I'm not confident using GetKeyboardLayout is the technically correct thing to do here.

WM_CHAR is defined as follows for non-Unicode windows:

[...] the system provides characters in the current process code page

Is the keyboard layout always the same as the current process code page? My instinct says no, but maybe that's the default?

If it's only the default, I would be concerned the adopted fix will end up breaking apps manifested as UTF-8 since in those apps all the old-school ANSI stuff will be UTF-8 but it's unclear if that'd extend to the keyboard layout. (Would need to investigate further.)

I could not find a 100% definitive API for getting the "process code page", I suspect that should actually probably read "thread code page", IE: CP_THREAD_ACP.

I'd be interested in hearing whether using the old code with CP_THREAD_ACP instead of CP_ACP fixes this issue. (I'm not currently set up to easily reproduce this bug, otherwise I'd check for myself.)

It'd also be worth checking if there's a disagreement between CP_ACP and CP_THREAD_ACP when this bug occurs. You can check with GetCPInfoExW:

CPINFOEXW info = { };
::GetCPInfoExW(CP_ACP, 0, &info);
wprintf(L"       CP_ACP: '%s'\n", info.CodePageName);
::GetCPInfoExW(CP_THREAD_ACP, 0, &info);
wprintf(L"CP_THREAD_ACP: '%s'\n", info.CodePageName);

[ocornut]

I did back then try to run GetCPInfoEx() on all meaningful defines including CP_THREAD_ACP and didn’t get any difference.

[sneakyevil]

It'd also be worth checking if there's a disagreement between CP_ACP and CP_THREAD_ACP when this bug occurs. You can check with GetCPInfoExW:

This has been discussed in the pull request that it doesn't change anything. Also I mention there that I fixed it by changing the language for non-unicode programs in advanced language settings in windows settings, which seems weird to even use that approach if you want for example change keyboard layout on the fly and type in different language.

[PathogenDavid]

This has been discussed in the pull request that it doesn't change anything.

Oops, I don't know how I missed/forgot that considering it was right at the start. Sorry about that!