Some keys on Japanese layout keyboards are incorrectly interpreted as @

[hiro-mi]

Environment

PS version: 7.1.1
PSReadline version: 2.1.0
os: 10.0.19041.800 (WinBuild.160101.0800)
PS file version: 7.1.1.0
HostName: ConsoleHost (Windows Terminal)
BufferWidth: 120
BufferHeight: 30

Exception report

N/A

Steps to reproduce

1. Open powershell with Japanese layout keyboard.
2. Press "無変換(Muhenkan)" key on keyboard.

Expected behavior

This is not character key - so nothing should happen and be displayed in console.

Actual behavior

Console prints @ key.

Backgrounds

I ran [console]::ReadKey() in pwsh and press "無変換(Muhenkan)" key.

KeyChar Key Modifiers
------- --- ---------
        29         0

Japanese is not an alphabet, so I use an IME - Input Method Editor (e.g. Microsoft IME) to type it.
The "Muhenkan" key is for IME operation, so it does not send any characters.

It didn't occur with the "半角/全角(Hankaku/Zenkaku)" key and the "変換(Henkan)" key for operating Japanese IME.
However, if I press it together with the ALT key, the "@" will be displayed like "Muhenkan" key.

[tsukumijima]

I'm having the same problem. I hope it will be resolved as soon as possible.

[cm3]

PSReadLine/PSReadLine/Keys.cs

Lines 295 to 302 in ad74cef

	case '\0':
	// This could be a dead key for a particular keyboard layout in Windows console.
	// The dead key is not an issue when there is tty involved, so on non-Windows, `isDeadKey` is always false.
	//
	// When we believe it's a dead key, we use '\0' so it can be properly ignored during rendering.
	// Otherwise, we use `@` in case `key.KeyChar = '\0'`. This is ugly but familiar.
	s = isDeadKey ? c.ToString() : "@";
	break;

I'm facing the same problem. This practice is not only ugly but also problematic.

[cm3]

PSReadLine/PSReadLine/Keys.cs

Lines 206 to 220 in 15850a4

	// Keys we want to ignore (and handling here will help to ignore)
	case ConsoleKey.BrowserBack: case ConsoleKey.BrowserForward: case ConsoleKey.BrowserRefresh: case ConsoleKey.BrowserStop:
	case ConsoleKey.BrowserSearch: case ConsoleKey.BrowserFavorites: case ConsoleKey.BrowserHome:
	case ConsoleKey.VolumeMute: case ConsoleKey.VolumeDown: case ConsoleKey.VolumeUp:
	case ConsoleKey.MediaNext: case ConsoleKey.MediaPrevious: case ConsoleKey.MediaStop: case ConsoleKey.MediaPlay:
	case ConsoleKey.LaunchMail: case ConsoleKey.LaunchMediaSelect: case ConsoleKey.LaunchApp1: case ConsoleKey.LaunchApp2:
	case ConsoleKey.LeftWindows: case ConsoleKey.RightWindows: case ConsoleKey.Applications:
	case ConsoleKey.PrintScreen:
	// Keys I'm not familiar with, presumably we'd want to ignore.
	case ConsoleKey.Clear: case ConsoleKey.Pause: case ConsoleKey.Select: case ConsoleKey.Print:
	case ConsoleKey.Execute: case ConsoleKey.Help: case ConsoleKey.Sleep: case ConsoleKey.Process:
	case ConsoleKey.Attention: case ConsoleKey.CrSel: case ConsoleKey.ExSel:
	case ConsoleKey.EraseEndOfFile: case ConsoleKey.Play: case ConsoleKey.Zoom: case ConsoleKey.NoName:
	case ConsoleKey.Pa1:

To add 0x1D (29) here will solve the problem. On the other hand, that constant should be available as ConsoleKey.NonConvert or such, so I added an issue to the .NET framework.

[cm3]

I happened to find that using the new version of the Microsoft IME solve this problem in my environment. That IME had many bugs and I've used the previous version for a long time https://support.microsoft.com/en-us/windows/revert-to-a-previous-version-of-an-ime-input-method-editor-adcc9caa-17cb-44d8-b46e-f5b473b4dd77.

[daxian-dbw]

@hiro-mi and @tsukumijima Can you try the new version of Microsoft IME and see if that works for you too?

[cm3]

I have three PCs (two Win10 and one Win 11), and all the devices are now free from this problem thanks to the new version of Microsoft IME. The key code 29 is still sent (I checked that on https://keycode.info/), but the problem never happens. As @hiro-mi reported, with the Alt key, @ is still input. That doesn't matter for most of the people, I think, however.

[tsukumijima]

I'm usually using Google Japanese Input. When I tried the new Microsoft-IME, pressing the "Muhenkan" key no longer typed "@".
However, as mentioned earlier, the new Microsoft-IME has many bugs. I'm used to Google Japanese Input, so I don't think I'll switch to the new Microsoft-IME just for this.

[ataihei]

On US layout keyboards, the "Hankaku/Zenkaku" key does not exist.
If you want to do the same, you need to type Alt + ` , in which case @ will be typed as well as Alt + Hankaku/Zenkaku.
This occurs not only with Google IME, but also with MS-IME.
We hope this will be fixed, as it is a key binding frequently used by Japanese users of US keyboards.

[StevenBucher98]

Just so I am understanding this correct, the MS IME provides different key handlers for keys not available on US keyboards and the problem is that PSReadLine is not recognizing them correctly? Is this issue still persistent with latest versions of MS IME and PSReadLine? There may be a need to unbind default key handlers that conflict

[hiro-mi]

@StevenBucher98
I have tested with Powershell 7.3.3 and this issue continues to occur.
My environment as output by "$PSVersionTable" is as follows

PSVersion 7.3.3
PSEdition Core
GitCommitId 7.3.3
OS Microsoft Windows 10.0.19045
Platform Win32NT
PSCompatibleVersions {1.0, 2.0, 3.0, 4.0...}
PSRemotingProtocolVersion 2.3
SerializationVersion 1.1.0.1
WSManStackVersion 3.0

However, the same Issue does not occur with the following versions in the same PC.

PSVersion 5.1.19041.2673
PSEdition Desktop
PSCompatibleVersions {1.0, 2.0, 3.0, 4.0...}
BuildVersion 10.0.19041.2673
CLRVersion 4.0.30319.42000
WSManStackVersion 3.0
PSRemotingProtocolVersion 2.3
SerializationVersion 1.1.0.1

The changes that were made between these versions may give some clues.