Keyboard Issues on x220t

[aphick]

Flashed the x220t, the only modification from master is CONFIG_CBFS_SIZE=0x750000.
I'm running into an issue on the majority of boots where the Backspace and Backslash keys are not working. However every few boots it will work.
I have noticed that when the keyboard is working it's because it is loaded as
input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0
and when it's not it shows up as
input: AT Raw Set 2 keyboard as /devices/platform/i8042/serio0/input/input0
I am running xfce on debian buster. Apologies if this isn't related to heads but due to the problem appearing after flashing I assume it is.
I can attach whatever logs would be helpful.

[snmcmillan]

I remember this issue was present on keyboard-modded X230t a long time ago. Are you using master or are you using one of the releases?

Edit: you mention master. I can't read today.
I'll have to look into it a bit.

[aphick]

No problem. I did confirm that the input: AT Raw Set 2 keyboard and input: At Translated Set 2 variances are happening when heads loads by checking dmesg in the recovery shell.

[snmcmillan]

I cannot reproduce this on my X220, and I no longer have my X230t.

On another note:
Is the keyboard counterfeit? Take a look at the ThinkVantage label. If it looks funny, it's counterfeit. Asking because my X230t saw these issues with a counterfeit classic keyboard as well.

[aphick]

It look legit to me. I'll take my computer apart and see if perhaps there's an underlying hardware issue.

[aphick]

So I checked out the hardware and there are no visible things that could be causing the issue. While I was at it I re-flashed the stock bios just to double check and sure enough the issue is not present and the keyboard showed up as At Translated on every boot.
I found a discussion on this as an issue in old kernels but their solutions didn't work for me.

[paulmenzel]

Please attach the full coreboot and Linux logs for the non-working and working case. Where you able to test the latest Linux kernel?

[snmcmillan]

Would you also mind uploading good pictures of your keyboard's top and bottom?

And if you were to restart (not power off and power back on) the machine, does the issue go away?

[aphick]

@paulmenzel I've attached the logs. The machine is currently running 4.19.98-1 but I plan on testing a different distro later today.
corebootdmesg-broken.txt
corebootdmesg-working.txt
linuxdmesg-broken.txt
linuxdmesg-working.txt

@SebastianMcMillan A reboot does not fix it. (EDIT: Today a reboot did fix it) Only fully powering off the machine will. Apologies for the dirty keyboard, I got this secondhand so it's not my mess :)

[aphick]

@paulmenzel I tried on 5.4.0-4 with debian bullseye and the issue is still present

[paulmenzel]

Thanks. The output comes from https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/input/keyboard/atkbd.c#n1064:

static void atkbd_set_device_attrs(struct atkbd *atkbd)
{
	struct input_dev *input_dev = atkbd->dev;
	int i;

	if (atkbd->extra)
		snprintf(atkbd->name, sizeof(atkbd->name),
			 "AT Set 2 Extra keyboard");
	else
		snprintf(atkbd->name, sizeof(atkbd->name),
			 "AT %s Set %d keyboard",
			 atkbd->translated ? "Translated" : "Raw", atkbd->set);
[…]

Please try to add some Linux kernel parameters to see if this changes something.

1. i8042.debug
2. i8042.reset
3. i8042.kbdreset

[paulmenzel]

I forgot to say, I believe it’s a timing issue, though I wonder why the boot kernel (non-payload) has the same problem.

[aphick]

Well that is quite the noisy debug. Let me know if you would like any logs.

I found that with the i8042.reset and/or i8042.kbdreset parameters enabled the keyboard showed up 100% of the time as AT Raw regardless of what the state was to the heads kernel.

[paulmenzel]

Just checking, you are running the latest EC firmware, right?

AT Raw means, it doesn’t work, right?

I have not seen the debug logs yet, but I am out of my comfort zone anyway. Probably it would be good to engage with the Linux subsystem developers.

Just to have more datapoints. I suppose the keyboard always correctly works with other payloads like SeaBIOS and GRUB?

[aphick]

Yes, to the EC firmware and about AT Raw.

After flashing seabios again the issue is not longer present

[paulmenzel]

After flashing seabios again the issue is not longer present

I totally missed that part. SeaBIOS does some PS/2 initialization itself, so maybe that is more correct than from the Linux kernel?

Somebody, should raise this on linux-input@vger.kernel.org, Linux’ INPUT subsystem.

[paulmenzel]

The libpayload patch series fixes some identified keyboard issues on Lenovo Thinkpads.

[i-c-o-n]

FWIW, ThinkPad keyboard controllers always start in translated mode. However the keyboards run a self-test without being asked for it and don't answer other requests while doing so. This can lead to wrong assumptions in drivers.
Maybe Linux wrongfully detects the untranslated (raw) mode, and that's the whole problem. But it could be more. I guess unexpected translation would result in many more keys failing.
i8042.direct=1 seems worth a shot. It would force the controller into untranslated mode which is also what most firmware does.

[tlaurion]

Next step is check inclusion of https://review.coreboot.org/c/coreboot/+/47088/4/payloads/libpayload/drivers/i8042/keyboard.c as recommended per @paulmenzel #698 (comment)

[tlaurion]

@0rb677 do you own a x220t platform?

[tlaurion]

Tag me in to reopen. This is fixed as part or ps2 coreboot init of keyboard and was tested on x230 tablet on my side