Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Keys at the same time #4904

Closed
stygmate opened this issue Jan 21, 2019 · 24 comments
Closed

Keys at the same time #4904

stygmate opened this issue Jan 21, 2019 · 24 comments
Labels
bug help wanted question stale Issues or pull requests that have become inactive without resolution.

Comments

@stygmate
Copy link

Is there a solution for having simultaneous hitted keys be reported in the same sub report.
for now the best i get is:

00 00 00 00  00 00 00 00  <- no input
00 00 00 00  00 00 00 00  
00 00 04 00  00 00 00 00  <- three keys stroked exactly at the same time
00 00 04 16  00 00 00 00  <- but interpreted as 3 report
00 00 04 16  07 00 00 00  < - ...
@drashna
Copy link
Member

drashna commented Jan 21, 2019

IIRC, it's one key per scan/report. Normally.

I think what you're lookin for is maybe this:

#define QMK_KEYS_PER_SCAN 4

Try adding that to your config.h, and see if it gets what you want.

@stygmate
Copy link
Author

stygmate commented Jan 22, 2019

Not better with this settings.
If i understand correctly each key generate a report, reports are buffered until computer poll ?

another question:
how to change polling rate in hid descriptors (stm32 based keyboard) ?

@fauxpark
Copy link
Member

fauxpark commented Jan 22, 2019

The host doesn't poll the keyboard, QMK only sends reports when it needs to.

It looks as though action_exec() is called immediately upon finding a changed key state, and I think somewhere down in there the actual report is sent as part of the keycode's action. So I very much doubt there is a way to send multiple keys at once.

If the STM32 boards run ChibiOS then the polling rates will be in tmk_core/protocol/usb_descriptor.c.

@alex-ong
Copy link
Contributor

alex-ong commented Apr 5, 2019

Hi, i've narrowed down this bug to this commit, which joins shared Endpoints together.
39bd760

First let's talk about expected results.
QMK_KEYS_PER_SCAN = 4
Lets say scanrate = 300hz
Press 4 keys within 10ms
Expected result: 1 to 4 messages within a 10-15ms window.
Actual result: always 4 messages with 5-8ms gap between = 20ms window.

Scanrate = 1000hz
Press 8 keys with 10ms
Expected result: 1 to 8 messages within a 10-15ms window.
Actual result: always 8 messages with 8ms apart, for 64ms total time.

This is especially annoying in rhythm games like osu! mania,
which require you to press 6 keys simultaneously in a 20ms window. Due to the bug, it will always report the keys in 48ms, so half your keys will register fine and the other half wont, even though you are hitting all of them in a 10-20ms window of each other.

I used this program i wrote (note: it's a game that relies on having at least 1000hz frame rate):
https://github.com/alex-ong/UnityInputDelayTester/releases

Before the bug, if i mashed 8 keys, it would report them all within 10ms, usually 4-6 different timestamps. Same timestamp would mean same usb report.

After the bug, it would always report exactly 8 timestamps exactly 8ms apart.
This is even when setting polling speed to 1ms.

Note there are two bugs here:

  1. seperate reports even though qmk_keys_per_scan is 4
  2. reports have 8ms gap even though it should be more like 1-2ms.

@drashna
Copy link
Member

drashna commented Apr 5, 2019

For the endpoint change, you can change some of the config, and that may help:
https://docs.qmk.fm/#/config_options?id=usb-endpoint-limitations
But it may not.

@alex-ong
Copy link
Contributor

alex-ong commented Apr 5, 2019

Setting KEYBOARD_SHARED_EP = yes and
KEYBOARD_SHARED_EP = no didn't do anything

Is there anything else i should be looking at?

@drashna
Copy link
Member

drashna commented Apr 5, 2019

Try SHARED_EP_ENABLE = no

@BeefaloKing
Copy link
Contributor

After the bug, it would always report exactly 8 timestamps exactly 8ms apart.
This is even when setting polling speed to 1ms.

I do not see this. When I change the polling rates in tmk_core/protocol/usb_descriptor.c I see timestamps exactly* 1ms apart (Using the same Unity test program linked above) instead of 8ms.

*#define QMK_KEYS_PER_SCAN 4 slightly changes this behavior. When set to 4, I'll see 4 key presses exactly 1ms apart, then some additional delay (+1 or 2ms) before seeing the next 4 key presses exactly 1ms apart. When set to 2 I see additional delay every 2 keys, etc.

SHARED_EP_ENABLE = no
and
KEYBOARD_SHARED_EP = no
don't appear to have any effect.

@alex-ong
Copy link
Contributor

I do not see this. When I change the polling rates in tmk_core/protocol/usb_descriptor.c I see timestamps exactly* 1ms apart (Using the same Unity test program linked above) instead of 8ms.

*#define QMK_KEYS_PER_SCAN 4 slightly changes this behavior. When set to 4, I'll see 4 key presses exactly 1ms apart, then some additional delay (+1 or 2ms) before seeing the next 4 key presses exactly 1ms apart. When set to 2 I see additional delay every 2 keys, etc.

Nice; just confirming that you are using FORCE_NKRO and mashing 8+ keys?
My test cases simply latest master (with polling rate changed) and master with 39bd760 unpatched, on a FORCE_NKRO keyboard.

Before the bug, if i mashed 8 keys, it would report them all within 10ms, usually 4-6 different timestamps. Same timestamp would mean same usb report.

Now that i think about it, QMK sends a usb report after each key change, even if QMK_KEYS_PER_SCAN == 4, so technically what i said about two+ keys on same timestamp shouldnt be possible right?

I'll do a quick retest and see what I get...

@BeefaloKing
Copy link
Contributor

just confirming that you are using FORCE_NKRO and mashing 8+ keys?

Yes, though I do see the same behavior without NKRO (minus not seeing all 8 keys).

@stale
Copy link

stale bot commented Nov 21, 2019

This issue has been automatically marked as resolved because it has not had activity in the last 90 days. It will be closed in the next 30 days unless it is tagged properly or other activity occurs.

@stale stale bot added the solved label Nov 21, 2019
@stygmate
Copy link
Author

Dear Stale Bot, this message to let you know you're wrong.

@stale stale bot removed the solved label Nov 21, 2019
@drashna drashna added the bug label Nov 21, 2019
@xyzz
Copy link
Contributor

xyzz commented Jan 2, 2020

This change https://gist.github.com/xyzz/07c7473d0739cac054cb24792bcdbb04 is a quick hack I threw together to have a single report contain multiple keys. With both

#define USB_POLLING_INTERVAL_MS 1
#define QMK_KEYS_PER_SCAN 4

slamming 3 keys 3 times before the change:

diff: 120.115000
diff: 0.831000
diff: 1.012000
diff: 118.124000
diff: 0.853000
diff: 1.101000
diff: 125.053000
diff: 0.914000
diff: 1.000000

every key is sent as its own report at 1ms intervals. After the change:

diff: 125.013000
diff: 0.024000
diff: 0.020000
diff: 121.069000
diff: 0.036000
diff: 0.008000
diff: 114.775000
diff: 0.024000
diff: 0.021000

keys are combined in reports.

@alex-ong
Copy link
Contributor

alex-ong commented Jan 2, 2020

Nice!

@xyzz
Copy link
Contributor

xyzz commented Jan 4, 2020

After reading the code some more, here's the simplified version:

diff --git a/tmk_core/common/action.c b/tmk_core/common/action.c
index 7fbdbd8c3..b5555757a 100644
--- a/tmk_core/common/action.c
+++ b/tmk_core/common/action.c
@@ -771,7 +771,7 @@ void register_code(uint8_t code) {
 #endif
             {
                 add_key(code);
-                send_keyboard_report();
+                // send_keyboard_report();
             }
         }
     else if
@@ -837,7 +837,7 @@ void unregister_code(uint8_t code) {
     else if
         IS_KEY(code) {
             del_key(code);
-            send_keyboard_report();
+            // send_keyboard_report();
         }
     else if
         IS_MOD(code) {
diff --git a/tmk_core/common/keyboard.c b/tmk_core/common/keyboard.c
index 794a9152f..1f2d68c8f 100644
--- a/tmk_core/common/keyboard.c
+++ b/tmk_core/common/keyboard.c
@@ -30,6 +30,7 @@ along with this program.  If not, see <http://www.gnu.org/licenses/>.
 #include "sendchar.h"
 #include "eeconfig.h"
 #include "action_layer.h"
+#include "action_util.h"
 #ifdef BACKLIGHT_ENABLE
 #    include "backlight.h"
 #endif
@@ -331,6 +332,8 @@ void keyboard_task(void) {
 
 MATRIX_LOOP_END:
 
+    send_keyboard_report();
+
 #ifdef DEBUG_MATRIX_SCAN_RATE
     matrix_scan_perf_task();
 #endif

This however will break literally everything as so many features depend on register_code and unregister_code sending the report immediately. I'm really not sure if there's a good way to implement this...

@BeefaloKing
Copy link
Contributor

While I can get both https://gist.github.com/xyzz/07c7473d0739cac054cb24792bcdbb04 and your simplified version to work (on my board at least), with the simplified version wireshark captures a URB_INTERRUPT every 1ms whereas the first version I only see an interrupt when I press/release a key.

Not sure which is more correct or what could potentially break with this change, but I can confirm it does send multiple keys per report.

@alex-ong
Copy link
Contributor

alex-ong commented Feb 20, 2020

Right, you can see in the modification:

 MATRIX_LOOP_END:
    send_keyboard_report();

this sends a report every time the loop finishes scanning. This would also limit your scanrate to the USB_POLLING_INTERVAL_MS which is at most 1000hz, since send_keyboard_report() is blocking afaik.

While I can get both https://gist.github.com/xyzz/07c7473d0739cac054cb24792bcdbb04 and your simplified version to work (on my board at least), with the simplified version wireshark captures a URB_INTERRUPT every 1ms whereas the first version I only see an interrupt when I press/release a key.

I wish i checked this gist first, because it's exactly what i ended up implementing (send was moved to after MATRIX_LOOP_END, and i check the needs_send inside the send function).

Unfortunately as xyzz said, it breaks everything that requires modifiers etc on my board which uses very minimal feature set, but passes the multiple-keys in same report test

hongaaronc added a commit to hongaaronc/qmk_firmware that referenced this issue Nov 15, 2020
This change defers calling "host_keyboard_send(keyboard_report)" until after the matrix loop.

This enables the ability to send multiple key changes in a single report, which also means that multiple key changes can be detected in a single polling cycle. This essentially allows us report more information in the same amount of time at no additional cost.

Before this change, only one key change (a press, or release) could be sent at a time. This is because "host_keyboard_send(..)" was created as a blocking method so that data cannot be changed mid-transmission to host, and this method was formerly called immediately after any change was made to the keyboard report. Calling this method at this point meant that the remainder of the polling interval would be used to send the report to host.

The impact of this can be very significant in gaming scenarios, when multiple keypresses need to be triggered simultaneously. See issue qmk#4904 for more details.

I tested the change manually on a Fox Leaf 60 and a Romac, and compared directly against the same firmware without the change. I've confirmed in manual testing that this allows multiple simultaneous keypresses, while not seeming to introduce any new issues.

Exact changes include:
- In "send_keyboard_report(..)" method, flag the last report's state as dirty instead of immediately sending report to host
- Create a new "send_keyboard_report_immediate(..)" method which is called after the matrix loop, which checks if the last report sent to host is dirty and sends an updated report if so, then sets it to clean
KarlK90 pushed a commit to KarlK90/qmk_firmware that referenced this issue Sep 15, 2021
This change defers calling "host_keyboard_send(keyboard_report)" until after the matrix loop.

This enables the ability to send multiple key changes in a single report, which also means that multiple key changes can be detected in a single polling cycle. This essentially allows us report more information in the same amount of time at no additional cost.

Before this change, only one key change (a press, or release) could be sent at a time. This is because "host_keyboard_send(..)" was created as a blocking method so that data cannot be changed mid-transmission to host, and this method was formerly called immediately after any change was made to the keyboard report. Calling this method at this point meant that the remainder of the polling interval would be used to send the report to host.

The impact of this can be very significant in gaming scenarios, when multiple keypresses need to be triggered simultaneously. See issue qmk#4904 for more details.

I tested the change manually on a Fox Leaf 60 and a Romac, and compared directly against the same firmware without the change. I've confirmed in manual testing that this allows multiple simultaneous keypresses, while not seeming to introduce any new issues.

Exact changes include:
- In "send_keyboard_report(..)" method, flag the last report's state as dirty instead of immediately sending report to host
- Create a new "send_keyboard_report_immediate(..)" method which is called after the matrix loop, which checks if the last report sent to host is dirty and sends an updated report if so, then sets it to clean
KarlK90 pushed a commit to KarlK90/qmk_firmware that referenced this issue Oct 14, 2021
This change defers calling "host_keyboard_send(keyboard_report)" until after the matrix loop.

This enables the ability to send multiple key changes in a single report, which also means that multiple key changes can be detected in a single polling cycle. This essentially allows us report more information in the same amount of time at no additional cost.

Before this change, only one key change (a press, or release) could be sent at a time. This is because "host_keyboard_send(..)" was created as a blocking method so that data cannot be changed mid-transmission to host, and this method was formerly called immediately after any change was made to the keyboard report. Calling this method at this point meant that the remainder of the polling interval would be used to send the report to host.

The impact of this can be very significant in gaming scenarios, when multiple keypresses need to be triggered simultaneously. See issue qmk#4904 for more details.

I tested the change manually on a Fox Leaf 60 and a Romac, and compared directly against the same firmware without the change. I've confirmed in manual testing that this allows multiple simultaneous keypresses, while not seeming to introduce any new issues.

Exact changes include:
- In "send_keyboard_report(..)" method, flag the last report's state as dirty instead of immediately sending report to host
- Create a new "send_keyboard_report_immediate(..)" method which is called after the matrix loop, which checks if the last report sent to host is dirty and sends an updated report if so, then sets it to clean
@ecerulm
Copy link

ecerulm commented Nov 24, 2021

@xyzz , @alex-ong , for my understanding,could you please explain a feature that gets broken by this change and why?

@alex-ong
Copy link
Contributor

@xyzz , @alex-ong , for my understanding,could you please explain a feature that gets broken by this change and why?

I'm not sure what you're asking? This is a bug not a PR. There isn't a solution yet afaik (or at least no one has looked into the vast amounts of underlying mechanisms that need changing)

@ecerulm
Copy link

ecerulm commented Nov 24, 2021

You said

Unfortunately as xyzz said, it breaks everything that requires modifiers etc on my board which uses very minimal feature set, but passes the multiple-keys in same report test.

So you tested some fix (https://gist.github.com/xyzz/07c7473d0739cac054cb24792bcdbb04) and it breaks some other QMK features if I understood that right, I just want to understand what kind of features stop working and why. You said "everything that requires modifiers" but like what exactly and how it breaks. I just want to understand how other features rely on register_code sending the report immediately.

@KarlK90
Copy link
Member

KarlK90 commented Nov 24, 2021

@ecerulm @alex-ong Just a heads up. I have been working on a PR #12686 for quite some time that fixes this issue but has some bugs remaining that will be fixed once we have more unit-tests in place.

@alex-ong
Copy link
Contributor

alex-ong commented Nov 24, 2021

You said "everything that requires modifiers" but like what exactly and how it breaks. I just want to understand how other features rely on register_code sending the report immediately.

@ecerulm
Well basically if you hold down FN1 then tap a key you wont layer shift, and shift + button, ctrl + button etc all don't work.
I don't know why it breaks because if i did, i would have patched it and sent a PR through! :D, all i know is there's some underlying reliance on register_code sending the report immediately. I hope my explaination has added some info but really i don't know, it just doesnt work with that simple fix

So best bet is to track #12686 since it looks like people are working on this very long lasting bug (i know it didnt exist in TMK, since i used TMK years and years ago)

@github-actions
Copy link

This issue has been automatically marked as stale because it has not had activity in the last 90 days. It will be closed in the next 30 days unless it is tagged properly or other activity occurs.
For maintainers: Please label with bug, in progress, on hold, discussion or to do to prevent the issue from being re-flagged.

@github-actions github-actions bot added the stale Issues or pull requests that have become inactive without resolution. label Jun 17, 2022
@tzarc
Copy link
Member

tzarc commented Jun 18, 2022

Closing due to inactivity.

@tzarc tzarc closed this as completed Jun 18, 2022
acyr0 added a commit to acyr0/qmk_firmware that referenced this issue Feb 6, 2024
This change defers calling "host_keyboard_send(keyboard_report)" until after the matrix loop.

This enables the ability to send multiple key changes in a single report, which also means that multiple key changes can be detected in a single polling cycle. This essentially allows us report more information in the same amount of time at no additional cost.

Before this change, only one key change (a press, or release) could be sent at a time. This is because "host_keyboard_send(..)" was created as a blocking method so that data cannot be changed mid-transmission to host, and this method was formerly called immediately after any change was made to the keyboard report. Calling this method at this point meant that the remainder of the polling interval would be used to send the report to host.

The impact of this can be very significant in gaming scenarios, when multiple keypresses need to be triggered simultaneously. See issue qmk#4904 for more details.

I tested the change manually on a Fox Leaf 60 and a Romac, and compared directly against the same firmware without the change. I've confirmed in manual testing that this allows multiple simultaneous keypresses, while not seeming to introduce any new issues.

Exact changes include:
- In "send_keyboard_report(..)" method, flag the last report's state as dirty instead of immediately sending report to host
- Create a new "send_keyboard_report_immediate(..)" method which is called after the matrix loop, which checks if the last report sent to host is dirty and sends an updated report if so, then sets it to clean

Gate deferred keyboard reports feature behind an optional define

Create a define to enable the experimental "deferred keyboard reports" feature (the feature is off by default).

- I manually tested a Fox Leaf 60 firmware with the define on and firmware with the define off.
- All unit tests pass when feature define is not specified.
- 9 unit tests fail when feature define is specified, keeping it in experimental status.

Refactor "defer send keyboard report" feature to support macros

This refactor makes the "defer send keyboard report" feature compatible with macros.

Rather than modify the functionality of all "send_keyboard_report(..)" calls - the method now sends the keyboard report immediately like before. A new function called "send_keyboard_report_deferred(..)" handles the deferred support if the define is set, and I've replaced former calls to "send_keyboard_report(..)" with the new "send_keyboard_report_deferred(..)" function along the code path for handling keys through the matrix scan (this will not affect other calls, such as how macros are handled).

When the feature define is included, it is now passing all but 5 unit tests.

Adapt for tapped and subsequent shifted key events

Tapped key events by default work only on **key release** by sending a key press
event and a subsequent key release event to the host. Now with deferring
the send event after the complete matrix scan these events are merged
and erase each other becoming effectively a nop. The idea is to buffer the
unregister event of the key release for all tapped keys (normal keys are
not buffered) and send these in a separate keyboard report just after
the key press events have been send.

Unregister events/key codes are stored in the unregister_keycodes
struct by unregister_code_buffered() which is called for all tapped
key codes and send after a complete scan is completed by
send_keyboard_report_buffered_unregister_keys().

Subsequent shifted key codes like pressing KC_EQL and KC_PLUS (KC_LSHIFT +
KC_EQL) while holding the former need an additional key release event
of KC_EQL before sending the combination again. This release event isn't
defeered anymore but send as a report immediatly.

All failing unit-tests have been fixed or rather adapted for this
feature. As it turns out most of the failing key press cases expect one
report per key event so they are false negatives.

DEFER_KEYBOARD_REPORT_ENABLE was renamed to REGISTER_MULTIPLE_KEYEVENTS_ENABLE
because it rather states a implementation detail but doesn't communicate purpose
very well in my opinion.
k557osuman pushed a commit to k557osuman/qmk_firmware that referenced this issue Oct 23, 2024
This change defers calling "host_keyboard_send(keyboard_report)" until after the matrix loop.

This enables the ability to send multiple key changes in a single report, which also means that multiple key changes can be detected in a single polling cycle. This essentially allows us report more information in the same amount of time at no additional cost.

Before this change, only one key change (a press, or release) could be sent at a time. This is because "host_keyboard_send(..)" was created as a blocking method so that data cannot be changed mid-transmission to host, and this method was formerly called immediately after any change was made to the keyboard report. Calling this method at this point meant that the remainder of the polling interval would be used to send the report to host.

The impact of this can be very significant in gaming scenarios, when multiple keypresses need to be triggered simultaneously. See issue qmk#4904 for more details.

I tested the change manually on a Fox Leaf 60 and a Romac, and compared directly against the same firmware without the change. I've confirmed in manual testing that this allows multiple simultaneous keypresses, while not seeming to introduce any new issues.

Exact changes include:
- In "send_keyboard_report(..)" method, flag the last report's state as dirty instead of immediately sending report to host
- Create a new "send_keyboard_report_immediate(..)" method which is called after the matrix loop, which checks if the last report sent to host is dirty and sends an updated report if so, then sets it to clean

Gate deferred keyboard reports feature behind an optional define

Create a define to enable the experimental "deferred keyboard reports" feature (the feature is off by default).

- I manually tested a Fox Leaf 60 firmware with the define on and firmware with the define off.
- All unit tests pass when feature define is not specified.
- 9 unit tests fail when feature define is specified, keeping it in experimental status.

Refactor "defer send keyboard report" feature to support macros

This refactor makes the "defer send keyboard report" feature compatible with macros.

Rather than modify the functionality of all "send_keyboard_report(..)" calls - the method now sends the keyboard report immediately like before. A new function called "send_keyboard_report_deferred(..)" handles the deferred support if the define is set, and I've replaced former calls to "send_keyboard_report(..)" with the new "send_keyboard_report_deferred(..)" function along the code path for handling keys through the matrix scan (this will not affect other calls, such as how macros are handled).

When the feature define is included, it is now passing all but 5 unit tests.

Adapt for tapped and subsequent shifted key events

Tapped key events by default work only on **key release** by sending a key press
event and a subsequent key release event to the host. Now with deferring
the send event after the complete matrix scan these events are merged
and erase each other becoming effectively a nop. The idea is to buffer the
unregister event of the key release for all tapped keys (normal keys are
not buffered) and send these in a separate keyboard report just after
the key press events have been send.

Unregister events/key codes are stored in the unregister_keycodes
struct by unregister_code_buffered() which is called for all tapped
key codes and send after a complete scan is completed by
send_keyboard_report_buffered_unregister_keys().

Subsequent shifted key codes like pressing KC_EQL and KC_PLUS (KC_LSHIFT +
KC_EQL) while holding the former need an additional key release event
of KC_EQL before sending the combination again. This release event isn't
defeered anymore but send as a report immediatly.

All failing unit-tests have been fixed or rather adapted for this
feature. As it turns out most of the failing key press cases expect one
report per key event so they are false negatives.

DEFER_KEYBOARD_REPORT_ENABLE was renamed to REGISTER_MULTIPLE_KEYEVENTS_ENABLE
because it rather states a implementation detail but doesn't communicate purpose
very well in my opinion.
k557osuman pushed a commit to k557osuman/qmk_firmware that referenced this issue Oct 24, 2024
This change defers calling "host_keyboard_send(keyboard_report)" until after the matrix loop.

This enables the ability to send multiple key changes in a single report, which also means that multiple key changes can be detected in a single polling cycle. This essentially allows us report more information in the same amount of time at no additional cost.

Before this change, only one key change (a press, or release) could be sent at a time. This is because "host_keyboard_send(..)" was created as a blocking method so that data cannot be changed mid-transmission to host, and this method was formerly called immediately after any change was made to the keyboard report. Calling this method at this point meant that the remainder of the polling interval would be used to send the report to host.

The impact of this can be very significant in gaming scenarios, when multiple keypresses need to be triggered simultaneously. See issue qmk#4904 for more details.

I tested the change manually on a Fox Leaf 60 and a Romac, and compared directly against the same firmware without the change. I've confirmed in manual testing that this allows multiple simultaneous keypresses, while not seeming to introduce any new issues.

Exact changes include:
- In "send_keyboard_report(..)" method, flag the last report's state as dirty instead of immediately sending report to host
- Create a new "send_keyboard_report_immediate(..)" method which is called after the matrix loop, which checks if the last report sent to host is dirty and sends an updated report if so, then sets it to clean
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug help wanted question stale Issues or pull requests that have become inactive without resolution.
Projects
None yet
Development

No branches or pull requests

9 participants