[Bug]: tabbing leads to undefined behavior

[nstepien]

Version

1.58.2

Steps to reproduce

1. clone my repo: https://github.com/nstepien/playwright-bugs
2. npm ci
3. npm test tabbing

(the rightclick.spec.ts test file is for a separate issue, unrelated to this one)

Expected behavior

Tests should pass, tabbing and focus states should behave similarly across browsers.
Focus/tabbing behavior should not change if elements are in an iframe.
There should be no DOM elements focused when the page/iframe isn't focused.
There should be no browser UI elements focused when the page/iframe is focused.

Actual behavior

• tabbing test:
  • Chromium:
    When focus is on the last element and pressing Tab,
    focus moves to a browser UI element,
    pressing Tab one more time moves focus to the first focusable element...
    but the browser UI element that was focused... is still focused! Double focus! 🐛
    I tried manually typing with headless: false but nothing happens,
    tabbing moved focus to another browser UI element instead of moving focus within the page. 🐛
  • Firefox:
    When focus is on the last element and pressing Tab,
    it seems like focus is set on a browser UI element,
    but document.hasFocus() is true, and Input C is the activeElement. 🐛
    Pressing Tab further does not cycle focus to Input A. 🐛
    Visually speaking it does look like focus is then removed from the browser UI element.
    I tried manually typing with headless: false but nothing happens,
    it's unclear where the focus actually is, despite what the DOM APIs report. 🐛
  • WebKit:
    No issues with WebKit.
    Although it's unclear if a browser UI element receives focus,
    focus does get correctly removed from Input C,
    and pressing Tab once more cycles focus to Input A as expected.
• shift tabbing test:
  • Chromium and WebKit: essentially same as above
  • Firefox: once focus leaves the page, focus shifts between two browser UI elements, despite what the DOM APIs report.
• tabbing in iframe test:
  • Chromium:
    Similar to above, but an additional Tab press is required for Input A to receive focus.
    A browser UI element remains focused though. 🐛
  • Firefox:
    Unlike the tests above, Tab does remove focus from Input C,
    but the parent document and the iframe have focus. 🐛
    After pressing Tab once more,
    Input C becomes focused again instead of Input A,
    but the iframe's document is not focused. Ghost focus? 🐛👻
    After pressing Tab one more time,
    the iframe's document is now focused.
  • WebKit:
    Once focus leaves Input C, pressing Tab does not move focus back into the page. 🐛
• shift tabbing in iframe test:
  • Chromium and WebKit: essentially same as the tabbing in iframe test.
  • Firefox:
    Unlike the tabbing in iframe test,
    shift-tabbing seems to cycle focus between Input A and a browser UI element,
    without ever restoring focus to the iframe's document. 🐛
• clicking to restore focus in iframe after tabbing test:
  • Chromium:
    While clicking does restore focus to the iframe's document/input,
    the browser UI element that was focused by tabbing, remains focused. 🐛
  • Firefox:
    When headless is disabled, clicking on the input immediately blurs, requiring another click to restore focus. 🐛
    When headless is enabled, it seems to behave like Chromium and WebKit, at least in terms of what the DOM APIs report, at least at the end.
  • WebKit:
    No issues.

Additional context

Focus states are highly unpredictable in Playwright, with results diverging between browsers, with/out iframe, and with/out headless, leading to flaky tests and hours spent debugging tests, especially as it affects both keyboard and mouse interactions.
Focus management is already hard as it is with DOM APIs, so this is another beast to tame.

I've tested with an iframe as well, as I'm using Playwright through Vitest's browser mode, and Vitest runs tests in an iframe.

My understanding of what's causing these bugs: the page.keyboard.* APIs simulate a virtual keyboard within the page's (or document's) context, but if focus moves outside the page then it should be impossible to interact with the page via keyboard interactions until focus moves back to the page again.
Instead, Playwright does try to simulate these keyboard events within the page even if the page isn't focused, and browsers get confused, which results in undefined behaviors, and "dual focus" where both page elements and UI elements are visually focused.
Maybe we need alternative APIs like browser.keyboard.* or window.keyboard.*?
Should the page.keyboard.* APIs should throw when the page isn't focused?
Should Playwright trap focus within the page/document?

It's unclear how much of this is an issue with browsers vs Playwright.
Manually tabbing instead of programmatically tabbing works fine in real browsers.
Although manually tabbing in:

• Playwright's Firefox browser + tabbing.html -> tabbing works, but stops moving focus once it cycles to Input A. I can shift+tab, but then it'll get stuck on Input C.
• Playwright's WebKit browser + tabbing-iframe.html -> can't manually tab back into the page.

Environment

System:
    OS: Windows 11 10.0.26200
    CPU: (32) x64 AMD Ryzen 9 9950X3D 16-Core Processor
    Memory: 42.69 GB / 63.58 GB
  Binaries:
    Node: 25.6.1 - C:\Program Files\nodejs\node.EXE
    npm: 11.7.0 - C:\Program Files\nodejs\npm.CMD
  IDEs:
    VSCode: 1.109.3 - C:\Users\Stepi\AppData\Local\Programs\Microsoft VS Code\bin\code.CMD
  npmPackages:
    @playwright/test: ^1.58.2 => 1.58.2

[nstepien]

I should add, when debugging this in Chrome, you should disable "emulate a focused page" in the devtools so you don't waste time figuring out why document.hasFocus() always returns true 😉

[Skn0tt]

This looks like many bug reports in one, and it's very hard for me to understand this. Could you point to one specific scenario you believe is broken, so we can start with that and then see?

[nstepien]

This looks like many bug reports in one, and it's very hard for me to understand this.

It's all related to how Playwright handles interactions when the page isn't focused.
I understand that it is complicated, I had to cover all my bases so we don't miss any edge cases, which resulted in many differing behaviors based on different scenarios.

Could you point to one specific scenario you believe is broken, so we can start with that and then see?

https://github.com/nstepien/playwright-bugs/blob/main/tests/tabbing.spec.ts#L64
You could start with the tabbing test, the shift tabbing test is essentially identical, just with tab-shifting.

[Skn0tt]

I see, so specifically the bug seems to be that tabbing outside the document has different behaviour among browsers. Here's a minimal repro:

test('tab outside the document', async function({ page }) {
  await page.setContent(`<input value="foo">`);
  await page.keyboard.press('Tab');
  expect.soft(await page.evaluate(() => (document.activeElement as HTMLInputElement).value)).toBe('foo');
  expect.soft(await page.evaluate(() => document.hasFocus())).toBe(true);
  await page.keyboard.press('Tab');
  expect.soft(await page.evaluate(() => (document.activeElement as HTMLInputElement).value)).toBe(undefined);
  expect.soft(await page.evaluate(() => document.hasFocus())).toBe(false);
});

It fails on Firefox, and on Chromium Headed, everywhere else it passes.

@nstepien can you elaborate on how what you're reporting is impacting you? What kind of application are you testing? What kinds of tests are it where you're tabbing outside the document?

[nstepien]

can you elaborate on how what you're reporting is impacting you? What kind of application are you testing? What kinds of tests are it where you're tabbing outside the document?

This primarily affects low-level UI libraries, where we make sure tabIndex order is respected, inert attributes are respected, usage of focus traps work as expected, ...
In some cases we test that input formatting gets reset on blur events, and these tests simply tab() out of the input, which leads to the page losing focus, which may impact following assertions/tests (in Vitest at least).
In other cases we make sure cycling focus back to the page focuses the correct "first" element, or "last" element when shift-tabbing, so the difference in behavior between browsers, especially when focusable elements are in iframes, means we have to tab() in a loop until the document re-gains focus.