Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make WebDriver DPI scaling-aware. #37

Merged
merged 4 commits into from
May 8, 2024

Conversation

grokys
Copy link
Contributor

@grokys grokys commented May 7, 2024

This PR makes the WebDriver process DPI scaling aware. If this is not done then capturing screenshots will not work on monitors with scaling other than 100% and sometimes element positioning is miscalculated.

It also makes the test application DPI-aware as it needs to communicate the current scaling factor to the unit tests. The test application exposes its current DPI scaling in the status bar, and this is read in tests that require checking bounds/positions.

Note that WinAppDriver is also DPI aware so this should also help match the behavior there.

Copy link
Collaborator

@aristotelos aristotelos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, but I have a question whether this is needed for the web API.

src/FlaUI.WebDriver.UITests/ElementTests.cs Outdated Show resolved Hide resolved
src/FlaUI.WebDriver.UITests/WindowTests.cs Outdated Show resolved Hide resolved
src/FlaUI.WebDriver/app.manifest Show resolved Hide resolved
src/FlaUI.WebDriver/app.manifest Show resolved Hide resolved
@grokys
Copy link
Contributor Author

grokys commented May 8, 2024

Yep, even though it's a web API it interacts with the screen. And every process that interacts with the screen on Windows is subject to translation of screen coordinates if it's not per-monitor DPI aware. A couple of examples:

I've found some relevant Microsoft documentation for this:

Scaling in UI Automation Clients

The advice there is:

First, make the client application dpi-aware. To do this, call the SetProcessDPIAware function at startup. This function makes the entire process dpi-aware, meaning that all windows that belong to the process are unscaled.

The client application in this case being the WebDriver. This PR doesn't call SetProcessDPIAware though, it uses the alternative (and simpler IMO) method of Setting default awareness with the application manifest.

Footnotes

  1. The documentation for GetCursorPos is confusing here. It says "The cursor position is always specified in screen coordinates and is not affected by the mapping mode of the window that contains the cursor." but the documentation for scaling in UI automation contradicts this by saying "For example, typically, the GetCursorPos function returns the logical coordinates". Perhaps "mapping mode" is different from logical coordinates, not sure. One can easily confirm that it uses logical coordinates though by using the API from a process that is not DPI aware.

@aristotelos aristotelos merged commit 19553d2 into FlaUI:main May 8, 2024
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants