Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CrUX presentation needs to lead with a metric number and more clearly communicate differences from test run #1721

Closed
scottjehl opened this issue Feb 3, 2022 · 4 comments · Fixed by #2731
Assignees
Labels
Type: Bug An issue or pull request relating to a bug in WebPageTest

Comments

@scottjehl
Copy link
Contributor

When we show Crux Data, we need to offer better context as to how it compares to the synthetic run and why it's useful and interesting. Also, some sort of messaging when crux varies dramatically could be helpful to display:

"WebPageTest's findings in this test varied from Chrome real user data significantly. On average, Chrome users experience better/worse performance than this run shows. That doesn't mean WPT's findings aren't useful, just that the location/speed/etc conditions of this particular test is atypical for this site. We recommend testing in a variety of browser/location/network speeds to inform your understanding of this site's overall performance in different conditions"

@scottjehl scottjehl added the Type: Bug An issue or pull request relating to a bug in WebPageTest label Feb 3, 2022
@scottjehl
Copy link
Contributor Author

New design sketch of the CrUX data section of our summary that attempts to present CrUX info clearer and in context alongside the synthetic result.

image

Some notes:

prioritize showing the metrics themselves (at p75) since that's most interesting
second, make note of the difference between that metric and the test run
offer a link to see the full report
offer contextual info about why a wpt test run may differ greatly.

@scottjehl scottjehl changed the title Crux presentation needs to lead with a metric number, and charts need better explanation CrUX presentation needs to lead with a metric number and more clearly communicate differences from test run Feb 16, 2023
@tkadlec
Copy link
Contributor

tkadlec commented Feb 16, 2023

So a couple off-the-cuff reactions here....

  • The biggest value of this section is the correlation (oh crap, we're so much slower/faster than what the real-world looks like) and that part feels a bit "quieter" here
  • Styling the numbers like we do our test metrics kinda makes sense, but I also wonder if it's confusing? Like...are people gonna connect that to the actual test run rather than recognize it as another source?
  • I kinda miss the good/bad/improve percentages. No good rationale for why I do, honestly. :)

@scottjehl
Copy link
Contributor Author

Thanks @tkadlec

As we chatted in a side thread, there are probably two use cases for this section to balance: one for folks who are looking for the CrUX metrics first, and one for folks who want to know how much CrUX differs from the test run.

Here's another pass that shifts priorities in the blocking a bit

image

@scottjehl
Copy link
Contributor Author

Moving to implementation / PR step!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Type: Bug An issue or pull request relating to a bug in WebPageTest
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants