Pixel-perfect screenshot tests on different environments #1646
SimonMarquis
started this conversation in
General
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
While having pixel-perfect screenshots seems a good idea, it makes tests fail almost certainly on environments that are not 100% exact to the CI env.
Could we lower our expectations and restore the default
changeThreshold
(0.01F
) inRoborazziOptions
?nowinandroid/core/screenshot-testing/src/main/kotlin/com/google/samples/apps/nowinandroid/core/testing/util/ScreenshotHelper.kt
Lines 42 to 43 in c4ebde1
For example, running on Windows with
jbr-17.0.7
leads to this output:As you can see, the
pixelDifferences
is generally very low, and we should probably allow some very slight variation, while still keeping the CI generated golden images as the single source of truth.PS: There might be a good reason for it, but I did not find any mention of why pixel-perfect screenshots were configured in the first place: https://github.com/android/nowinandroid/pull/876/files#diff-4d54482e44efb1225f70ed9435e0c5da894e62592a8058f607ef7d3e22ba9e6dR35 If it is for CI purposes only, maybe we could switch behaviors based on env variables.
Beta Was this translation helpful? Give feedback.
All reactions