New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Calculation of image difference percentage is incorrect #107

Closed

NathanielAvery opened this issue Sep 2, 2022 · 2 comments

Assignees

Labels

NathanielAvery commented Sep 2, 2022

Hi 👋🏻 Thanks for writing this plugin!

Describe the bug

When the plugin determines the percentage of changed pixels, its calculation is incorrect:

cypress-plugin-visual-regression-diff/src/plugins.ts

Line 164 in b4c6617

imgDiff = (diffPixels / width) * height;

As pixelmatch returns the number of different pixels, this line should be:

imgDiff = diffPixels / (width * height);

To Reproduce

Create an empty 100x100 <div> element
Capture a snapshot of that element in a Cypress test with a threshold of 0.5.
Modify the snapshot image so that just below 50% of the pixels are different.
Re-run the test.
Expected behavior: Test passes.
Actual behavior: Test fails.

The text was updated successfully, but these errors were encountered:

NathanielAvery added the bug label

FRSgit self-assigned this

Member

FRSgit commented Sep 2, 2022

Hi @NathanielAvery,

🤦 You're absolutely right. Thanks for the good catch!

FRSgit closed this as completed in

1e9bb6d

FRSgit added a commit that referenced this issue


fix: image diff calculation

529cb22

closes #107

Signed-off-by: Jakub Freisler <jakub@frsource.org>

Member

FRSgit commented Sep 2, 2022 •

edited

Loading

Fixed in 529cb22 (1.9.14)

github-actions bot pushed a commit that referenced this issue


chore(release): 1.9.14 [skip ci]

124d464

## [1.9.14](v1.9.13...v1.9.14) (2022-09-02)

### Bug Fixes

* image diff calculation ([529cb22](529cb22)), closes [#107](#107)

github-actions bot pushed a commit to braze-inc/cypress-plugin-visual-regression-diff that referenced this issue


chore(release): 1.0.0 [skip ci]

9f3104b

# 1.0.0 (2022-10-31)

### Bug Fixes

* btoa utf8 encoding/decoding error ([FRSOURCE#114](https://github.com/braze-inc/cypress-plugin-visual-regression-diff/issues/114)) ([0137014](0137014))
* create missing dirs when renaming screenshot files ([38e5ff5](38e5ff5))
* **deps:** pin dependency vue to 3.2.37 ([FRSOURCE#68](https://github.com/braze-inc/cypress-plugin-visual-regression-diff/issues/68)) ([d09a762](d09a762))
* **deps:** update dependency @frsource/base64 to v1.0.3 ([FRSOURCE#144](https://github.com/braze-inc/cypress-plugin-visual-regression-diff/issues/144)) ([09ecbd8](09ecbd8))
* **deps:** update dependency move-file to v3 ([FRSOURCE#62](https://github.com/braze-inc/cypress-plugin-visual-regression-diff/issues/62)) ([4f6eaf6](4f6eaf6))
* **deps:** update dependency pixelmatch to v5.3.0 ([FRSOURCE#55](https://github.com/braze-inc/cypress-plugin-visual-regression-diff/issues/55)) ([ca5d278](ca5d278))
* **deps:** update dependency sharp to v0.31.1 ([FRSOURCE#132](https://github.com/braze-inc/cypress-plugin-visual-regression-diff/issues/132)) ([15f0f5d](15f0f5d))
* **deps:** update dependency vue to v3.2.38 ([FRSOURCE#101](https://github.com/braze-inc/cypress-plugin-visual-regression-diff/issues/101)) ([e2d3c82](e2d3c82))
* **deps:** update dependency vue to v3.2.39 ([FRSOURCE#110](https://github.com/braze-inc/cypress-plugin-visual-regression-diff/issues/110)) ([8a7f055](8a7f055))
* **deps:** update dependency vue to v3.2.40 ([FRSOURCE#131](https://github.com/braze-inc/cypress-plugin-visual-regression-diff/issues/131)) ([537fd16](537fd16))
* image diff calculation ([529cb22](529cb22)), closes [FRSOURCE#107](https://github.com/braze-inc/cypress-plugin-visual-regression-diff/issues/107)
* proper readme info ([dd87e19](dd87e19))
* remove alias leftovers from dist bundles ([407ce79](407ce79))
* remove automated screenshots update ([acb3ef0](acb3ef0))
* reset name cache after tests run ([bfbf138](bfbf138))
* sanitize screenshot filenames ([fc57380](fc57380))
* security vulnerabilities ([d0bda44](d0bda44))
* security vulnerability ([d6f849c](d6f849c))
* text overflowing when image is small ([3b04f8e](3b04f8e))

### Features

* add forceDeviceFactor option ([8d69632](8d69632))
* add matchAgainstPath option ([FRSOURCE#146](https://github.com/braze-inc/cypress-plugin-visual-regression-diff/issues/146)) ([7a5e3a8](7a5e3a8)), closes [FRSOURCE#88](https://github.com/braze-inc/cypress-plugin-visual-regression-diff/issues/88)
* add possibility to change images dirname ([b831e94](b831e94))
* add queue flushing in after block ([70f828f](70f828f))
* add title option to matchImage command ([FRSOURCE#81](https://github.com/braze-inc/cypress-plugin-visual-regression-diff/issues/81)) ([4d03866](4d03866))
* add typings ([0a0e8e6](0a0e8e6))
* auto clean unused files ([FRSOURCE#124](https://github.com/braze-inc/cypress-plugin-visual-regression-diff/issues/124)) ([38679a7](38679a7)), closes [FRSOURCE#118](https://github.com/braze-inc/cypress-plugin-visual-regression-diff/issues/118)
* don't override screenshots if not needed ([9066017](9066017))
* externalize important APIs ([9f94086](9f94086))
* first implementation ([388cccf](388cccf))
* img diff when resolution differs ([FRSOURCE#108](https://github.com/braze-inc/cypress-plugin-visual-regression-diff/issues/108)) ([c8a5044](c8a5044)), closes [FRSOURCE#94](https://github.com/braze-inc/cypress-plugin-visual-regression-diff/issues/94)
* introduce imagesPath option ([FRSOURCE#152](https://github.com/braze-inc/cypress-plugin-visual-regression-diff/issues/152)) ([961e137](961e137)), closes [FRSOURCE#147](https://github.com/braze-inc/cypress-plugin-visual-regression-diff/issues/147)
* make library cypress 10 compatible ([b26beb3](b26beb3))
* make plugin Cypress 10 compatible ([a03a17d](a03a17d))
* migrate to @frsource/base64 package ([e4f3a14](e4f3a14))
* provide modern exports ([5c911a1](5c911a1))
* show comparison for successful tests ([FRSOURCE#137](https://github.com/braze-inc/cypress-plugin-visual-regression-diff/issues/137)) ([c09bab3](c09bab3)), closes [FRSOURCE#104](https://github.com/braze-inc/cypress-plugin-visual-regression-diff/issues/104)
* show scrollbar for overflowing images ([de994b9](de994b9))
* stop logging all of the tasks ([573e728](573e728))

### BREAKING CHANGES

* deprecate imagesDir option in favor of imagesPath - see docs for additional information
* To use autocleanup feature you need to update all of the screenshots, best do it by running your test suite with cypress env 'pluginVisualRegressionUpdateImages' set to true.
* matchImage returns object containing comparison details from now on (previously was returning subject element from a chain)
* different resolution doesn't fail test immediately - img diff is being done

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment