Feature request: ability to define masks that allow for varying thresholds

[GrayedFox]

Background

Original idea found in comment here.

Always best to avoid phrasing things as an X/Y problem so going to expose the use case: we have sections of the PDFs we generate that contain signatures and in our test suite it's possible for this (and certain other areas like initials) to occasionally be a few pixels out based on the device and OS the signature or initials were drawn on.

Since our core product is producing legally compliant PDFs, we still want diffs to fail if even something as simple as a comma is missing, given that a comma can change the meaning of a sentence and we need to ensure our legal copy matches what we get from the lawyers exactly.

This feature would meet our need to maintain a 100% pixel match for the vast majority of the content on the PDF.

User Story

As a test engineer,
I would like to be able to specify masks that allow for varied thresholds,
so that I can reduce thresholds for specific sections of a PDF/PNG instead of the entire document.

Acceptance Criteria

• test writers can specify a threshold-mask (or some other name) which marks a designated area as having a seperate threshold to that of the base one
• when comparing the snapshots areas inside of a threshold-mask should use the specified threshold and not the global threshold

Nice to Haves

If a snapshot fails based on a pixel difference from inside of a threshold-mask it wouldbe nice to indicate this in some way.

Some ideas:

• in the resulting diff based on the threshold-mask properties
• in the resulting console output
• programatically so that the test can act on it by adding a new signature for the comparison method that allows specifying a callback function so the returned result of the diff can either be a boolean OR a callback method that exposes more info through via an object passed into the callback function

Are you willing to work on this yourself?

Yes, ties in very well with our business needs, I'd need to go through the correct channels at work but would be happy to give this a go.

[moshensky]

All contributions are welcomed!

A good start would be to look at compare-images.ts. I've linked to the diff line. Right before it you have the expected and current images.

The biggest challenge would be to come up with a nice, simple and expandable API.

[GrayedFox]

Great! I have made a ticket in our project management tool and will be pushing it, most likely won't be until Q1 next year, but I'll fork your repo and create a PR pointing back in and will make sure the API suits whatever standards and patterns you'd like us to adhere to for your comparison library.

I did my homework and IMO this is the best in class option for PDF comparisons based on what's out here, so happy to contribute when and where I can 😁