jest-snapshot: Highlight substring differences when matcher fails, part 3

[pedrottimark]

Summary

When expected and received are both string, snapshot matchers call printDiffOrStringify

If neither received nor expected (snapshot) is multiline, or either is empty:

• display following Snapshot: and Received: labels

If either received or expected (snapshot) is multiline, and neither is empty:

• if semantic cleanup leaves a common substring, highlight substring differences
• otherwise display line diff, however without enclosing double quote marks

Test plan

The change in report for single-line strings affects a few tests:

• updated 1 toMatch assertion in e2e/__tests__/toMatchSnapshot.test.ts
• updated 2 snapshots in e2e/__tests__/failures.test.ts
• updated 1 snapshot in watchModeUpdateSnapshot.test.ts

Added printDiffOrStringified.test.ts in jest-snapshot package

See pictures of reports from local tests in following comments

Example pictures baseline at left and improved at right

[pedrottimark]

Display single line strings following Snapshot: and Received: labels

Display multiple line strings with highlight for substring differences

[pedrottimark]

The --expand and default --no-expand options affect string diff:

[pedrottimark]

If semantic cleanup doesn’t leave a common substring, then display line diff

The baseline with enclosing double quote marks from pretty-format serialization:

The improved without enclosing double quote marks has a cleaner line diff:

[codecov-io]

Codecov Report

Merging #8569 into master will increase coverage by 0.02%.
The diff coverage is 62.5%.

@@            Coverage Diff             @@
##           master    #8569      +/-   ##
==========================================
+ Coverage    63.2%   63.22%   +0.02%     
==========================================
  Files         271      272       +1     
  Lines       11284    11304      +20     
  Branches     2749     2757       +8     
==========================================
+ Hits         7132     7147      +15     
- Misses       3538     3540       +2     
- Partials      614      617       +3

Impacted Files	Coverage Δ
packages/jest-snapshot/src/index.ts	28.77% <50%> (+0.4%)	⬆️
packages/jest-snapshot/src/print.ts	63.63% <63.63%> (ø)
packages/jest-snapshot/src/utils.ts	94.87% <0%> (+1.28%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fe14493...1da96c1. Read the comment docs.

[thymikee]

<3

[pedrottimark]

EDIT: After researching the history, I see that unescape is limited by design :)

[Uh oh, I need to fix a mistake for backslash in unescape helper function:

• which had been consistently incorrect because called for both expected and received
• but now is inconsistently incorrect because called only for expected if both are strings]

Also I will factor out one or more pure helper functions for report into utils.ts so it can have unit tests to increase coverage and prevent regression.

[pedrottimark]

unescape defined in #2852 removes backslash escape preceding double quote marks

The escape is added again for single line string because both are formatted by pretty-format

To be consistent because we know that the expected snapshot is a serialized string, let’s also remove backslash escape preceding backslash.

Incorrect at left and corrected at right for multi line string which does not have escape:

Incorrect at left and corrected at right for single line string which does have escape:

Regular expression still has consistent extra escaping but that is out of scope for this pull request:

[pedrottimark]

Although I forgot to make pictures of baseline here are improved when diff is not relevant:

[jeysal]

Awesome! Do we need some tests for the non-trivial cases here?

[pedrottimark]

Added another test and got a mistake-1-fix-2 special. When I added the test with snapshot of baseline report, and then corrected the code, and then ran test again without -u option, the diff report for baseline versus improved was another edge case to fix. TDD for the win, in this case :)

Here is baseline and improved for the first fix:

[jeysal]

In the empty/multiline case, maybe start the multiline string on the next line so that the indentation is less off (still off by one because of the ")?

[pedrottimark]

Super question, as usual. Extra credit for thinking about alignment.

Here are 2 alternatives for response or maybe they provoke some other solution:

I though about omitting double quote marks, but content of string might be ambiguous with serialization of other data types.

I’m thinking backticks for multiline string property in future data-driven diff at right:

[jeysal]

Second one is probably easier on the eyes if the string itself has lines indented by 2. Wondering if inserting indentation into people's strings is a good idea though - is there a legitimate use case for copying it out of the terminal?

[pedrottimark]

Yes, inserting indentation is a concern. Although Jest indents the reports in console.

Of alternatives above, I agree with you to prefer at the right.

Here is the minimal solution at the left and another indented alternative at the right:

Unless there is a clear winner, I think to leave it alone (as it has been) for this pull request.

[jeysal]

Nice! Does some things more and some things less than https://github.com/ikatyang/jest-snapshot-serializer-raw - maybe we should think about making a full-featured serializer for edge case strings 😄

[pedrottimark]

Thank you for the link! It answers questions in back of my mind about prettier snapshots.

So I am exploring if this pull request can also highlight substrings when:

• received is string
• expected is serialized by application-specific plugin instead of default stringify

[pedrottimark]

@ikatyang your thought about file-level snapshots sounds super for prettier tests. For example:

tests/typescript_conditional_types contains:

• conditional-types.ts
• infer-type.ts

tests/typescript_conditional_types/__snapshots__/jsfmt.spec.js.snap contains:

exports[`conditonal-types.ts 1`] = `…`
exports[`infer-type.ts 1`] = `…`

Here is alternative mapping to file-level “snapshot” per test file:

• tests/typescript_conditional_types/__snapshots__/conditional-type.ts.snap
• tests/typescript_conditional_types/__snapshots__/infer-type.ts.snap

Although my understanding of stateful code in jest-snapshot is too little to know how to hack it :)

[SimenB]

There is https://www.npmjs.com/package/jest-file-snapshot

[pedrottimark]

Simen and Michał an example of your forward thinking was request in part 2 to move getStringDiff into jest-diff therefore future version of jest-file-snapshot can highlight substrings in its report.

[pedrottimark]

Here is an example adapted from ESLint tests which include backtick quote marks to show that regardless of escaping in a snapshot file, the report displays without any escape.

A follow-up pull request unrelated to substring highlight will compare unescape function for double quote marks in snapshots with stringify in reports for other matchers.

[pedrottimark]

@SimenB It looks like the CI failure is bogus. Is it okay for me to merge so next minor has all 3 parts?

[jeysal]

None of the topics I brought up need to be addressed right here, right now, so go for it ;)

[duailibe]

There is https://www.npmjs.com/package/jest-file-snapshot

@SimenB I didn't know that package, I've built almost the same thing for Prettier but discarded after I've realised there's no way to remove obsolete snapshots, at least not in the worst case scenario when the matcher isn't called at all and there's no chance to detect obsolete snapshots.

I got excited for a second thinking that maybe that lib fixed it somehow, but unfortunately it didn't.

[pedrottimark]

@duailibe If plain text snapshot file is important in Prettier, can run_spec.js do the following:

• For each test file, assert toMatchFile for path in __file_snapshots__ subdirectory and the corresponding test file name with whatever extension makes sense to y’all
• If expect.getState().snapshotState._updateSnapshot === 'all' then it (not the matcher) deletes any files in the __file_snapshots__ subdirectory that do not correspond to a test file

For each subdirectory of tests to clean up its own file snapshots is more limited than global cleanup with jest -u but seems good enough even if y’all delete or rename entire subdirectory.

To close a gap in simulated integration, it is even possible for run_spec.js to cause Jest to report a snapshot file without corresponding test file as either obsolete or removed for -u option:

expect.getState()._uncheckedKeys.add(fileName + ' 1'); // must end with a number

[github-actions]

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.
Please note this issue tracker is not a help forum. We recommend using StackOverflow or our discord channel for questions.