Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

--retry=2 gives wrong error: toMatchInlineSnapshot cannot be called multiple times at the same location #6589

Closed
6 tasks done
lydell opened this issue Sep 28, 2024 · 1 comment · Fixed by #6817
Closed
6 tasks done
Labels
p3-minor-bug An edge case that only affects very specific usage (priority)

Comments

@lydell
Copy link

lydell commented Sep 28, 2024

Describe the bug

When you use the --retry=n flag (where n > 0), inline snapshots that succeeded on the first try fail on the retries with toMatchInlineSnapshot cannot be called multiple times at the same location. This means that if you precede a flaky assertion with an inline snapshot, the --retry flag does not work.

It sounds like the some state that toMatchInlineSnapshot carries needs to be reset between retries.

I later learned that --retry=1 is enough – it’s the number of retries not the number of total tries.

Reproduction

https://github.com/lydell/vitest-inline-snapshot-retry

import {expect, test} from "vitest";

test("demo", () => {
  // First an up-to-date snapshot.
  // Unfortunately, this fails on the second retry.
  expect("a").toMatchInlineSnapshot(`"a"`);

  // Then an assertion of a flaky function.
  expect(flakyFunction()).toBe(2);
});

// Imagine this being a flaky function that only sometimes gives the correct value.
// In this demo, we make it always succeed the second time. Which means that if you
// run with `--retry=2` the test should pass. However, since the snapshot fails on
// the second attempt, the test never passes. Comment out the snapshot test above
// to see it succeed.
let flakyCounter = 1;
function flakyFunction() {
  return flakyCounter++;
}

System Info

System:
    OS: macOS 14.7
    CPU: (10) arm64 Apple M1 Pro
    Memory: 3.02 GB / 32.00 GB
    Shell: 3.7.1 - /opt/homebrew/bin/fish
  Binaries:
    Node: 22.8.0 - ~/.local/share/nvm/v22.8.0/bin/node
    Yarn: 1.22.22 - /opt/homebrew/bin/yarn
    npm: 10.8.2 - ~/.local/share/nvm/v22.8.0/bin/npm
    pnpm: 9.9.0 - /opt/homebrew/bin/pnpm
    bun: 1.1.27 - /opt/homebrew/bin/bun
  Browsers:
    Chrome: 129.0.6668.71
    Safari: 18.0
  npmPackages:
    vitest: 2.1.1 => 2.1.1

Used Package Manager

npm

Validations

@hi-ogawa
Copy link
Contributor

hi-ogawa commented Sep 29, 2024

toMatshSnapshot is known to have an issue with retry #5312, but the recent validation made retry not work with toMatshInlineSnapshot as well (though technically there was an issue with retry and toMatshInlineSnapshot in some cases).

Both could be fixed by the same approach mentioned in #5312 (comment)

@hi-ogawa hi-ogawa added p3-minor-bug An edge case that only affects very specific usage (priority) and removed pending triage labels Sep 29, 2024
@github-actions github-actions bot locked and limited conversation to collaborators Dec 20, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
p3-minor-bug An edge case that only affects very specific usage (priority)
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants