[Bug]: 'npm audit signatures' fails for 1.42.1 packages

[jattasNI]

Version

1.42.1

Steps to reproduce

1. npm install -D playwright
2. npm audit signatures

Expected behavior

No error messages

Actual behavior

2 packages have invalid attestations:

playwright@1.42.1 (https://registry.npmjs.org/)
playwright-core@1.42.1 (https://registry.npmjs.org/)

Someone might have tampered with these packages since they were published on the registry!

Additional context

npm audit signatures is used to ensure the integrity of packages you download from the public npm registry. Here's the documentation on npm

The command succeeds with no errors or warnings for versions prior to 1.42.1.

Environment

System:
    OS: macOS 14.3
    Memory: 41.00 MB / 32.00 GB
  Binaries:
    Node: 20.10.0 - /usr/local/bin/node
    Yarn: 3.5.0 - /usr/local/bin/yarn
    npm: 10.2.3 - /usr/local/bin/npm
    pnpm: 8.6.9 - /usr/local/bin/pnpm
  Languages:
    Bash: 3.2.57 - /bin/bash
  npmPackages:
    playwright: ^1.42.1 => 1.42.1

[mxschmitt]

It looks like an upstream NPM bug, I was able to reproduce on NPM 10.2.4 and not on 10.5.0. Could you try updating NPM?

npm i -g npm@latest

[jattasNI]

@mxschmitt, I see the same behavior as you where it reproduces on npm 10.2.4 but not 10.5.0.

There isn't a Node installer available yet containing npm 10.5 so it'll be a minor inconvenience to use it but at least we have a feasible workaround.

Would you recommend filing an issue to npm? If so do you have any additional context about how playwright is signing packages that might be influencing this behavior?

[mxschmitt]

Further bisect:

10.4.0 bad
10.5.0 good

Range: npm/cli@v10.4.0...v10.5.0

npm/cli@d6521ac or npm/cli@dafa903 could be related.

[dgozman]

Would you recommend filing an issue to npm?

Sure, if you feel like it!

If so do you have any additional context about how playwright is signing packages that might be influencing this behavior?

Not really. We just run npm publish: https://github.com/microsoft/playwright/blob/main/utils/publish_all_packages.sh#L97.

[jattasNI]

I started filing an issue to npm but their issue template requires you to confirm that the bug reproduces in the latest version. Since this doesn't, I suspect they would reject or deprioritize the issue so I didn't end up filing it.

I'm happy to revisit that if this continues to be an issue in future Playwright releases or starts reproducing with the latest npm.

[dgozman]

I started filing an issue to npm but their issue template requires you to confirm that the bug reproduces in the latest version. Since this doesn't, I suspect they would reject or deprioritize the issue so I didn't end up filing it.

This makes sense 😄

I guess I'll close this issue for now, since there is no action item for Playwright as of today. We'll keep an eye on this, hopefully the npm fix will be widely available before our next release. Thank you for your help! Feel free to open a new issue if you run into this problem again.