[BUG] Inconsistent integrity checksums for Github repositories when switching between machines

[skress]

What / Why

I am running into problems installing packages from Github when those packages contain other Github packages in their package-lock.json files and when those repositories were uploaded from "different" computers.

My understanding of npm install is that it uses pacote to fetch the package and create an integrity checksum that is stored in package-lock.json. When a project commits it package-lock.json file and someone else checks out that project and runs "npm install" and a package is downloaded from Github, it should compute the same checksum when downloading the same revision. But that's not happening in my case, and as such "npm install" fails with "ERR! code EINTEGRITY ... integrity checksum failed ..."

How

Current Behavior

Running "npx pacote tarball" returns different checksums for the same Github repository on different machines.

Steps to Reproduce

git clone git@github.com:skress/test-pacote
cd test-pacote
npm i
npx pacote tarball "github:skress/test-pacote-data" xx.tar.gz

Running this on two Intel-based Macs (Node v15.6.0, npm 7.4.0, pacote 11.2.3) returns the same result.
Running this on an Apple Silicon-based Mac (same versions) returns a different result.
Running this in an Ubuntu Docker container (Node v10.19.0, npm 6.14.4, pacote 11.2.3) returns the same results as on the Apple Silicon machine.

When disabling gzip in dir.js all machines return the same checksum.

Expected Behavior

npm install / npx pacote tarball should create the same checksum regardless of the OS/architecture of the computer being used.

[isaacs]

If you are installing with npm 6, then yes, the tarballs will be different across different operating systems, because of the OS flag included in all gzip files by zlib. That's fixed in npm v7.

I'm getting the same results in OS X and Ubuntu using npm 7 and pacote 11. I also get the same result creating a tarball from a checkout of the same repo locally:

$ npx pacote tarball ./test-pacote-data yy.tar.gz
{
  integrity: 'sha512-PjIPQu6kTUJLRLaAqb9XRWjt32RMjUCBFe92IpOKRQBlK6ACqnI4mUFmFxC8NuZo67KXbX9MLTebE9xMKIqECQ==',
  resolved: '/home/isaacs/test-pacote/test-pacote-data',
  from: 'file:test-pacote-data'
}

$ npx pacote tarball "github:skress/test-pacote-data" xx.tar.gz
{
  integrity: 'sha512-PjIPQu6kTUJLRLaAqb9XRWjt32RMjUCBFe92IpOKRQBlK6ACqnI4mUFmFxC8NuZo67KXbX9MLTebE9xMKIqECQ==',
  resolved: 'git+ssh://git@github.com/skress/test-pacote-data.git#9b7d86d0ad05e737a54c5f887e732bbd3b1471d8',
  from: 'github:skress/test-pacote-data'
}

I don't have an apple silicon-based mac to test with, but I can try to track down someone who does.

[skress]

Thanks for looking at my problem ... so, I just retried it on all systems and I don't know what has happened since my bug report but it seems to be working without problems on all systems now. I had upgraded nodejs/npm to the latest versions already, but on macOS I had already been on npm v7 when reporting the problem (and you wrote that npm v7 fixed it). So that should not explain it. I have upgraded macOS to 11.2 and then 11.2.1 in the last couple of days. Don't know if that might explain anything.

Whatever problem it was, I can't reproduce it anymore. So sorry for the false report (although I am sure that it did not work at that time ...)

Thanks, again ... closing it now