nvm.sh: Add more xz platform support checks

[DeeDeeG]

macOS only supports extracting xz tarballs with tar in 10.9 and up.

GNU tar needs an xz executable on the PATH to extract xz tarballs.

Fixes #2155.

[DeeDeeG]

I force pushed to update this PR as follows:

• Correctly get the macOS version with sw_vers -productVersion, rather than plain sw_vers with no flag.
• Use nvm_version_greater() to check for a minimum macOS version.
• Do the more conservative check for non-macOS systems; Assume an xz executable is required on the PATH everywhere other than macOS, not just where we can detect tar is GNU tar.
  • This is because I haven't tested on BSD or Windows, etc. (Tested and accurate for macOS and Linux.)
  • Incidentally, should make the test I mentioned above pass.
  • Falling back to gzip if unsure should be a safe bet, as gzip is more-widely supported than xz.

[ljharb]

should the previous line also check nvm_has sw_vers?

Suggested change

	MACOS_VERSION="$(sw_vers -productVersion)"
	local MACOS_VERSION
	MACOS_VERSION="$(sw_vers -productVersion)"

(the "local" line is needed for ksh)

[DeeDeeG]

Semi-tangential point: it could be neat to test to some extent in a macOS Travis CI environment.

[ljharb]

If it's not too slow, we could certainly add a MacOS build to the test matrix.

[DeeDeeG]

Sorry for all the comments at once... Realized my check can still false positive on older macOS where we don't have liblzma.dylib, IF the user also has xz on the PATH such as from homebrew. Updated so the xz on PATH check only happens if we are not on macOS.

Had to redo the performance numbers, so:

For nvm_supports_xz 2.3.3:

• master: 51 to 52 milliseconds (one outlier run was 53 milliseconds)
• basic_macOS_check_update: 52 to 54 milliseconds

For nvm_supports_xz 10:

• master: 22 to 24 milliseconds (one outlier was 29 milliseconds)
• basic_macOS_check_update: 24 to 25 milliseconds (one outlier was 26, another was 37 milliseconds)

---

For fun, master with no xz on the PATH, which results in nvm just falling back on gzip:

• nvm_supports_xz 10: 16 to 17 milliseconds (one outlier was 20 milliseconds)
• nvm_supports_xz 2.3.3: 16 to 19 milliseconds (should be the same, maybe just background programs producing performance noise).

[DeeDeeG]

New snippet used for basic_macOS_check_update above (this time with no apparent false positive scenarios):

nvm_supports_xz() {
  if [ -z "${1-}" ]; then
    return 1
  fi

  if [ "$(command uname -s)" = 'Darwin' ]; then
    if ! [ -f "/usr/lib/liblzma.dylib" ]; then
      return 1
    fi
  elif ! command which xz >/dev/null 2>&1; then
    return 1
  fi

# [ . . . rest of nvm_supports_xz() as it appears in master . . . ]

(Also added to the previously linked gist, which again is here.)

[DeeDeeG]

I have been testing this on the various BSDs, and I only managed to compile Node on FreeBSD. (For context: There are no pre-compiled Node binaries for BSD.)

I don't think the OS detection for the other BSDs is done wrong, but until I can confirm Node is compileable on other BSDs, detecting them adds complexity to nvm for little benefit.

I'm on the fence about moving the "detect other BSDs" stuff out of this PR. I lean toward removing it. (If anyone is interested and can get Node compiling on other BSDs, then maybe they can pick this back up or at least help test.)

---

That matter having been brought up, this is ready for review again.

[DeeDeeG]

I removed the "detect other BSDs" stuff. This is now a simple update to nvm_supports_xz().

Pinging again for possible review?

[ljharb]

is there a reason not to fall back to the current functionality, which is to return 1 only when command which xz >/dev/null 2>&1? fails?

[DeeDeeG]

Practically, no. I'd be comfortable doing that fallback.

I did the thing again with the long answers. Click to expand if you want.

Maybe there is some OS out there that behaves different, but I have tested a lot of them, and xz on $PATH is a pretty reliable indicator for support in tar. We handle the big exception to the rule (namely macOS) in this PR.

(Taking this to the furthest extent, and being slightly more "YOLO" about it, you could just do the macOS >= 10.9 check on macOS, then fall back to the xz on $PATH check).

Deeper dive/technical under the hood stuff (click to expand if you want a deeper answer):

When this wouldn't work: A user could install an OS that doesn't have xz support for tar out of the box (such as OpenBSD, or really old OS X/Linux), and install a standalone xz binary on the $PATH somewhere. Then the code from this PR would over-eagerly ask tar to extract xz-compressed tarballs, which would probably fail.

Under the hood, I know that on macOS and BSD there is a shared library actually responsible for xz support in tar. (/usr/lib/liblzma.dylib on macOS, /usr/lib/liblzma.so on BSD). And on older Linux distros, tar could be too old to support xz compression.

As a coincidence, or by convention, those distros that support xz in tar also ship an xz binary on the $PATH. So it's practically a strong indicator for tar xz support. It's just that if an intepid user installs an xz binary it could fool the simplistic check we do.

I believe Linux and macOS must cover the vast majority of nvm users. FreeBSD covers most BSD users, if there are any BSD users out there at all using nvm. By reasoning correctly about these, "other users" are so few as to be mostly a theoretical consideration, unless they pop up in the issues of this repo saying something about it.

---

Philosophically: It's "conservative" to fall back to gzip just because falling back to gzip is expected to work everywhere. Being conservatively wrong and using gzip when xz could've been used isn't a big deal. On the flip side: being over-optimistically wrong and using xz when it doesn't work means the tarball doesn't extract and nvm errors out I guess. To be honest, the difference in size between the gzip and xz tarballs isn't huge... And gzip decompresses faster. It's not awful to fall back on gzip.

Leaning toward having all three platform checks then falling back to "xz on $PATH?".

(But we can delete the Linux check if you want, since it would be redundant to do the same check twice.)

[ljharb]

I'm always fine with having a more specific platform check that does a better thing on that platform, but it seems useful to ensure that any machine where it correctly used xz, continues to do so :-)

I think if someone has an xz binary that is lying, it's fine if they get an error.

[DeeDeeG]

Agree. 👍 Updating this PR when I get a moment.

More technical stuff to support that conclusion

I just want to clarify that on BSD/macOS, tar ignores the xz binary and directly links the shared library (liblzma) in.

On BSD and macOS it appears the convention is for the xz binary to rely on the shared library. If you delete the shared library or rename it, xz and tar -J break. If you delete or rename the xz binary, tar -J still works. So the xz binary is for end-users or for scripts, tar doesn't need it at all.

But yeah if that's the convention, being on a BSD-style OS and having a standalone xz binary that doesn't need liblzma.[so/dylib] seems unlikely in any case. Fair to expect it very likely won't happen.

(And of BSDs I tried, only OpenBSD lacks xz support in tar so only some fairly niche OSes actually should be detected as "doesn't support xz".)

If they are more like macOS, where the standalone extraction binary for xz isn't xz but gunzip, then we get a (maybe false) negative and fall back to gzip, which is like I said "not awful".

I have said most or all of this before, but if anyone wants to know without reading the backlog of comments in this PR, this is a pretty compact summary of things.

[DeeDeeG]

Updated again as discussed.

Reason for force push

Force pushed to overwrite a commit where I typoed the BSD check, omitting test brackets [ ... ] causing errors like _linux: command not found, _darwin: command not found or _freebsd: command not found.

I didn't want people doing git bisectand running into that random bug.