curl-install-sh: fatal: bad object from git clone steps

[avengerx]

Operating system and version:

Windows Server 2012 R2
NodeJS 20.15.1
Git for Windows 2.45.2.windows.1
Cygwin1.dll 3.4.5

nvm debug output:

# command not found?

nvm ls output:

# command... not found.

How did you install nvm?

curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.7/install.sh | bash

What steps did you perform?

Ran the install command

What happened?

install.sh script output

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 16555  100 16555    0     0   140k      0 --:--:-- --:--:-- --:--:--  141k
=> Downloading nvm from git to '/home/avenger/.nvm'
=> Cloning into '/home/avenger/.nvm'...
remote: Enumerating objects: 369, done.
remote: Counting objects: 100% (369/369), done.
remote: Compressing objects: 100% (315/315), done.
remote: Total 369 (delta 42), reused 166 (delta 27), pack-reused 0
Receiving objects: 100% (369/369), 368.22 KiB | 30.68 MiB/s, done.
Resolving deltas: 100% (42/42), done.
remote: Enumerating objects: 366, done.
remote: Counting objects: 100% (366/366), done.
remote: Compressing objects: 100% (315/315), done.
remote: Total 366 (delta 42), reused 162 (delta 25), pack-reused 0
Receiving objects: 100% (366/366), 365.90 KiB | 30.49 MiB/s, done.
Resolving deltas: 100% (42/42), done.
fatal: bad object cdc56687ada0c47f97632b0049b0446aadbe3a3a
error: remote did not send all necessary objects
Failed to fetch origin with v0.39.7. Please report this!

What did you expect to happen?

Install nvm.

Is there anything in any of your profile files that modifies the PATH?

~/.bash_profile

If you are having installation issues, or getting "N/A", what does curl -I --compressed -v https://nodejs.org/dist/ print out?

$ curl -I --compressed -v https://nodejs.org/dist/
* STATE: INIT => CONNECT handle 0xa0001eef8; line 1909 (connection #-5000)
* Added connection 0. The cache now contains 1 members
* STATE: CONNECT => RESOLVING handle 0xa0001eef8; line 1955 (connection #0)
* [CONN-0-0] setup, init filter chain
* [CONN-0-0] cf_add(filter=SOCKET)
* [CONN-0-0] cf_add(filter=SSL)
* [CONN-0-0][CF-SOCKET] setup(remotehost=nodejs.org)
* STATE: RESOLVING => CONNECTING handle 0xa0001eef8; line 2029 (connection #0)
* family0 == v4, family1 == v6
*   Trying 104.20.22.46:443...
* [CONN-0-0] connect(block=0)-> 0, done=0
* Connected to nodejs.org (104.20.22.46) port 443 (#0)
* ALPN: offers h2
* ALPN: offers http/1.1
*  CAfile: /etc/pki/tls/certs/ca-bundle.crt
*  CApath: none
* [CONN-0] Didn't find Session ID in cache for host HTTPS://nodejs.org:443
* [CONN-0-0][CF-SSL] TLSv1.3 (OUT), TLS handshake, Client hello (1):
* [CONN-0-0] connect(block=0)-> 0, done=0
* [CONN-0-0][CF-SSL] TLSv1.3 (IN), TLS handshake, Server hello (2):
* [CONN-0-0][CF-SSL] TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* [CONN-0-0][CF-SSL] TLSv1.3 (IN), TLS handshake, Certificate (11):
* [CONN-0-0][CF-SSL] TLSv1.3 (IN), TLS handshake, CERT verify (15):
* [CONN-0-0][CF-SSL] TLSv1.3 (IN), TLS handshake, Finished (20):
* [CONN-0-0][CF-SSL] TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* [CONN-0-0][CF-SSL] TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN: server accepted h2
* Server certificate:
*  subject: CN=*.nodejs.org
*  start date: Feb 28 00:00:00 2024 GMT
*  expire date: Mar 30 23:59:59 2025 GMT
*  subjectAltName: host "nodejs.org" matched cert's "nodejs.org"
*  issuer: C=GB; ST=Greater Manchester; L=Salford; O=Sectigo Limited; CN=Sectigo RSA Domain Validation Secure Server CA
*  SSL certificate verify ok.
* [CONN-0-0] connect(block=0)-> 0, done=1
* STATE: CONNECTING => PROTOCONNECT handle 0xa0001eef8; line 2073 (connection #0)
* [CONN-0-0] connect(block=0)-> 0, done=1
* STATE: PROTOCONNECT => DO handle 0xa0001eef8; line 2103 (connection #0)
* Using HTTP2, server supports multiplexing
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* h2h3 [:method: HEAD]
* h2h3 [:path: /dist/]
* h2h3 [:scheme: https]
* h2h3 [:authority: nodejs.org]
* h2h3 [user-agent: curl/7.87.0]
* h2h3 [accept: */*]
* h2h3 [accept-encoding: deflate, gzip, br, zstd]
* Using Stream ID: 1 (easy handle 0xa0001eef8)
> HEAD /dist/ HTTP/2
> Host: nodejs.org
> user-agent: curl/7.87.0
> accept: */*
> accept-encoding: deflate, gzip, br, zstd
>
* STATE: DO => DID handle 0xa0001eef8; line 2199 (connection #0)
* multi changed, check CONNECT_PEND queue
* STATE: DID => PERFORMING handle 0xa0001eef8; line 2318 (connection #0)
* Curl_readwrite(handle=0xa0001eef8) -> 0
* [CONN-0-0][CF-SSL] TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* [CONN-0] Didn't find Session ID in cache for host HTTPS://nodejs.org:443
* [CONN-0] Added Session ID to cache for HTTPS://nodejs.org:443 [server]
* [CONN-0-0][CF-SSL] TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* [CONN-0] Found Session ID in cache for host HTTPS://nodejs.org:443
* old SSL session ID is stale, removing
* [CONN-0] Added Session ID to cache for HTTPS://nodejs.org:443 [server]
* readwrite_data(handle=0xa0001eef8) -> 0
* Curl_readwrite(handle=0xa0001eef8) -> 0
* readwrite_data(handle=0xa0001eef8) -> 0
* Curl_readwrite(handle=0xa0001eef8) -> 0
* HTTP/2 found, allow multiplexing
< HTTP/2 200
HTTP/2 200
< date: Tue, 23 Jul 2024 22:31:52 GMT
date: Tue, 23 Jul 2024 22:31:52 GMT
< content-type: text/html
content-type: text/html
< cache-control: public, max-age=3600, s-maxage=14400
cache-control: public, max-age=3600, s-maxage=14400
< last-modified: Tue, 23 Jul 2024 19:35:45 GMT
last-modified: Tue, 23 Jul 2024 19:35:45 GMT
< cf-cache-status: HIT
cf-cache-status: HIT
< age: 10355
age: 10355
< vary: Accept-Encoding
vary: Accept-Encoding
< strict-transport-security: max-age=31536000; includeSubDomains; preload
strict-transport-security: max-age=31536000; includeSubDomains; preload
< x-content-type-options: nosniff
x-content-type-options: nosniff
< server: cloudflare
server: cloudflare
< cf-ray: 8a7f1f8aab5c5794-IAD
cf-ray: 8a7f1f8aab5c5794-IAD
< content-encoding: br
content-encoding: br

<
* readwrite_data(handle=0xa0001eef8) -> 0
* Curl_readwrite(handle=0xa0001eef8) -> 0
* STATE: PERFORMING => DONE handle 0xa0001eef8; line 2517 (connection #0)
* multi_done: status: 0 prem: 0 done: 0
* Connection #0 to host nodejs.org left intact
* Expire cleared (transfer 0xa0001eef8)

bottomline

If I download/checkout and edit install.sh and change all --git-dir="${INSTALL_DIR}/.git" --work-tree="${INSTALL_DIR}" with just -C "${INSTALL_DIR}" the script runs to completion.

`$ cat install.sh | bash`

=> nvm is already installed in /home/user/.nvm, trying to update using git
=> * (HEAD detached at FETCH_HEAD)
  master
=> Compressing and cleaning up git repository

=> Appending nvm source string to /home/user/.bashrc
=> Appending bash_completion source string to /home/user/.bashrc
=> You currently have modules installed globally with `npm`. These will no
=> longer be linked to the active version of Node when you install a new node
=> with `nvm`; and they may (depending on how you construct your `$PATH`)
=> override the binaries of modules installed with `nvm`:

C:\Users\user\AppData\Roaming\npm
└── nvm@0.0.4
=> If you wish to uninstall them at a later point (or re-install them under your
=> `nvm` Nodes), you can remove them from the system Node as follows:

     $ nvm use system
     $ npm uninstall -g a_module

=> Close and reopen your terminal to start using nvm or run the following to use it now:

export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"  # This loads nvm
[ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion"  # This loads nvm bash_completion

This happens because I'm running git for windows from a cygwin shell, so the linux-like path provided in --git-dir doesn't get translated correctly. It seems both -C and --work-tree are translated correctly. I didn't retry the script just dropping --git-dir as it would probably fail as my current working directory was not ~/.nvm.

So, perhaps replacing the command to use -C could be a bit more portable. Maybe checking "${OSTYPE}" == "cygwin" could create an exception, if using both work-tree and git-dir is meant to solve issues with a specific OS/environment.

Hope this helps improve NVM!

edit: this issue is probably the same reported in #3329.

[ljharb]

The nvm@0.0.4 output indicates you have an npm-installed nvm, which isn't actually a working version of anything.

The issue with the git syntax suggests your git version is unsupported - we require v1.7.10, per the readme. What version do you have?

[avengerx]

I had installed the nvm@0.0.4 where it informed me I needed to install it from the shell script. I understand it didn't do anything, I just didn't bother to remove it back then.

Git is 2.45.2 as I stated in the top (operating system/version).

I don't know, to me it really look like just a cygwin-to-git4win path conversion issue. Again, it installed when I switched to -C (which should be equivalent to the pair of git arguments). Thing is, I am calling git for windows with linux(cygwin) paths, thus the paths (in /home/user/.nvm format) doesn't work, as it would expect something like c:\cygwin64\home\user\.nvm, which cygwin can translate when -C is passed.

Unless I'm mistaken, cygwin automatically translates paths passed to non-cygwin commands. When it's in the format -C /home/user/.nvm. But it doesn't really try it when the arguments are tied --git-dir="/home/user/.nvm".

This issue should only trigger when using WSL or cygwin with non-native builds of git. The solution that works for me though, probably doesn't for WSL.

[ljharb]

What happens if you use a cygwin-installed git, so that it's not using the windows git?

[avengerx]

Using cygwin-installed git (yet, of course, Windows-installed node), it works. Tested in a somewhat similar station with git 2.31.1, node 14.17.6, and cygwin1.dll 3.2.0.

$ curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.7/install.sh | bash

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 16555  100 16555    0     0  22252      0 --:--:-- --:--:-- --:--:-- 22251
=> Downloading nvm from git to '/home/avenger/.nvm'
=> Cloning into '/home/avenger/.nvm'...
remote: Enumerating objects: 369, done.
remote: Counting objects: 100% (369/369), done.
remote: Compressing objects: 100% (315/315), done.
remote: Total 369 (delta 42), reused 166 (delta 27), pack-reused 0
Receiving objects: 100% (369/369), 368.22 KiB | 611.00 KiB/s, done.
Resolving deltas: 100% (42/42), done.
* (HEAD detached at FETCH_HEAD)
  master
=> Compressing and cleaning up git repository

=> Appending nvm source string to /home/avenger/.bashrc
=> Appending bash_completion source string to /home/avenger/.bashrc
=> You currently have modules installed globally with `npm`. These will no
=> longer be linked to the active version of Node when you install a new node
=> with `nvm`; and they may (depending on how you construct your `$PATH`)
=> override the binaries of modules installed with `nvm`:

C:\Users\avenger\AppData\Roaming\npm
+-- heroku@7.67.1
+-- json2yaml@1.1.0
+-- proggy@0.1.0
+-- retypeapp-win-x64@2.3.0-202205021640
+-- typescript@3.8.3
+-- windows-build-tools@5.2.2
`-- yarn@1.22.11
=> If you wish to uninstall them at a later point (or re-install them under your
=> `nvm` Nodes), you can remove them from the system Node as follows:

     $ nvm use system
     $ npm uninstall -g a_module

=> Close and reopen your terminal to start using nvm or run the following to use it now:

export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"  # This loads nvm
[ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion"  # This loads nvm bash_completion

[ljharb]

Windows-installed node shouldn't matter because nvm doesn't itself use node.

OK, thanks! That means that nvm's install script either needs to detect windows-installed git, ideally with feature detection, and either a) error out, or b) change the git commands so they work.

Alternatively, cygwin could (and probably should) add support for --git-dir if it already supports -C?

[avengerx]

Windows-installed node shouldn't matter because nvm doesn't itself use node.

Oh, sure, right!

OK, thanks! That means that nvm's install script either needs to detect windows-installed git, ideally with feature detection, and either a) error out, or b) change the git commands so they work.

Ya, erroring (a) could be frustrating to the user, but not bulldozing directories where they shouldn't will definitively help at least measure the worth of treating this edge case. Adapting the commands (b) should involve some magic I'm not sure is worth for the script maintainability over the long run.
It would involve, after resolving ${INSTALL_DIR} something like this

conceptual script

if [ "${OSTYPE}" == "cygwin" ]; then
  if ldd "$(which git)" | egrep -q "^[ "$'\t'"]*cygwin1.dll "; then
    INSTALL_DIR_CONVERTED="$(cygpath -w "$(readlink -f "${INSTALL_DIR}")")"
  fi
fi

Basically, if under cygwin -and- the git command in path does not link against cygwin1.dll, we can safely tell it's windows version. Then path conversion only for git calls should take place.

Git-windows ldd output

$ ldd "$(which git)"
        ntdll.dll => /cygdrive/c/Windows/SYSTEM32/ntdll.dll (0x7ff81c780000)
        KERNEL32.DLL => /cygdrive/c/Windows/system32/KERNEL32.DLL (0x7ff81b6d0000)
        KERNELBASE.dll => /cygdrive/c/Windows/system32/KERNELBASE.dll (0x7ff819cf0000)
        msvcrt.dll => /cygdrive/c/Windows/system32/msvcrt.dll (0x7ff81ba00000)
        SHELL32.dll => /cygdrive/c/Windows/system32/SHELL32.dll (0x7ff819ed0000)
        SHLWAPI.dll => /cygdrive/c/Windows/system32/SHLWAPI.dll (0x7ff819e70000)
        USER32.dll => /cygdrive/c/Windows/system32/USER32.dll (0x7ff81c220000)
        combase.dll => /cygdrive/c/Windows/SYSTEM32/combase.dll (0x7ff81bff0000)
        GDI32.dll => /cygdrive/c/Windows/system32/GDI32.dll (0x7ff81bea0000)
        RPCRT4.dll => /cygdrive/c/Windows/system32/RPCRT4.dll (0x7ff81b590000)
        SspiCli.dll => /cygdrive/c/Windows/system32/SspiCli.dll (0x7ff819980000)
        sechost.dll => /cygdrive/c/Windows/SYSTEM32/sechost.dll (0x7ff81bb60000)

Git-cygwin ldd output

$ ldd $(which git)
        ntdll.dll => /cygdrive/c/WINDOWS/SYSTEM32/ntdll.dll (0x7ffc42e30000)
        KERNEL32.DLL => /cygdrive/c/WINDOWS/System32/KERNEL32.DLL (0x7ffc42cd0000)
        KERNELBASE.dll => /cygdrive/c/WINDOWS/System32/KERNELBASE.dll (0x7ffc40760000)
        cygintl-8.dll => /usr/bin/cygintl-8.dll (0x3fd310000)
        cygz.dll => /usr/bin/cygz.dll (0x3fa050000)
        cygiconv-2.dll => /usr/bin/cygiconv-2.dll (0x3fcdb0000)
        cygwin1.dll => /usr/bin/cygwin1.dll (0x180040000)

So, not sure this edgy case is worth this much trouble.

Alternatively, cygwin could (and probably should) add support for --git-dir if it already supports -C?

This brings us to another long piece. In summary, I was mistaken. Cygwin doesn't do any path conversion and it worked by luck. The -C was required to get rid of the bad object error, though, which I'm still not completely sure why it happened, as all other git steps were "going through". Details follows.

Gory details

I was double-checking this... and seems cygwin doesn't really do any path translation.
The output of
[ -z "${XDG_CONFIG_HOME-}" ] && printf %s "${HOME}/.nvm" || printf %s "${XDG_CONFIG_HOME}/nvm"
is
/home/user/.nvm

And I tried some commands to see what happens. And turns out that git-for-windows version does interpret the path and creates the directories.

Say, I have /home/user/ as my home directory, which from cygwin would translate to C:\cygwin64\home\user\.

I noticed what happened here was, all stuff was installed and set up in C:\home\user\.nvm.
When I tried git init /home/user/.nvm (based on v0.39.7's install.sh), what I got was:

Initialized empty Git repository in C:/home/user/.nvm/.git/

Thus, I see there's no conversion made by cygwin at all, git-for-win just install everything outside of the actual user's home. Just, for some reason, --git-dir= and/or --work-tree= mostly works. It clones in that "foreign" directory, checks stuff out. All having the "build environment" actually outside actual user home.
But then that step to check out the tag fails. Even though it shouldn't by looking into C:\home\user\.nvm (which is not the same ascygwin's /home/user/.nvm)

Then I believe -C does work because it is equivalent to the syntax used for git clone, that just passes the install path (that means, "change to that directory and do the commands").

In the end, I had /home/user/.nvm (install.sh:152)[https://github.com/nvm-sh/nvm/blob/v0.39.7/install.sh#L152] but also had C:\home\user\.nvm (which corresponds to /cygdrive/c/home/user/.nvm in cygwin).

It indeed worked because, at least
(a) I manually cloned nvm-sh/nvm into real cygwin /home/user/.nvm and checked out tag
(b) The commands from the install script succeeded with -C /home/user/.nvm being interpreted as -C C:\home\user\.nvm

So I had very close branches in both directories, so whenever the script called git-win, it worked on the "foreign" repo, but whenever the rest of the script referred to the actual directory, it found what it expected. 😄

Then again, no telling what else was left not done in one directory but done in another, if you follow the thought. It seemed to have worked, but I didn't try too many things, I'm new with nvm.

bottomline

In the end, for my use case, I was setting up a server service, something I'd need to run as network service, so as nvm is something user-scoped it wouldn't really help and I will need to keep node up-to-date in the server the oldschool way.

So that's another reason it could be safer to just error out and see if more people raise here hitting this wall.

This issue might also be affected by git-bash-here (git for windows context menu item). I believe they use MSYS, and it seems this one does auto path conversion so...