buildGoModule: module fod wants different hashes

[colin-arnott-xero]

Describe the bug

I have somehow managed to craft a go.{mod,sum} (and buildGoModule) that wants different hashes for x86_64-linux and x86_64-darwin, unsure about other platforms. currently the embedded archive works on x86_64-linux.

Steps To Reproduce

Steps to reproduce the behavior:
0. on x86_64-darwin

1. tar xf controller-flake.tar.gz

{
  outputs = {nixpkgs, ...}: {
    packages.${system}.default = nixpkgs.legacyPackages.${system}.buildGoModule {
      pname = "paas-k8s-controller";
      version = "latest";
      src = ./.;
      vendorHash = "";
      proxyVendor = true;
    };
  };
}

3. cd app.paas-k8s-controller/
4. nix build

Actual behavior

error: hash mismatch in fixed-output derivation '/nix/store/a01j8rnh59jh4bq0ncxfq8n205m97w9i-paas-k8s-controller-latest-go-modules.drv':
         specified: sha256-GUmClldvOyqEiTl8yp3ERFDJ8BP/mDkIABmpwdkVLlg=
            got:    sha256-ibK7FgXNMPZa+F/Um2lG9pjaxoOMaJ6H5/qsVtoLgqM=
error: 1 dependencies of derivation '/nix/store/8av1pg59ypjx95vqz48fjhcfhn3ph5dq-paas-k8s-controller-latest.drv' failed to build

Additional context

while we are using proxyVendor = true;, and this is not the default, I was of the impression that feature derived consistent hashes across platforms. also, while my x86_64-darwin system does use a case insensitive filesystem, I thought /nix/store did not, and I know that go modules are all lowercase, as they use !wo!rd!s to represent WoRdS internally.

Notify maintainers

@c00w @kalbasit @Mic92 @zowoq

Metadata

Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

[user@x86_64-darwin:~]$ nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-darwin"`
 - host os: `Darwin 22.1.0, macOS 10.16`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.11.0`
 - channels(user): `"darwin, home-manager"`
 - channels(root): `"nixpkgs"`
 - nixpkgs: `/nix/store/x070biyjfvlvkf7qpypmfspxzy9a3y3n-source`

[user@x86_64-linux:~]$ nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 5.15.0-1026-aws, Ubuntu, 20.04.5 LTS (Focal Fossa), nobuild`
 - multi-user?: `no`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.12.0`
 - channels(user): `"nixpkgs"`
 - nixpkgs: `/home/user/.nix-defexpr/channels/nixpkgs`

[zowoq]

Looks like you pinged the members of gnome instead of go?

[colin-arnott-xero]

sorry about that all; annoying that I cannot tag @nixos/golang

[zowoq]

I was of the impression that feature derived consistent hashes across platforms

No, inconsistent hashes was the primary reason proxyVendor was added.

while my x86_64-darwin system does use a case insensitive filesystem, I thought /nix/store did not

No, it is case insensitive as well.

I know that go modules are all lowercase ...

See #129730

[zowoq]

while we are using proxyVendor = true;, and this is not the default, I was of the impression that feature derived consistent hashes across platforms.

Does this mean your example doesn't have consistent hashes when using proxyVendor?

I'm getting consistent hashes on both darwin and linux with and without proxyVendor?

[colin-arnott-xero]

Does this mean your example doesn't have consistent hashes when using proxyVendor?

That is correct, and why I provided the tarball in the description. I have not been able to reproduce it otherwise, and hopefully that shell of a reproducer does the same on other people's machines.

[zowoq]

I've checked again, still consistent hashes.

diff --git a/flake.nix b/flake.nix
index 3790276..1d11b1d 100644
--- a/flake.nix
+++ b/flake.nix
@@ -4,7 +4,7 @@
       pname = "paas-k8s-controller";
       version = "latest";
       src = ./.;
-      vendorHash = "sha256-GUmClldvOyqEiTl8yp3ERFDJ8BP/mDkIABmpwdkVLlg=";
+      vendorHash = "";
       proxyVendor = true;
     };
   };

warning: Git tree '/Users/zowoq/Downloads/app.paas-k8s-controller' is dirty
warning: found empty hash, assuming 'sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA='
error: hash mismatch in fixed-output derivation '/nix/store/mn14a6r8ml09cz3jrcga1in5pmwi4kzv-paas-k8s-controller-latest-go-modules.drv':
         specified: sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=
            got:    sha256-ibK7FgXNMPZa+F/Um2lG9pjaxoOMaJ6H5/qsVtoLgqM=
error: 1 dependencies of derivation '/nix/store/br9sv4b8gy40jixrm8zrvqjg0y67znmk-paas-k8s-controller-latest.drv' failed to build

diff --git a/flake.nix b/flake.nix
index 3790276..abe2c24 100644
--- a/flake.nix
+++ b/flake.nix
@@ -1,10 +1,10 @@
 {
   outputs = {nixpkgs, ...}: {
-    packages.x86_64-darwin.default = nixpkgs.legacyPackages.x86_64-darwin.buildGoModule {
+    packages.x86_64-linux.default = nixpkgs.legacyPackages.x86_64-linux.buildGoModule {
       pname = "paas-k8s-controller";
       version = "latest";
       src = ./.;
-      vendorHash = "sha256-GUmClldvOyqEiTl8yp3ERFDJ8BP/mDkIABmpwdkVLlg=";
+      vendorHash = "";
       proxyVendor = true;
     };
   };

warning: Git tree '/Users/zowoq/Downloads/app.paas-k8s-controller' is dirty
warning: found empty hash, assuming 'sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA='
error: hash mismatch in fixed-output derivation '/nix/store/w66jg48gpa9bjvi2s4pnmpz9llrf9qcj-paas-k8s-controller-latest-go-modules.drv':
         specified: sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=
            got:    sha256-ibK7FgXNMPZa+F/Um2lG9pjaxoOMaJ6H5/qsVtoLgqM=
error: 1 dependencies of derivation '/nix/store/ynrc20lgx209a54sh9c4j7a43n014sx2-paas-k8s-controller-latest.drv' failed to build

[colin-arnott-xero]

is your second example on linux? warning: Git tree '/Users/zowoq/Downloads/app.paas-k8s-controller' is dirty looks like a macos path?

[zowoq]

is your second example on linux

remote builder

[colin-arnott-xero]

not sure what to say, but my linux ci system reliably reproduces the other hash: (see the description for system details, it is a different version of nix and single user)

error: hash mismatch in fixed-output derivation '/nix/store/x8h0n3h3brcxagd0yg4b7fh1afz0x7g4-paas-k8s-controller-latest-go-modules.drv':
            specified: sha256-ibK7FgXNMPZa+F/Um2lG9pjaxoOMaJ6H5/qsVtoLgqM=
               got:    sha256-GUmClldvOyqEiTl8yp3ERFDJ8BP/mDkIABmpwdkVLlg=

is there additional debugging information I can provide or do?

[qbit]

I recently noticed that using postBuild and preBuild (maybe others.. I didn't test) cause the vendor hash to change. Perhaps related?

Can you do a recursive diff of the two store entries? Maybe that will offer a clue?

[flokli]

#225973 looks related.