treewide: use NIX_SSL_CERT_FILE instead of pkgs.cacert #1174

zowoq · 2024-11-16T14:06:37Z

Based on the discussion in #675 this seems to be the preferred style.

Enzime

I think we should instead not set NIX_SSL_CERT_FILE at all as Nix 2.0 (possibly even earlier) has used /etc/ssl/certs/ca-certificates.crt as one of the default locations which we already populate

zowoq · 2024-11-16T14:37:15Z

I don't understand?

Enzime · 2024-11-16T14:41:19Z

We should remove this line:

nix-darwin/modules/security/pki/default.nix

Line 87 in f86f158

    
           environment.variables.NIX_SSL_CERT_FILE = mkDefault "/etc/ssl/certs/ca-certificates.crt";

As well as removing all usage of NIX_SSL_CERT_FILE

We could even add an assertion to prevent users from setting that environment variable and tell them to use nix.settings.ssl-cert-file instead (available since 2.15.0)

zowoq · 2024-11-16T22:48:14Z

How will problems like #657 (comment) be handled?

Enzime · 2024-11-17T01:21:35Z

I had a look at that issue and it's not really clear to me why setting NIX_SSL_CERT_FILE is necessary

$ env | grep NIX_SSL_CERT_FILE
NIX_SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt
$ unset NIX_SSL_CERT_FILE
$ env | grep NIX_SSL_CERT_FILE
$ which git
/etc/profiles/per-user/enzime/bin/git
$ git clone https://github.com/LnL7/nix-darwin
Cloning into 'nix-darwin'...
remote: Enumerating objects: 10857, done.
remote: Counting objects: 100% (2497/2497), done.
remote: Compressing objects: 100% (813/813), done.
remote: Total 10857 (delta 1927), reused 2014 (delta 1659), pack-reused 8360 (from 1)
Receiving objects: 100% (10857/10857), 5.70 MiB | 2.88 MiB/s, done.
Resolving deltas: 100% (6849/6849), done.

{
  launchd.daemons.my-test-daemon = {
    script = ''
      set -x

      env | grep NIX_SSL_CERT_FILE || true
      rm -rf /tmp/my-test-daemon
      ${lib.getExe pkgs.git} clone -v https://github.com/LnL7/nix-darwin /tmp/my-test-daemon
    '';
    serviceConfig = {
      RunAtLoad = true;
      StandardOutPath = "/var/log/my-test-daemon.log";
      StandardErrorPath = "/var/log/my-test-daemon.log";
    };
  };
}

$ tail -F /var/log/my-test-daemon.log
+ env
+ grep NIX_SSL_CERT_FILE
+ true
+ rm -rf /tmp/my-test-daemon
+ /nix/store/cyd9di9q6vsdnffn63m75nsnpf425v90-git-2.47.0/bin/git clone -v https://github.com/LnL7/nix-darwin /tmp/my-test-daemon
Cloning into '/tmp/my-test-daemon'...
POST git-upload-pack (175 bytes)
POST git-upload-pack (317 bytes)
$ ls -lah /tmp/my-test-daemon
total 96
drwxr-xr-x  17 root  wheel   544B 17 Nov 04:56 .
drwxrwxrwt  10 root  wheel   320B 17 Nov 04:56 ..
drwxr-xr-x  13 root  wheel   416B 17 Nov 04:56 .git
drwxr-xr-x   4 root  wheel   128B 17 Nov 04:56 .github
-rw-r--r--   1 root  wheel    14B 17 Nov 04:56 .gitignore
-rw-r--r--   1 root  wheel   7.0K 17 Nov 04:56 CHANGELOG
-rw-r--r--   1 root  wheel   1.0K 17 Nov 04:56 LICENSE
-rw-r--r--   1 root  wheel   6.7K 17 Nov 04:56 README.md
-rw-r--r--   1 root  wheel   937B 17 Nov 04:56 default.nix
drwxr-xr-x   4 root  wheel   128B 17 Nov 04:56 doc
-rw-r--r--   1 root  wheel   640B 17 Nov 04:56 eval-config.nix
-rw-r--r--   1 root  wheel   508B 17 Nov 04:56 flake.lock
-rw-r--r--   1 root  wheel   2.8K 17 Nov 04:56 flake.nix
drwxr-xr-x  22 root  wheel   704B 17 Nov 04:56 modules
drwxr-xr-x   5 root  wheel   160B 17 Nov 04:56 pkgs
-rw-r--r--   1 root  wheel   5.6K 17 Nov 04:56 release.nix
drwxr-xr-x  57 root  wheel   1.8K 17 Nov 04:56 tests

zowoq · 2024-11-17T01:36:50Z

Interesting, that doesn't work for me:

customer@darwin01 ~ % tail -F /var/log/my-test-daemon.log
+ env
+ grep NIX_SSL_CERT_FILE
+ true
+ rm -rf /tmp/my-test-daemon
+ /nix/store/cyd9di9q6vsdnffn63m75nsnpf425v90-git-2.47.0/bin/git clone -v https://github.com/LnL7/nix-darwin /tmp/my-test-daemon
Cloning into '/tmp/my-test-daemon'...
fatal: unable to access 'https://github.com/LnL7/nix-darwin/': OpenSSL/3.3.2: error:80000002:system library::No such file or directory

Enzime · 2024-11-17T01:44:44Z

What version of macOS are you using?

In a regular terminal:

$ echo $NIX_SSL_CERT_FILE
/etc/ssl/certs/ca-certificates.crt
$ ls -lah $NIX_SSL_CERT_FILE
lrwxr-xr-x  1 root  wheel    41B 13 Aug  2023 /etc/ssl/certs/ca-certificates.crt@ -> /etc/static/ssl/certs/ca-certificates.crt
$ realpath $NIX_SSL_CERT_FILE
/nix/store/npzsc982xmckzlmyissmw2r066d03353-ca-certificates.crt

zowoq · 2024-11-17T02:07:45Z

customer@darwin01 ~ % sw_vers
ProductName:            macOS
ProductVersion:         14.7.1
BuildVersion:           23H222

customer@darwin01 ~ % env | grep NIX_SSL_CERT_FILE
NIX_SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt
customer@darwin01 ~ % unset NIX_SSL_CERT_FILE
customer@darwin01 ~ % env | grep NIX_SSL_CERT_FILE
customer@darwin01 ~ % which git
/run/current-system/sw/bin/git
customer@darwin01 ~ % git clone https://github.com/LnL7/nix-darwin
Cloning into 'nix-darwin'...
fatal: unable to access 'https://github.com/LnL7/nix-darwin/': OpenSSL/3.3.2: error:80000002:system library::No such file or directory

customer@darwin01 ~ % echo $NIX_SSL_CERT_FILE
/etc/ssl/certs/ca-certificates.crt
customer@darwin01 ~ % ls -lah $NIX_SSL_CERT_FILE
lrwxr-xr-x  1 root  wheel    41B 16 May  2024 /etc/ssl/certs/ca-certificates.crt -> /etc/static/ssl/certs/ca-certificates.crt
customer@darwin01 ~ % realpath $NIX_SSL_CERT_FILE
/nix/store/5yqs6fsgrxz5wvzlf6qv7ymhy5nmlkk3-ca-certificates.crt

Enzime · 2024-11-17T02:36:54Z

Can you run:

$ nix shell nixpkgs#cctools -c otool -L $(which git)
/etc/profiles/per-user/enzime/bin/git:
	/System/Library/Frameworks/CoreServices.framework/Versions/A/CoreServices (compatibility version 1.0.0, current version 1122.11.0)
	/nix/store/rgc42scp99ji7mammdlmxg6jhb7sci5s-pcre2-10.44/lib/libpcre2-8.0.dylib (compatibility version 14.0.0, current version 14.0.0)
	/nix/store/9cpzwxa2fmkdl2da43968idffwfpcs53-zlib-1.3.1/lib/libz.dylib (compatibility version 1.0.0, current version 1.3.1)
	/nix/store/5v9hx01bnc73jrs2h6n8c1g1fq2g1038-gettext-0.21.1/lib/libintl.8.dylib (compatibility version 12.0.0, current version 12.0.0)
	/nix/store/cjzmz9dskblgkbv4by2wnsyvdjw6jpcm-libiconv-99/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0)
	/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1770.255.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1292.60.1)

Also could you try with a more up to date git maybe:

$ nix run nixpkgs/nixpkgs-unstable#git -- clone https://github.com/LnL7/nix-darwin

zowoq · 2024-11-17T03:18:36Z

customer@darwin01 ~ % realpath $(which git) && nix shell nixpkgs#cctools -c otool -L $(which git)
/nix/store/cyd9di9q6vsdnffn63m75nsnpf425v90-git-2.47.0/bin/git
/run/current-system/sw/bin/git:
        /System/Library/Frameworks/CoreServices.framework/Versions/A/CoreServices (compatibility version 1.0.0, current version 1122.33.0)
        /nix/store/7srv57hn2b24q7idwdc9a4y3pqkn6wy8-pcre2-10.44/lib/libpcre2-8.0.dylib (compatibility version 14.0.0, current version 14.0.0)
        /nix/store/lcdcai6gs0dbsx7a0hz43xb3xwwmb9dl-zlib-1.3.1/lib/libz.dylib (compatibility version 1.0.0, current version 1.3.1)
        /nix/store/dvmq3xa54hfik85259wyf281m076s14h-libiconv-107/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0)
        /nix/store/ymlrvh4zq6imlr4fqsricmhzyrf7g6s9-gettext-0.21.1/lib/libintl.8.dylib (compatibility version 12.0.0, current version 12.0.0)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1292.100.5)

customer@darwin01 ~ % nix run github:nixos/nixpkgs/nixpkgs-unstable#git -- clone https://github.com/LnL7/nix-darwin
Cloning into 'nix-darwin'...
fatal: unable to access 'https://github.com/LnL7/nix-darwin/': TLS connect error: error:80000002:system library::No such file or directory

Enzime · 2024-11-17T03:33:52Z

Hmm, very odd, it works for me

$ unset NIX_SSL_CERT_FILE
$ env | grep SSL
$ /nix/store/cyd9di9q6vsdnffn63m75nsnpf425v90-git-2.47.0/bin/git clone https://github.com/Enzime/aarch64-vm
Cloning into 'aarch64-vm'...
remote: Enumerating objects: 19, done.
remote: Counting objects: 100% (19/19), done.
remote: Compressing objects: 100% (17/17), done.
remote: Total 19 (delta 5), reused 16 (delta 2), pack-reused 0 (from 0)
Receiving objects: 100% (19/19), done.
Resolving deltas: 100% (5/5), done.

Do you have any packages installed through Homebrew?

Enzime · 2024-11-17T03:36:22Z

OpenSSL/3.3.2: error:80000002:system library::No such file or directory

This error seems significant, but it's not very clear what file is not found and I'm not sure if there's an easy way to get OpenSSL to print it out

Can you print out:

$ env | grep SSL
NIX_SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt

zowoq · 2024-11-17T03:57:51Z

customer@darwin01 ~ % env | grep SSL
NIX_SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt

No, homebrew has never been installed.

I've been testing this on the on the nix-community darwin-build-box so I can give you ssh access to it if you'd like.

I can reproduce the same error via remote desktop and ssh on this machine and on our CI builder, they both on the same flake and macos version.

Honestly, I'm rather surprised that it works for you. I don't recall curl, git, etc ever working if NIX_SSL_CERT_FILE is unset, either in a launchd daemon or when run manually.

Enzime · 2024-11-17T04:18:40Z

Yeah if you could give me access, I can investigate further

From my understanding, setting NIX_SSL_CERT_FILE was mainly necessary when using custom SSL certificates, however adding them through security.pki.certificateFiles or security.pki.certificates should work

zowoq · 2024-11-17T04:27:17Z

Is https://github.com/NixOS/aarch64-build-box/blob/b0c87da550f5d954db7fb2c58c59acee0517cf28/keys/enzime still correct?

Enzime · 2024-11-17T04:32:39Z

Yep 👍

https://github.com/Enzime.keys

zowoq · 2024-11-17T04:41:34Z

Ah, I always forget keys are accessible via github.

Deployed in nix-community/infra@1f97557. I gave you access to the nixos community box as well, it is only x86_64 at the moment but nix-community will likely be running a replacement for https://github.com/NixOS/aarch64-build-box in a few weeks so you'll have access to that as well.

https://nix-community.org/community-builder/

Enzime · 2024-11-17T06:10:50Z

I think I've figured it out

OpenSSL is patched to look at /etc/ssl/certs/ca-certificates.crt on Linux and /nix/var/nix/profiles/default/etc/ssl/certs/ca-bundle.crt on macOS:

https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/libraries/openssl/use-etc-ssl-certs-darwin.patch

As I have not uninstalled anything from the default profile since I installed Nix, /nix/var/nix/profiles/default/etc/ssl/certs/ca-bundle.crt exists:

$ nix profile list --profile /nix/var/nix/profiles/default
Name:               nix
Store paths:        /nix/store/x11cpsjg4q236msfz5scc325pfp9xy64-nix-2.15.1

Name:               nss-cacert
Store paths:        /nix/store/pl497357lkfpvdr8v4pfan16k7jna2l2-nss-cacert-3.83

I think the correct solution would be to update the OpenSSL patches to check both /etc/ssl/certs/ca-certificates.crt and /nix/var/nix/profiles/default/etc/ssl/certs/ca-bundle.crt on macOS to better support nix-darwin

zowoq · 2024-11-17T06:46:07Z

Yes, that is it, good find.

I think the correct solution would be to update the OpenSSL patches to check both /etc/ssl/certs/ca-certificates.crt and /nix/var/nix/profiles/default/etc/ssl/certs/ca-bundle.crt on macOS to better support nix-darwin

Agreed.

treewide: use NIX_SSL_CERT_FILE instead of pkgs.cacert

5909f59

Enzime requested changes Nov 16, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

treewide: use NIX_SSL_CERT_FILE instead of pkgs.cacert #1174

treewide: use NIX_SSL_CERT_FILE instead of pkgs.cacert #1174

zowoq commented Nov 16, 2024

Enzime left a comment

zowoq commented Nov 16, 2024

Enzime commented Nov 16, 2024

zowoq commented Nov 16, 2024

Enzime commented Nov 17, 2024 •

edited

Loading

zowoq commented Nov 17, 2024

Enzime commented Nov 17, 2024

zowoq commented Nov 17, 2024

Enzime commented Nov 17, 2024

zowoq commented Nov 17, 2024

Enzime commented Nov 17, 2024

Enzime commented Nov 17, 2024

zowoq commented Nov 17, 2024

Enzime commented Nov 17, 2024

zowoq commented Nov 17, 2024

Enzime commented Nov 17, 2024

zowoq commented Nov 17, 2024 •

edited

Loading

Enzime commented Nov 17, 2024

zowoq commented Nov 17, 2024

treewide: use NIX_SSL_CERT_FILE instead of pkgs.cacert #1174

Are you sure you want to change the base?

treewide: use NIX_SSL_CERT_FILE instead of pkgs.cacert #1174

Conversation

zowoq commented Nov 16, 2024

Enzime left a comment

Choose a reason for hiding this comment

zowoq commented Nov 16, 2024

Enzime commented Nov 16, 2024

zowoq commented Nov 16, 2024

Enzime commented Nov 17, 2024 • edited Loading

zowoq commented Nov 17, 2024

Enzime commented Nov 17, 2024

zowoq commented Nov 17, 2024

Enzime commented Nov 17, 2024

zowoq commented Nov 17, 2024

Enzime commented Nov 17, 2024

Enzime commented Nov 17, 2024

zowoq commented Nov 17, 2024

Enzime commented Nov 17, 2024

zowoq commented Nov 17, 2024

Enzime commented Nov 17, 2024

zowoq commented Nov 17, 2024 • edited Loading

Enzime commented Nov 17, 2024

zowoq commented Nov 17, 2024

Enzime commented Nov 17, 2024 •

edited

Loading

zowoq commented Nov 17, 2024 •

edited

Loading