Vault needs to be built with CGO to use native macOS DNS resolution -- Split DNS not working

[archoversight]

brew gist-logs <formula> link OR brew config AND brew doctor output

HOMEBREW_VERSION: 3.3.12
ORIGIN: https://github.com/Homebrew/brew
HEAD: 768b172393339b4654107dc6a840a2cd08dc64b9
Last commit: 7 days ago
Core tap ORIGIN: https://github.com/Homebrew/homebrew-core
Core tap HEAD: 0ef9ed8ba398cd57fbc7eb216363d37f77e5598c
Core tap last commit: 32 minutes ago
Core tap branch: master
HOMEBREW_PREFIX: /usr/local
HOMEBREW_CASK_OPTS: []
HOMEBREW_CORE_GIT_REMOTE: https://github.com/Homebrew/homebrew-core
HOMEBREW_EDITOR: /usr/local/bin/nvim
HOMEBREW_MAKE_JOBS: 12
HOMEBREW_NO_ANALYTICS: set
Homebrew Ruby: 2.6.8 => /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/bin/ruby
CPU: dodeca-core 64-bit kabylake
Clang: 13.0.0 build 1300
Git: 2.34.1 => /usr/local/bin/git
Curl: 7.77.0 => /usr/bin/curl
macOS: 12.2-x86_64
CLT: N/A
Xcode: 13.2.1

Please note that these warnings are just used to help the Homebrew maintainers
with debugging if you file an issue. If everything you use Homebrew for is
working fine: please don't worry or file an issue; just ignore this. Thanks!

Warning: You have unlinked kegs in your Cellar.
Leaving kegs unlinked can lead to build-trouble and cause formulae that depend on
those kegs to fail to run properly once built. Run `brew link` on these:
  docker-compose
  gnupg
  python@3.9

Verification

• I ran brew update and am still able to reproduce my issue.
• I have resolved all warnings from brew doctor and that did not fix my problem.

What were you trying to do (and why)?

Trying to use the vault binary installed from brew:

% brew info vault
vault: stable 1.9.3 (bottled), HEAD
Secures, stores, and tightly controls access to secrets
https://vaultproject.io/
/usr/local/Cellar/vault/1.9.2 (8 files, 178.7MB) *
  Poured from bottle on 2022-01-10 at 11:09:31
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/vault.rb
License: MPL-2.0
==> Dependencies
Build: go ✘, gox ✘, node@14 ✘, yarn ✔
==> Options
--HEAD
	Install HEAD version
==> Caveats
To restart vault after an upgrade:
  brew services restart vault
Or, if you don't want/need a background service you can just run:
  /usr/local/opt/vault/bin/vault server -dev

With a split DNS setup using OpenVPN from Viscosity and vault is unable to do the DNS lookup because it is attempting to use the DNS server located in /etc/resolv.conf as it is using the netdns (go internal DNS server)

Upstream issue: hashicorp/vault#12012
Issue on Golang specifying work-around by using CGO enabled build: golang/go#12524 (comment)

What happened (include all command output)?

% vault login -method=oidc
Error authenticating: Put "https://vault.example.internal/v1/auth/oidc/oidc/auth_url": dial tcp: lookup vault.example.internal on 172.16.108.111:53: no such host

What did you expect to happen?

Have vault properly resolve the DNS, such as for example ping:

% ping vault.example.internal
PING internal-vault-863034294.us-east-1.elb.amazonaws.com (10.64.12.192): 56 data bytes
64 bytes from 10.64.12.192: icmp_seq=0 ttl=254 time=36.531 ms
64 bytes from 10.64.12.192: icmp_seq=1 ttl=254 time=64.606 ms
^C
--- internal-vault-863034294.us-east-1.elb.amazonaws.com ping statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss

Step-by-step reproduction instructions (by running brew commands)

brew install vault
export VUALT_ADDR=https://vault.example.internal/
vault login -method=oidc

Split DNS can be seen with:

scutil --dns

Which results in output similar to:

DNS configuration (for scoped queries)

resolver #1
  search domain[0] : home.arpa
  nameserver[0] : 172.16.108.111
  nameserver[1] : fd9b:d0c4:d2c8:6c6f::1
  if_index : 13 (en6)
  flags    : Scoped, Request A records, Request AAAA records
  reach    : 0x00000002 (Reachable)

resolver #2
  search domain[0] : home.arpa
  nameserver[0] : 172.16.108.111
  nameserver[1] : fd9b:d0c4:d2c8:6c6f::1
  if_index : 5 (en0)
  flags    : Scoped, Request A records, Request AAAA records
  reach    : 0x00000002 (Reachable)

resolver #3
  search domain[0] : example.internal
  search domain[1] : us-east-1.elb.amazonaws.com
  nameserver[0] : 10.64.22.241
  nameserver[1] : 10.64.20.129
  nameserver[2] : 10.64.24.124
  if_index : 21 (utun10)
  flags    : Scoped, Request A records
  reach    : 0x00000002 (Reachable)

[carlocab]

Upstream seem to have adopted a different approach: hashicorp/vault#13728

[archoversight]

Great news that there is movement upstream!

[carlocab]

Can you try applying this patch and rebuilding vault to see if it fixes things?

diff --git a/Formula/vault.rb b/Formula/vault.rb
index 85a637afa86..5b98e45eba2 100644
--- a/Formula/vault.rb
+++ b/Formula/vault.rb
@@ -32,7 +32,7 @@ class Vault < Formula
 
   def install
     ENV.prepend_path "PATH", "#{ENV["GOPATH"]}/bin"
-    system "make", "bootstrap", "static-dist", "dev-ui"
+    ENV["CGO_ENABLED"] = "1"
+    system "make", "BUILD_TAGS=netcgo", "bootstrap", "static-dist", "dev-ui"
     bin.install "bin/vault"
   end

Just copy the patch above and then do

cd "$(brew --repository homebrew/core)"
pbpaste | patch -p1
brew reinstall -s vault
git restore Formula/vault.rb

(Of course, you'll need to have the patch as the contents of your clipboard to make sure pbpaste works.)

[archoversight]

You need to have CGO_ENABLED=1 as well, see my comments here: hashicorp/vault#13878 (comment)

[carlocab]

I updated the patch. I guess that works?

[archoversight]

@carlocab I have validated that the patch you provided works with a minor change, I am also setting GO_TAGS, since BUILD_TAGS is only used for make dev,

diff --git a/Formula/vault.rb b/Formula/vault.rb
index 85a637afa86..335444521d2 100644
--- a/Formula/vault.rb
+++ b/Formula/vault.rb
@@ -32,7 +32,8 @@ class Vault < Formula
 
   def install
     ENV.prepend_path "PATH", "#{ENV["GOPATH"]}/bin"
-    system "make", "bootstrap", "static-dist", "dev-ui"
+    ENV["CGO_ENABLED"] = "1"
+    system "make", "BUILD_TAGS=netcgo", "GO_TAGS=netcgo", "bootstrap", "static-dist", "dev-ui"
     bin.install "bin/vault"
   end
 

With this patch I am able to use split DNS for accessing various different vaults.

It would be great to get that into homebrew itself so that this works out of the box for everyone.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.