os/user: LookupUser() doesn't find users on macOS when compiled with CGO_ENABLED=0 #24383
Comments
jeffreydwalter
changed the title
|
Hello @jeffreydwalter, I am not sure what you were expecting to see but when running as root, you are a different user entirely so hardcoding the username wouldn't work unless you actually used "root" Here is an update of your repro that'll get you the information from the environment whether ran as root or as an ordinary user, it uses package main
import (
"fmt"
"os"
"os/user"
)
func main() {
u, err := user.Lookup(os.ExpandEnv("$USER"))
if err != nil {
fmt.Printf("%s", err)
return
}
fmt.Printf("%+v\n", u)
}Ran as a normal usere$ go run main.go
&{Uid:501 Gid:20 Username:emmanuelodeke Name:Emmanuel Odeke HomeDir:/Users/emmanuelodeke}Ran as rootsudo su && go run main.go
&{Uid:0 Gid:0 Username:root Name:System Administrator HomeDir:/var/root}Hope this helps. |
|
What I'm expecting is to get back the user info for the username I passed in,
regardless of which user I'm running as.
If that function only returns the current user, then it's misnamed and
should probably be something like LookupCurrentUser().
…On Tue, Mar 13, 2018, 8:30 PM Emmanuel T Odeke ***@***.***> wrote:
Hello @jeffreydwalter <https://github.com/jeffreydwalter>, I am not sure
what you were expecting to see but when running as root, you are a
different user entirely so hardcoding the username wouldn't work unless you
actually used "root"
Here is an update of your repro that'll get you the information from the
environment whether ran as root or as an ordinary user, it uses
os.ExpandEnv("$USER")
package main
import (
"fmt"
"os"
"os/user"
)
func main() {
u, err := user.Lookup(os.ExpandEnv("$USER"))
if err != nil {
fmt.Printf("%s", err)
return
}
fmt.Printf("%+v\n", u)
}
Ran as a normal user
e$ go run main.go &{Uid:501 Gid:20 Username:emmanuelodeke Name:Emmanuel Odeke HomeDir:/Users/emmanuelodeke}
Ran as root
sudo su && go run main.go&{Uid:0 Gid:0 Username:root Name:System Administrator HomeDir:/var/root}
Hope this helps.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#24383 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AAinBwZLGdqRSvJOBd-rGMEXNS6vkBVCks5teHKwgaJpZM4Sprb3>
.
|
|
please use user.Current() |
Gotcha, sorry @jeffreydwalter, I misunderstood your intention. Okay, re-running your original repro but searching for "emmanuelodeke" my user, even as root gives me the correct output &{%!V(string=501) %!V(string=20) %!V(string=emmanuelodeke) %!V(string=Emmanuel Odeke) %!V(string=/Users/emmanuelodeke)}or with sh-3.2# go run main.go
&user.User{Uid:"501", Gid:"20", Username:"emmanuelodeke", Name:"Emmanuel Odeke", HomeDir:"/Users/emmanuelodeke"} |
|
And vice versa, searching for root as my ordinary user gives me the correct output |
|
Thanks @odeke-em, Are you running on macOS? Did you run as root? |
|
Interesting... I wonder if it has to do with the underscore in my username? |
|
@odeke-em, so there must be something with my user. If I lookup root, everything works as it does for you. |
|
I tried creating a new user, same issue as my existing user. :/ |
|
@jeffreydwalter interesting, no dice in a new shell? How about with other users on there? Without underscores either? |
|
@odeke-em no dice... :/ I downloaded iTerm and still no luck. The only things I haven't tried are to log out or reboot. I created a user without underscores, named This is really baffling to me... :) |
|
There are two implementations of
I can't find my normal user in |
|
@crvv on mine with |
|
I am on linux/amd64 (Ubuntu 16.04). Unable to reproduce. With both |
|
@odeke-em |
|
I do have GCO_ENABLED=0 set. I am only able to reproduce this issue on macOS. |
MacOS Darwin Emmanuels-MacBook-Pro-2.local 15.6.0 Darwin Kernel Version 15.6.0: Sun Jun 4 21:43:07 PDT 2017; root:xnu-3248.70.3~1/RELEASE_X86_64 x86_64
No, I can't cat /etc/passwd | grep -i emmproduces nothing |
|
Apparently, macOS doesn't really use the /etc/passwd file anymore. Apparently, you can get the user info by parsing some plist files. See https://apple.stackexchange.com/a/186899 |
|
Just confirmed that everything works fine if I compile with CGO_ENABLED=1. So, seems like parsing /etc/passwd is no bueno on macOS. |
|
So, to recap. Compiling with GCO_ENABLED=0 and calling user.Lookup() on a user other than yourself or root fails. The following commands were run as jeff_walter:The following commands were run as root: |
ALTree
added
the
NeedsInvestigation
jeffreydwalter
changed the title
|
Compiling your program and running (both as root and as my normal username) with CGO_ENABLED=1 and CGO_ENABLED=0 has the same exact output for me on macOS High Sierra 10.13.3: |
bradfitz
changed the title
|
@andybons yes, the code works to find the user you are running as. It does not find other users. I see in your comment, you said, "both as root and as my normal username", but in the output you pasted, I don't see where you ran it as root. |
|
I ran into this issue recently using go 1.11.2 when trying to build a darwin binary from linux. The macos host is 10.14. Given this code Building this on linux with this command: Or leaving CGO_ENABLED=0 out builds the same way, results in a 64-bit mach-o program that can be executed on macos and produces different behavior depending on whether it is run as root or not. Interestingly (and this is somewhat unrelated to the current issue), if I try to use CGO_ENABLED=1 on my linux host while building a darwin binary then the resulting binary cannot be executed on the macos host due to missing libraries (libc.so.6 in particular). (Also note that weird warning as well, but seems harmless I guess) The mach-o binary produced with CGO_ENABLED=0 has just libSystem.B.dylib as a dependency, rather than libc and libpthread. For me what this all means is that I must produce the darwin binary on the macos host itself, since CGO_ENABLED=1 is required to get the |
|
I came across the problem and had a workaround by shell out to macOS's directory service tool The root cause is macOS queries directory service instead of relying on This didn't cause a big problem because we take another path when looking for current user and then fallback to However, I think we should address this problem because:
Some possible solutions:
References
SnippetTo build and run it on a mac echo "Without CGO"
CGO_ENABLED=0 go build -o macuserlookup-gostd main.go
./macuserlookup-gostd --user $USER
sudo ./macuserlookup-gostd --user $USER
echo "With CGO"
CGO_ENABLED=1 go build -o macuserlookup-cgo main.go
./macuserlookup-cgo --user $USER
sudo ./macuserlookup-cgo --user $USER
echo "With dscacheutil"
go build -o macuserlookup-ds main.go
./macuserlookup-ds --user $USER --method ds
sudo ./macuserlookup-ds --user $USER --method dsThe output should be like NOTE: only using std without cgo failed to lookup myself from root package main
/* macuserlookup shows how current os/user.Lookup does not work when not using cgo.
It also provides a workaround using dscacheutil based on https://github.com/tweekmonster/luser
The original issue is found in https://github.com/golang/go/issues/24383#issuecomment-372908869
echo "Without CGO"
CGO_ENABLED=0 go build -o macuserlookup-gostd main.go
./macuserlookup-gostd --user $USER
sudo ./macuserlookup-gostd --user $USER
echo "With CGO"
CGO_ENABLED=1 go build -o macuserlookup-cgo main.go
./macuserlookup-cgo --user $USER
sudo ./macuserlookup-cgo --user $USER
echo "With dscacheutil"
go build -o macuserlookup-ds main.go
./macuserlookup-ds --user $USER --method ds
sudo ./macuserlookup-ds --user $USER --method ds
References
- https://superuser.com/questions/191330/users-dont-appear-in-etc-passwd-on-mac-os-x
- man page for getpwnam_r `These functions obtain information from opendirectoryd(8)`
- iOS https://developer.apple.com/library/archive/documentation/System/Conceptual/ManPages_iPhoneOS/man3/getpwnam_r.3.html
- FreeBSD https://www.freebsd.org/cgi/man.cgi?getpwnam_r
- Linux https://linux.die.net/man/3/getpwnam_r
*/
import (
"flag"
"fmt"
"log"
"os"
"os/exec"
"os/user"
"strings"
)
var (
fUser string
fMethod string
)
func init() {
// Set binary name to prefix because we use different name in the example command
log.SetPrefix(os.Args[0] + " ")
}
func main() {
flag.StringVar(&fUser, "user", "at15", "User to lookup")
flag.StringVar(&fMethod, "method", "std", "std (use cgo), dsc (shell out to macOS's directory service)")
flag.Parse()
lookUp(fMethod, fUser)
}
func lookUp(method string, username string) {
var (
u *user.User
err error
)
log.Printf("Current user is %s Look up user %s using %s", os.Getenv("USER"), username, method)
switch method {
case "std":
u, err = user.Lookup(username)
case "ds":
u, err = dsLookup(username)
default:
log.Fatalf("Unsupported look up method %s", method)
}
if err != nil {
log.Fatalf("Look up user %s using %s failed: %s", username, method, err)
} else {
log.Printf("Found user %s using %s: %s", username, method, u)
}
}
const (
// dsbin is a cli for querying macOS directory service.
dsbin = "dscacheutil"
)
// dsLookup shells out to dscacheutil to get uid, gid from username.
func dsLookup(username string) (*user.User, error) {
// dscacheutil -q user -a name at15
// name: at15
// password: ********
// uid: 123456
// gid: 123456
// dir: /Users/at15
// shell: /bin/zsh
// gecos: At15
//
m, err := runDS("-q", "user", "-a", "name", username)
if err != nil {
return nil, err
}
u := &user.User{
Uid: m["uid"],
Gid: m["gid"],
Username: m["name"],
Name: m["gecos"],
HomeDir: m["dir"],
}
if u.Username == "" || u.Username != username {
return nil, user.UnknownUserError(username)
}
return u, nil
}
// runDS shells out query to dscacheutil and parse the output to key value pair.
func runDS(args ...string) (map[string]string, error) {
b, err := exec.Command(dsbin, args...).CombinedOutput()
if err != nil {
cmd := strings.Join(append([]string{dsbin}, args...), " ")
return nil, fmt.Errorf("error query directory service using %s: %w output %s", cmd, err, b)
}
return parseDSOutput(string(b))
}
// parseDSOutput splits dscacheutil output into key value pair.
// It returns error if no pair is found.
func parseDSOutput(s string) (map[string]string, error) {
const sep = ": "
lines := strings.Split(s, "\n")
m := make(map[string]string)
for _, line := range lines {
keyEnd := strings.Index(line, sep)
if keyEnd <= 0 { // the name must be longer than 1, i.e. `: value` does not exist
continue
}
m[line[:keyEnd]] = line[keyEnd+len(sep):]
}
if len(m) == 0 {
return m, fmt.Errorf("error parse %s output %s", dsbin, s)
}
return m, nil
} |
…cross platforms
The `shell` module now correctly handles `scp` and similar commands. This commit resolves 3 inter-related matters:
1. The issue was recognized when a friend reported `scp` and `rsync` not working. I found the culprit to be not hooking up the session's/channel's I/O to spawned process, otherwise the new process assumed the null devices as its std{in,out,err}. In other words, the newly created process was not reading/writing to/from the client's shell, rather from, e.g., secondary tty device (pair of the pty device) [^1]. Thus the I/O was not channeled from the client to the process, rather to the PTY session. The PR hooks the client's I/O directly as the newly spawned process' std{in,out}. This means when issuing a command which expects to read bytes from stdin, the stdin of the remote process is fed from the local client, not the remote tty. Now `scp` et. al. can work.
2. If you're on macOS, go look for your user in `/etc/passwd`. You will not find it. macOS uses Open Directory[^2] to manage the users, so the users' {meta,}data are stored in the DirectoryService(8) database. In Go, implementation of user lookup in `os/user` pkg uses the native function `getpwnam_r` with cgo; but only parses `/etc/passwd` otherwise, except for lookups of current user which goes through syscall. Lookups return empty results for other users (see golang/go#24383). To avoid cgo and still maintain robust and comprehensive offering, I had to shell out to `dscacheutil` on macOS to obtain full user details and parse /etc/passwd on other *nix platforms.
3. Change the `Shell` field in the `Shell` struct to `ForceCommand`, and use the opportunity to line-up the semantics to match OpenSSH behavior. The `ForceCommand`, if set, is executed using the user's shell, regardless of the command sent along the connection. The designated shell is always the user's default shell, which on *nix systems typically found in `/etc/passwd` and `$SHELL`.
[^1]: https://dev.to/napicella/linux-terminals-tty-pty-and-shell-192e
[^2]: https://en.wikipedia.org/wiki/Apple_Open_Directory
…cross platforms
The `shell` module now correctly handles `scp` and similar commands. This commit resolves 3 inter-related matters:
1. The issue was recognized when a friend reported `scp` and `rsync` not working. I found the culprit to be not hooking up the session's/channel's I/O to spawned process, otherwise the new process assumed the null devices as its std{in,out,err}. In other words, the newly created process was not reading/writing to/from the client's shell, rather from, e.g., secondary tty device (pair of the pty device) [^1]. Thus the I/O was not channeled from the client to the process, rather to the PTY session. The PR hooks the client's I/O directly as the newly spawned process' std{in,out}. This means when issuing a command which expects to read bytes from stdin, the stdin of the remote process is fed from the local client, not the remote tty. Now `scp` et. al. can work.
2. If you're on macOS, go look for your user in `/etc/passwd`. You will not find it. macOS uses Open Directory[^2] to manage the users, so the users' {meta,}data are stored in the DirectoryService(8) database. In Go, implementation of user lookup in `os/user` pkg uses the native function `getpwnam_r` with cgo; but only parses `/etc/passwd` otherwise, except for lookups of current user which goes through syscall. Lookups return empty results for other users (see golang/go#24383). To avoid cgo and still maintain robust and comprehensive offering, I had to shell out to `dscacheutil` on macOS to obtain full user details and parse /etc/passwd on other *nix platforms.
3. Change the `Shell` field in the `Shell` struct to `ForceCommand`, and use the opportunity to line-up the semantics to match OpenSSH behavior. The `ForceCommand`, if set, is executed using the user's shell, regardless of the command sent along the connection. The designated shell is always the user's default shell, which on *nix systems typically found in `/etc/passwd` and `$SHELL`.
[^1]: https://dev.to/napicella/linux-terminals-tty-pty-and-shell-192e
[^2]: https://en.wikipedia.org/wiki/Apple_Open_Directory
Please answer these questions before submitting your issue. Thanks!
What version of Go are you using (
go version)?go version go1.9.2 darwin/amd64
Does this issue reproduce with the latest release?
Yes.
What operating system and processor architecture are you using (
go env)?GOARCH="amd64"
GOBIN=""
GOCACHE="/home/jeff_walter/.cache/go-build"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="freebsd"
GOOS="darwin"
GOPATH="/home/jeff_walter/src/agent"
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/freebsd_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="0"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build053729832=/tmp/go-build -gno-record-gcc-switches -fno-common"
What did you do?
If I run the following program with sudo I get a different result than I get when I run it as 'someuser' (where 'someuser' is my current logged in user).
What did you expect to see?
$ ./user
&{%!V(string=502) %!V(string=20) %!V(string=someuser) %!V(string=) %!V(string=/Users/someuser)}
What did you see instead?
$ sudo ./user
user: unknown user someuser
The text was updated successfully, but these errors were encountered: