MIPS support?

[polyfractal]

Thanks for xgo, this looks really great! I was curious if MIPS support is possible? The 1.6 release notes mentioned that linux/mips64 and linux/mips64le were now experimentally supported: https://golang.org/doc/go1.6#ports.

I was hoping to crosscompile some go programs to my router :)

[karalabe]

Hmmm, will need to look into MIPS C/C++ toolchains too for this. Can't promise anything just yet but I'll look into it :)

[polyfractal]

Thanks! And no worries if it doesn't work out, I was just curious since it seemed semi-possible now that the go compiler supports it.

Thanks again! :)

[karalabe]

Currently Go doesn't support CGO linking on MIPS64x platforms :(

Relevant issues:

• cmd/link: support cgo internal linking on linux/mipsx, linux/mips64x golang/go#14449
• cmd/cgo: cgo with external linking for linux/mips64 golang/go#14126

Relevant CLs:

• https://go-review.googlesource.com/#/c/19809/
• https://go-review.googlesource.com/#/c/19807/

Until these CLs get merged in I can't add cross compilation support. But will keep this issue open for future memo :)

[polyfractal]

Ah, that's a shame. Thanks for looking into it!

[karalabe]

Seems 1.7 has cgo support for MIPS too. Will investigate this again.

[polyfractal]

Oh awesome, thanks!

[karalabe]

Pling :) I've pushed in a change that should enable MIPS64 and MIPS64le support. Note, that you'll need the 1.7beta1 image at least as it's needed for CGO stuff. Also I don't have a MIPS platform to try out, so apart from checking that builds produce a binary, I couldn't actually try it out. Perhaps if you'd be so kind, I'd appreciate some feedback on this front.

PS: Docker hub seems to be many hours behind in building automated builds, so you'll probably need to build the images yourself for now if you want to try it out.

[polyfractal]

Woo! \o/

I'm on the road at the moment for a work trip, but I'll test it out as soon as I get home

[vincent-olivert-riera]

Hello @karalabe and @polyfractal ,

I was having an issue when trying to build the "flannel" package for MIPS, which needs go, and it was failing like this:

# runtime/cgo
/tmp/go-build396590679/runtime/cgo/_obj/gcc_util.o: In function `x_cgo_thread_start':
../../host/usr/lib/go/src/runtime/cgo/gcc_util.c:48: undefined reference to `_cgo_sys_thread_start'
../../host/usr/lib/go/src/runtime/cgo/gcc_util.c:48: undefined reference to `_cgo_sys_thread_start'
collect2: error: ld returned 1 exit status

That was with go-1.6.1. Then I updated go to 1.7beta2 and it worked fine.

[polyfractal]

Hi @karalabe, finally had a chance to give this a shot (sorry for the delay). I'm running into an error, but I think it's related to the layout of the target Go package, and not the xgo specifically.

I'm trying to compile MetricBeat. The various beats all live in sub-directories of the main Beats repo, and I think that may be causing the problem since it isn't at the "root" of the repo?

$ xgo github.com/elastic/beats/tree/master/metricbeat

Cross compiling github.com/elastic/beats/tree/master/metricbeat...
Fetching main repository github.com/elastic/beats/tree/master/metricbeat...
github.com/elastic/beats (download)
package github.com/elastic/beats/tree/master/metricbeat: cannot find package "github.com/elastic/beats/tree/master/metricbeat" in any of:
    /usr/local/go/src/github.com/elastic/beats/tree/master/metricbeat (from $GOROOT)
    /go/src/github.com/elastic/beats/tree/master/metricbeat (from $GOPATH)
    /go/src/github.com/elastic/beats/tree/master/metricbeat/Godeps/_workspace/src/github.com/elastic/beats/tree/master/metricbeat
    /go/src/github.com/elastic/beats/tree/master/Godeps/_workspace/src/github.com/elastic/beats/tree/master/metricbeat
    /go/src/github.com/elastic/beats/tree/Godeps/_workspace/src/github.com/elastic/beats/tree/master/metricbeat
    /go/src/github.com/elastic/beats/Godeps/_workspace/src/github.com/elastic/beats/tree/master/metricbeat
    /go/src/github.com/elastic/Godeps/_workspace/src/github.com/elastic/beats/tree/master/metricbeat
    /go/src/github.com/Godeps/_workspace/src/github.com/elastic/beats/tree/master/metricbeat
/build.sh: line 73: cd: /go/src/github.com/elastic/beats/tree/master/metricbeat: No such file or directory
2016/06/24 15:06:27 Failed to cross compile package: exit status 1.```

[karalabe]

@polyfractal Sorry, been having some crisis at my day job + been on a holiday afterwards. Seems your import path is wrong, should be just github.com/elastic/beats/metricbeat

[polyfractal]

Sorry, again, for the massive delay. Life and all that :)

I tried again with a fresh pull of the docker image, and this is the result:

xgo github.com/elastic/beats/metricbeat                         
Checking docker installation...  
Client:  
 Version:      1.12.1  
 API version:  1.24  
 Go version:   go1.6.2  
 Git commit:   23cf638  
 Built:        Tue, 27 Sep 2016 12:25:38 +1300  
 OS/Arch:      linux/amd64  

Server:  
 Version:      1.12.1  
 API version:  1.24  
 Go version:   go1.6.2  
 Git commit:   23cf638  
 Built:        Tue, 27 Sep 2016 12:25:38 +1300  
 OS/Arch:      linux/amd64  

Checking for required docker image karalabe/xgo-latest... found.  
Cross compiling github.com/elastic/beats/metricbeat...  
Fetching main repository github.com/elastic/beats/metricbeat...  
github.com/elastic/beats (download)  
Assembling toolchain for android-16/arm...
Bootstrapping android-16/arm...
Compiling for android-16/arm...
Assembling toolchain for android-16/386...
Bootstrapping android-16/386...
Compiling for android-16/386...
No API C header specified, skipping android-16/aar...
Compiling for linux/amd64...
Compiling for linux/386...
Bootstrapping linux/arm-5...
Compiling for linux/arm-5...
Cleaning up Go runtime for linux/arm-5...
Bootstrapping linux/arm-6...
Compiling for linux/arm-6...
Cleaning up Go runtime for linux/arm-6...
Bootstrapping linux/arm-7...
Compiling for linux/arm-7...
Cleaning up Go runtime for linux/arm-7...
Compiling for linux/arm64...
Compiling for linux/mips64...
# github.com/elastic/beats/metricbeat/module/docker/vendor/github.com/fsouza/go-dockerclient/external/github.com/docker/docker/pkg/system
module/docker/vendor/github.com/fsouza/go-dockerclient/external/github.com/docker/docker/pkg/system/stat_linux.go:13: cannot use s.Rdev (type uint32) as type uint64 in field value
2016/11/14 17:03:30 Failed to cross compile package: exit status 2.

[polyfractal]

So I had a closer look at this, and it's just a problem with one of the vendored dependencies. I forked the project, patched it with some build tags... but seem to be unable to get xgo to use the correct code. I even removed the offending file in my forked repo. Xgo still tries to compile it, making me think it's working off the parent repo instead:

xgo --branch mips --targets=*/mips64 --tags='mips64' --remote https://github.com/polyfractal/beats/  github.com/polyfractal/beats/metricbeat
Checking docker installation...
Client:
 Version:      1.12.1
 API version:  1.24
 Go version:   go1.6.2
 Git commit:   23cf638
 Built:        Tue, 27 Sep 2016 12:25:38 +1300
 OS/Arch:      linux/amd64

Server:
 Version:      1.12.1
 API version:  1.24
 Go version:   go1.6.2
 Git commit:   23cf638
 Built:        Tue, 27 Sep 2016 12:25:38 +1300
 OS/Arch:      linux/amd64

Checking for required docker image karalabe/xgo-latest... found.
Cross compiling github.com/polyfractal/beats/metricbeat...
Fetching main repository github.com/polyfractal/beats/metricbeat...
github.com/polyfractal/beats (download)
github.com/elastic/beats (download)
Switching over to remote https://github.com/polyfractal/beats/...
Fetching origin
HEAD is now at 2707369 Rewrite elasticsearch connection URL (#3058)
Switching over to branch mips...
HEAD is now at 1ad51ac Temporarily remove non-mips
Compiling for linux/mips64...
# github.com/elastic/beats/metricbeat/module/docker/vendor/github.com/fsouza/go-dockerclient/external/github.com/docker/docker/pkg/system
../../../elastic/beats/metricbeat/module/docker/vendor/github.com/fsouza/go-dockerclient/external/github.com/docker/docker/pkg/system/stat_linux.go:13: cannot use s.Rdev (type uint32) as type uint64 in field value
2016/11/23 14:02:36 Failed to cross compile package: exit status 2.

Notice that it's trying to compile stat_linux, which was removed in the commit 1ad51ac Temporarily remove non-mips. Xgo reports that the correct branch is checked out, so I'm not sure what's going on.

The command above has all the bell's and whistle's enabled in an attempt to fix the problem, but it should be noted that the basic command has similar behavior (xgo --branch mips --targets=*/mips64 github.com/polyfractal/beats/metricbeat)

[jhammant]

I've been having a bit of a poke around and have managed to get one of the other builds in your repo building:

xgo --branch mips --targets=*/mips64 github.com/polyfractal/beats/filebeat
Checking docker installation...
Client:
 Version:      1.12.1
 API version:  1.24
 Go version:   go1.6.2
 Git commit:   23cf638
 Built:        Tue, 27 Sep 2016 12:25:38 +1300
 OS/Arch:      linux/amd64

Server:
 Version:      1.12.1
 API version:  1.24
 Go version:   go1.6.2
 Git commit:   23cf638
 Built:        Tue, 27 Sep 2016 12:25:38 +1300
 OS/Arch:      linux/amd64

Checking for required docker image karalabe/xgo-latest... found.
Cross compiling github.com/polyfractal/beats/filebeat...
Fetching main repository github.com/polyfractal/beats/filebeat...
github.com/polyfractal/beats (download)
github.com/elastic/beats (download)
Switching over to branch mips...
HEAD is now at 1ad51ac Temporarily remove non-mips
Compiling for linux/mips64...
Cleaning up build environment...
root@mips:~/go/src/github.com# ls -ltr
total 16020
drwxr-xr-x 3 root root     4096 Nov 23 18:41 karalabe
drwxr-xr-x 3 root root     4096 Nov 23 18:42 elastic
-rwxr-xr-x 1 root root 16394480 Nov 24 15:29 filebeat-linux-mips64
root@mips:~/go/src/github.com# file filebeat-linux-mips64
filebeat-linux-mips64: ELF 64-bit MSB executable, MIPS, MIPS64 rel2 version 1, dynamically linked, interpreter /lib64/ld.so.1, BuildID[sha1]=07d41d37abf787f9a8991182b973659a8f48a6f3, for GNU/Linux 3.2.0, not stripped

Did your remove stat_linux from that component as well? Or is it just luck that's working? Will try running the binary on my MIPS hardware later.

[polyfractal]

Hm, interesting! stat_linux is a shared/vendored dependency for all the beats in that repo, but perhaps filebeat doesn't use it directly? Or maybe there's some kind of caching issue on my machine? Not sure, I'll wipe my local install and try again with a fresh image.

[jhammant]

Uploaded it onto my router - still getting issues however:

jon@ubnt:~$ ls -ltr
total 18628
-rwxr-xr-x    1 jon      users     16394480 Nov 25 11:27 filebeat-linux-mips64
jon@ubnt:~$ ./filebeat-linux-mips64
-vbash: ./filebeat-linux-mips64: No such file or directory
jon@ubnt:~$ ldd filebeat-linux-mips64
	not a dynamic executable
jon@ubnt:~$ uname -a
Linux ubnt 3.10.20-UBNT #1 SMP Fri Jul 29 16:51:50 PDT 2016 mips64 GNU/Linux
jon@ubnt:~$

root@mips:~# file filebeat-linux-mips64
filebeat-linux-mips64: ELF 64-bit MSB executable, MIPS, MIPS64 rel2 version 1, dynamically linked, interpreter /lib64/ld.so.1, BuildID[sha1]=07d41d37abf787f9a8991182b973659a8f48a6f3, for GNU/Linux 3.2.0, not stripped

Going to keep poking around to see if I can get it running.

[karalabe]

I just played around with MIPS 32 support (will pish soon)in an emulator, and apparently CGO/gcc may pull in a lot of dynamically linked libraries that won't be available on the device you're building to. I managed to solve those issue for my use case by adding --ldflags '-extldflags "-static"' to the build command.