go-fuzz does not work with cgo

[mdlayher]

Hello, I previously used go-fuzz with Go 1.4.2, but now that I'm on 1.5.1, it no longer seems to work. In addition, when switching back to 1.4.2, it doesn't seem to work anymore either.

I just nuked my $GOPATH and reinstalled from scratch, just to see what would happen. It still doesn't seem to work. Any ideas?

Thanks for your time.

[zsh|matt@nerr-2]:~/git/go 0 % go version
go version go1.5.1 linux/amd64
[zsh|matt@nerr-2]:~/git/go 0 % go get github.com/mdlayher/ethernet                                                                                                                                       [zsh|matt@nerr-2]:~/git/go 2 % go get github.com/dvyukov/go-fuzz/go-fuzz                                                                            
[zsh|matt@nerr-2]:~/git/go 0 % go get github.com/dvyukov/go-fuzz/go-fuzz-build
[zsh|matt@nerr-2]:~/git/go 0 % cd src/github.com/mdlayher/ethernet 
[zsh|matt@nerr-2]:~/git/go/src/github.com/mdlayher/ethernet 0 (master) ± cat fuzz.go 
// +build gofuzz

package ethernet

func Fuzz(data []byte) int {
        f := new(Frame)
        if err := f.UnmarshalBinary(data); err != nil {
                return 0
        }

        if _, err := f.MarshalBinary(); err != nil {
                panic(err)
        }

        if err := f.UnmarshalFCS(data); err != nil {
                return 0
        }

        if _, err := f.MarshalFCS(); err != nil {
                panic(err)
        }

        return 1
}
[zsh|matt@nerr-2]:~/git/go/src/github.com/mdlayher/ethernet 0 (master) ± go-fuzz-build github.com/mdlayher/ethernet
[zsh|matt@nerr-2]:~/git/go/src/github.com/mdlayher/ethernet 0 *(master) ± go-fuzz -bin=./ethernet-fuzz.zip -workdir=fuzz/
2015/09/16 11:19:51 slaves: 8, corpus: 1 (3s ago), crashers: 0, restarts: 1/0, execs: 0 (0/sec), cover: 0, uptime: 3s
2015/09/16 11:19:54 slaves: 8, corpus: 1 (6s ago), crashers: 0, restarts: 1/0, execs: 0 (0/sec), cover: 0, uptime: 6s
2015/09/16 11:19:57 slaves: 8, corpus: 1 (9s ago), crashers: 0, restarts: 1/0, execs: 0 (0/sec), cover: 0, uptime: 9s
2015/09/16 11:20:00 slaves: 8, corpus: 1 (12s ago), crashers: 0, restarts: 1/0, execs: 0 (0/sec), cover: 0, uptime: 12s
2015/09/16 11:20:03 slaves: 8, corpus: 1 (15s ago), crashers: 0, restarts: 1/0, execs: 0 (0/sec), cover: 0, uptime: 15s
2015/09/16 11:20:06 slaves: 8, corpus: 1 (18s ago), crashers: 0, restarts: 1/0, execs: 0 (0/sec), cover: 0, uptime: 18s
2015/09/16 11:20:09 slaves: 8, corpus: 1 (21s ago), crashers: 0, restarts: 1/0, execs: 0 (0/sec), cover: 0, uptime: 21s
2015/09/16 11:20:12 slaves: 8, corpus: 1 (24s ago), crashers: 0, restarts: 1/0, execs: 0 (0/sec), cover: 0, uptime: 24s
2015/09/16 11:20:15 slaves: 8, corpus: 1 (27s ago), crashers: 0, restarts: 1/0, execs: 0 (0/sec), cover: 0, uptime: 27s
^C2015/09/16 11:20:17 shutting down...

[dvyukov]

Thanks for the report!
It seems to be broken after switch to go/types package for instrumentation. The bug affects packages that transitively use cgo (in your case it is net package).
While I am working on a proper fix, a workaround is to set CGO_ENABLED=0 for go-fuzz-build:

$ CGO_ENABLED=0 go-fuzz-build github.com/mdlayher/ethernet

This should fix fuzzing for now.

[mdlayher]

That fixed it!

[zsh|matt@nerr-2]:~/go/ethernet 0 *(master) ± CGO_ENABLED=0 go-fuzz-build github.com/mdlayher/ethernet
[zsh|matt@nerr-2]:~/go/ethernet 0 *(master) ± go-fuzz -bin=./ethernet-fuzz.zip -workdir=fuzz/         
2015/09/16 12:35:03 slaves: 8, corpus: 8 (1s ago), crashers: 0, restarts: 1/0, execs: 0 (0/sec), cover: 0, uptime: 3s
2015/09/16 12:35:06 slaves: 8, corpus: 8 (4s ago), crashers: 0, restarts: 1/0, execs: 0 (0/sec), cover: 46, uptime: 6s
2015/09/16 12:35:09 slaves: 8, corpus: 8 (7s ago), crashers: 0, restarts: 1/6946, execs: 250081 (27783/sec), cover: 46, uptime: 9s
2015/09/16 12:35:12 slaves: 8, corpus: 8 (10s ago), crashers: 0, restarts: 1/8016, execs: 480970 (40080/sec), cover: 46, uptime: 12s

Thanks for the information.

[dvyukov]

Good!

You may also consider that the package is not changed after marshal/unmarshal roundtrip as:

        f := new(Frame)
        if err := f.UnmarshalBinary(data); err != nil {
                return 0
        }
        if data1, err := f.MarshalBinary(); err != nil {
                panic(err)
        }
        f1 := new(Frame)
        if err := f1.UnmarshalBinary(data1); err != nil {
                return 0
        }
        if !reflect.DeepEqual(f, f1) {
                panic("bad")
        }

If reflect.DeepEqual won't work for you, there is also github.com/dvyukov/go-fuzz/examples/fuzz.Equal which is slightly more relaxed version of DeepEqual.

Also if UnmarshalBinary fails, then you don't invoke UnmarshalFCS. If these are different formats, then I would just write two separate tests with different workdir's (so that they have different input corpus).

[mdlayher]

Thanks for the advice! I haven't gone too in-depth yet with my testing, but it is certainly something I'd like to investigate in the future.

[dvyukov]

The problem is that go/types does not support cgo well.
I've filed golang/go#12667 upstream.
When it is fixed, we will need to figure out how to feed cgo-generated sources into go/types.
This sucks. Sorry. I was not aware of the go/types cgo issues.

[q6r]

This is still not fixed.

[dgryski]

It's seems like of your package is cgo you're better off using afl or libfuzzer to fuzz the C library directly.

[dvyukov]

@q6r Does exporting CGO_ENABLED=0 help in your case?

[q6r]

I'm fuzzing something that requires cgo. There are other better options for
my case like afl.
On Feb 22, 2016 1:20 AM, "Dmitry Vyukov" notifications@github.com wrote:

@q6r https://github.com/q6r Does exporting CGO_ENABLED=0 help in your
case?

—
Reply to this email directly or view it on GitHub
#101 (comment).

[dvyukov]

@q6r you can also consider libfuzzer (http://llvm.org/docs/LibFuzzer.html) which is way faster than AFL.

[obscuren]

I have a program that requires cgo and I was wondering if there is any (planned) progress on this issue?

[dvyukov]

@obscuren do you want to test native code? can you stub native code in tests?

[obscuren]

@dvyukov I guess I'm just looking for the easy way out ;-) I'll see if I can stub them away

[AlekSi]

I've filed golang/go#12667 upstream.
When it is fixed, we will need to figure out how to feed cgo-generated sources into go/types.

That upstream bug was fixed.
I wonder how hard it will be to make go-fuzz skip cgo code while fuzzing Go.

[pventuzelo]

Hey, any updates regarding cgo support in go-fuzz?

[elichai]

I'm also interested, I want to fuzz my go implementation against a C implementation

[josharian]

One workaround is to get the cgo/C results by executing another binary, or making an RPC call. It’s ugly, but it does work—go-fuzz has found a bunch of compiler bugs that way.

[elichai]

For future reference, I ended up using libfuzzer directly, example with msan:

$ CC=clang go build -buildmode c-archive -msan -gcflags "all=-d=libfuzzer" -tags gofuzz,gofuzz_libfuzzer,libfuzzer -trimpath -o fuzz.a
$ clang -fsanitize=fuzzer,memory fuzz.a  -o fuzz