cmd/compile: efficient way to compare a byte slice and a string

[ibukanov]

Currently Go does not provide a utility to compare efficiently a byte slice against a string. Doing string(byteSlice) == str is inefficient as it copies the slice first into a new string. It would be nice to have a utility like bytes.EqualString(b []byte, s string) that is just as efficient as bytes.Equal(a, b []byte). Another alternative is to optimize string(byteSlice) compareOp str so compareOp accesses the slice directly. The latter would benefit the existing code even if it is harder to implement.

[nussjustin]

Go 1.5 has what you want. It was implemented in CL #3120.

[ibukanov]

@nuss-justin Now I see why string(byteSlice) == str became 3 times faster in Go 1.5 according to http://stackoverflow.com/a/31484416/2898400 . But it still 8 times slower than calling bytes.Equal via an unsafe cast. And this is in an ideal case when whole 100 byte long slice has to be read for the answer. So escape analyzes helps, but not that match as bytes still has to be copied.

[ALTree]

Doing string(byteSlice) == str is inefficient as it copies the slice first into a new string

I don't believe this is true. If you disassemble

var bs []byte = []byte{104, 97, 108, 108, 111}

func main() {
    x := string(bs) == "hello"
    println(x)
}

you'll see that string(bs) is compiled to a call to runtime.slicebytetostringtmp, which, as you can see, does not copy the data:

func slicebytetostringtmp(b []byte) string {
    // Return a "string" referring to the actual []byte bytes.
    // This is only for use by internal compiler optimizations
    // that know that the string form will be discarded before
    // the calling goroutine could possibly modify the original
    // slice or synchronize with another goroutine.
    // First such case is a m[string(k)] lookup where
    // m is a string-keyed map and k is a []byte.
    // Second such case is "<"+string(b)+">" concatenation where b is []byte.
    // Third such case is string(b)=="foo" comparison where b is []byte.

    if raceenabled && len(b) > 0 {
        racereadrangepc(unsafe.Pointer(&b[0]),
            uintptr(len(b)),
            getcallerpc(unsafe.Pointer(&b)),
            funcPC(slicebytetostringtmp))
    }
    if msanenabled && len(b) > 0 {
        msanread(unsafe.Pointer(&b[0]), uintptr(len(b)))
    }
    return *(*string)(unsafe.Pointer(&b))
}

Benchmarking also shows that there's almost no difference between the cast and a call to bytes.Equal. On tip:

var res bool

func BenchmarkStrCast(b *testing.B) {
    s1 := strings.Repeat("a", 1000)
    b2 := []byte(strings.Repeat("a", 999) + "b")
    for n := 0; n < b.N; n++ {
        res = string(b2) == s1
    }
}

func BenchmarkBytesCmp(b *testing.B) {
    b1 := []byte(strings.Repeat("a", 1000))
    b2 := []byte(strings.Repeat("a", 999) + "b")
    for n := 0; n < b.N; n++ {
        res = bytes.Equal(b1, b2)
    }
}

gives

BenchmarkStrCast-4  50000000            28.5 ns/op         0 B/op          0 allocs/op
BenchmarkBytesCmp-4 50000000            27.1 ns/op         0 B/op          0 allocs/op