Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid some cgoCheckPointer calls #122

Open
wants to merge 1 commit into
base: 1.x
Choose a base branch
from
Open

Conversation

jwebb
Copy link

@jwebb jwebb commented Nov 14, 2022

This is I believe a more correct version of #31 which was attempted and reverted back in 2018. The problem currently is that use of unsafe.Pointer leads to cgo defensively inserting calls to this monster: https://github.com/golang/go/blob/master/src/runtime/cgocall.go#L393

Aside from the cost of running that code, it also triggers allocations in order to wrap the slice headers into the any arguments. This is also the cause of the panics worked around previously in 0ead11a

By passing char* rather than void* we allow cgo to understand that there cannot possibly be any pointers hiding in our data buffers, and skip the checks entirely. In principle this could be applied to every use of unsafe.Pointer, but the three updated here seem the most likely cases to be used with small inputs, where the performance effect is noticeable.

Before:

$ PAYLOAD=LICENSE go test -bench='BenchmarkCompression|BenchmarkDecompression' -benchmem -run='^$' .
goos: linux
goarch: amd64
pkg: github.com/DataDog/zstd
cpu: Intel(R) Core(TM) i7-8700 CPU @ 3.20GHz
BenchmarkCompression-12      	   75825	     15826 ns/op	  98.51 MB/s	      48 B/op	       2 allocs/op
BenchmarkDecompression-12    	  295032	      3954 ns/op	 394.28 MB/s	      72 B/op	       3 allocs/op
--- BENCH: BenchmarkDecompression-12
    zstd_test.go:339: Reduced from 1559 to 855
    zstd_test.go:339: Reduced from 1559 to 855
    zstd_test.go:339: Reduced from 1559 to 855
    zstd_test.go:339: Reduced from 1559 to 855
PASS
ok  	github.com/DataDog/zstd	6.293s

After:

$ PAYLOAD=LICENSE go test -bench='BenchmarkCompression|BenchmarkDecompression' -benchmem .
goos: linux
goarch: amd64
pkg: github.com/DataDog/zstd
cpu: Intel(R) Core(TM) i7-8700 CPU @ 3.20GHz
BenchmarkCompression-12      	   75852	     15731 ns/op	  99.11 MB/s	       0 B/op	       0 allocs/op
BenchmarkDecompression-12    	  296256	      3645 ns/op	 427.73 MB/s	       0 B/op	       0 allocs/op
--- BENCH: BenchmarkDecompression-12
    zstd_test.go:340: Reduced from 1559 to 855
PASS
ok  	github.com/DataDog/zstd	7.998s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant