Replace Datadog/zstd with Klauspost/compress

[jarifibrahim]

Edit - This is a stale PR. Please see #1706 for the latest one.

Fixes #1162
PR #1176 was the first attempt at replacing the library but we decided to not switch the library because of performance issues.

https://github.com/klauspost/compress has become much faster since the last time we benchmarked it in #1176. The new benchmarks in #1162 (comment) look very promising and we should get rid of the CGO dependency.

---

This change is 

[klauspost]

The block size of 4KB was converted to a size of 5554 bytes.

Let me double check. That shouldn't be possible.

[ivanjaros]

Any progress with this?

If I run CGO_ENABLED=0 go get github.com/dgraph-io/badger/v2 from the readme I will get "cannot find package "github.com/dgraph-io/badger/v2" in any of... GOROOT, GOPATH and with cgo/standard I will get exec: "gcc": executable file not found in %PATH% since I do not have gcc. This used to work without any errors....so now I have to switch to BoltDB.

...
i noticed there is build tag in the y/* files so CGO_ENABLED=0 go build works fine. Maybe the readme should be updated.

[klauspost]

@jarifibrahim Ping. You might want to upgrade to v1.11.3, there are some minor tweaks and fixes. I don't see them invalidating the tests, but might as well get them in.

[emolitor]

Has this PR been abandoned? Slightly confused by the state of the ticket as reading through the comments it appears all the issues which were raised have been addressed. Are there other issues which are not documented or is this ticket just waiting for time by the author to test and review?

[mholt]

We'd hate to have to rip this out of Caddy if this PR can't be merged.

[NamanJain8]

@jarifibrahim closing this because the issue is addressed by #1706

[jarifibrahim]

We have merged #1706 which adds the pure go-based ZSTD to badger. This will be released as part of the badger release in July.

@mholt @emolitor @ivanjaros @klauspost Thank you all for your feedback and helping improve badger ❤️ . Apologies for the delay in merging this PR. It got lost in all the other work we're doing on badger/dgraph.

[francislavoie]

I don't think that PR actually solves the problem we have with badger re Caddy. The dependency is still part of the chain, so our users will still end up needing to explicitly specify CGO_ENABLED=0 (I think). We were hoping for this to not be necessary to specify.

[jarifibrahim]

@francislavoie Okay. I think we can set pure go as the default one. I think @klauspost already mentioned that the klauspost/compress should be able to decompress data written by datadog/zstd. So we shouldn't have a problem.

[klauspost]

Yes and vice versa if that should become relevant.