[Auditbeat] Add XXH64 File Integrity Checks (#6626) #7311
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
houndci-bot
reviewed
Jun 13, 2018
| func HashAddXx64(builder *flatbuffers.Builder, xx64 flatbuffers.UOffsetT) { | ||
| builder.PrependUOffsetTSlot(15, flatbuffers.UOffsetT(xx64), 0) | ||
| } | ||
| func HashStartXx64Vector(builder *flatbuffers.Builder, numElems int) flatbuffers.UOffsetT { |
There was a problem hiding this comment.
exported function HashStartXx64Vector should have comment or be unexported
| @@ -374,6 +391,12 @@ func HashAddBlake2b512(builder *flatbuffers.Builder, blake2b512 flatbuffers.UOff | |||
| func HashStartBlake2b512Vector(builder *flatbuffers.Builder, numElems int) flatbuffers.UOffsetT { | |||
| return builder.StartVector(1, numElems, 1) | |||
| } | |||
| func HashAddXx64(builder *flatbuffers.Builder, xx64 flatbuffers.UOffsetT) { | |||
There was a problem hiding this comment.
exported function HashAddXx64 should have comment or be unexported
| return 0 | ||
| } | ||
|
|
||
| func (rcv *Hash) Xx64Length() int { |
There was a problem hiding this comment.
exported method Hash.Xx64Length should have comment or be unexported
| @@ -281,8 +281,25 @@ func (rcv *Hash) Blake2b512Length() int { | |||
| return 0 | |||
| } | |||
|
|
|||
| func (rcv *Hash) Xx64(j int) int8 { | |||
There was a problem hiding this comment.
exported method Hash.Xx64 should have comment or be unexported
original-brownbear
force-pushed
the
6626
branch
2 times, most recently
from
c3d98fb to
f6094bf
Compare
andrewkroh
reviewed
Jun 13, 2018
| @@ -172,6 +172,7 @@ func TestHashFile(t *testing.T) { | |||
| SHA3_256: mustDecodeHex("3cb5385a2987ca45888d7877fbcf92b4854f7155ae19c96cecc7ea1300c6f5a4"), | |||
| SHA3_384: mustDecodeHex("f19539818b4f29fa0ee599db4113fd81b77cd1119682e6d799a052849d2e40ef0dad84bc947ba2dee742d9731f1b9e9b"), | |||
| SHA3_512: mustDecodeHex("f0a2c0f9090c1fd6dedf211192e36a6668d2b3c7f57a35419acb1c4fc7dfffc267bbcd90f5f38676caddcab652f6aacd1ed4e0ad0a8e1e4b98f890b62b6c7c5c"), | |||
| XXH64: mustDecodeHex("d3e8573b7abf279a"), | |||
There was a problem hiding this comment.
✅ Checks out for me.
$ pip install xxhash
macbook15:beats akroh$ python
Python 2.7.14 (default, Mar 10 2018, 00:01:04)
[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.39.2)] on darwin
>>> import xxhash
>>> xxhash.xxh64(b'hello world!\n').hexdigest()
'd3e8573b7abf279a'
| @@ -20,6 +20,8 @@ import ( | |||
| "golang.org/x/crypto/blake2b" | |||
| "golang.org/x/crypto/sha3" | |||
|
|
|||
| "github.com/OneOfOne/xxhash" | |||
There was a problem hiding this comment.
Normally we have three import groups (stdlib, non-elastic, then elastic). So just merge this one into the group above.
| @@ -58,7 +58,7 @@ | |||
|
|
|||
| # Hash types to compute when the file changes. Supported types are | |||
| # blake2b_256, blake2b_384, blake2b_512, md5, sha1, sha224, sha256, sha384, | |||
| # sha512, sha512_224, sha512_256, sha3_224, sha3_256, sha3_384 and sha3_512. | |||
| # sha512, sha512_224, sha512_256, sha3_224, sha3_256, sha3_384, sha3_512 and xxh64. | |||
| @@ -96,7 +96,7 @@ a suffix to the value. The supported units are `b` (default), `kib`, `kb`, `mib` | |||
| *`hash_types`*:: A list of hash types to compute when the file changes. | |||
| The supported hash types are `blake2b_256`, `blake2b_384`, `blake2b_512`, `md5`, | |||
| `sha1`, `sha224`, `sha256`, `sha384`, `sha512`, `sha512_224`, `sha512_256`, | |||
| `sha3_224`, `sha3_256`, `sha3_384`, and `sha3_512`. The default value is `sha1`. | |||
| `sha3_224`, `sha3_256`, `sha3_384`, `sha3_512` and `xxh64`. The default value is `sha1`. | |||
|
@andrewkroh thanks, all fixed I think :) |
|
I'm probably hitting the upper read limit of the SSD with these benchmarks. I should probably refactor them to be performed on data from memory only.
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adds XXH64 for #6626 (as instructed by #6626 (comment))
Definitely significantly faster than the other algorithms:
Output of
BenchmarkHashFile(last one is xxh64)