Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve set operation speeds #43

Merged
merged 2 commits into from
Dec 5, 2024
Merged

Improve set operation speeds #43

merged 2 commits into from
Dec 5, 2024

Conversation

seiflotfy
Copy link
Member

use intmap.Set instead of map[uint32]struct for set

benchstat old.txt new.txt
goos: darwin
goarch: arm64
pkg: github.com/axiomhq/hyperloglog
cpu: Apple M3 Max
                                      │    old.txt    │               new.txt               │
                                      │    sec/op     │    sec/op     vs base               │
_Merge/size1=100/size2=100-16           17.976µ ± ∞ ¹   7.794µ ± ∞ ¹  -56.64% (p=0.008 n=5)
_Merge/size1=100/size2=10000-16          31.91µ ± ∞ ¹   26.78µ ± ∞ ¹  -16.08% (p=0.008 n=5)
_Merge/size1=100/size2=1000000-16        16.70µ ± ∞ ¹   11.86µ ± ∞ ¹  -29.00% (p=0.008 n=5)
_Merge/size1=10000/size2=100-16          22.21µ ± ∞ ¹   23.00µ ± ∞ ¹        ~ (p=0.548 n=5)
_Merge/size1=10000/size2=10000-16        61.63µ ± ∞ ¹   60.64µ ± ∞ ¹   -1.60% (p=0.008 n=5)
_Merge/size1=10000/size2=1000000-16      34.78µ ± ∞ ¹   33.31µ ± ∞ ¹        ~ (p=0.222 n=5)
_Merge/size1=1000000/size2=100-16        8.316µ ± ∞ ¹   7.948µ ± ∞ ¹   -4.43% (p=0.008 n=5)
_Merge/size1=1000000/size2=10000-16      14.68µ ± ∞ ¹   14.38µ ± ∞ ¹   -2.05% (p=0.008 n=5)
_Merge/size1=1000000/size2=1000000-16    29.66µ ± ∞ ¹   29.03µ ± ∞ ¹        ~ (p=0.548 n=5)
geomean                                  22.78µ         19.36µ        -15.02%
¹ need >= 6 samples for confidence interval at level 0.95

                                      │    old.txt    │                new.txt                │
                                      │     B/op      │      B/op       vs base               │
_Merge/size1=100/size2=100-16           5.847Ki ± ∞ ¹   10.008Ki ± ∞ ¹  +71.17% (p=0.008 n=5)
_Merge/size1=100/size2=10000-16         18.96Ki ± ∞ ¹    21.07Ki ± ∞ ¹  +11.16% (p=0.008 n=5)
_Merge/size1=100/size2=1000000-16       18.96Ki ± ∞ ¹    21.07Ki ± ∞ ¹  +11.16% (p=0.008 n=5)
_Merge/size1=10000/size2=100-16         16.14Ki ± ∞ ¹    16.16Ki ± ∞ ¹   +0.15% (p=0.008 n=5)
_Merge/size1=10000/size2=10000-16       16.14Ki ± ∞ ¹    16.16Ki ± ∞ ¹   +0.15% (p=0.008 n=5)
_Merge/size1=10000/size2=1000000-16     16.14Ki ± ∞ ¹    16.16Ki ± ∞ ¹   +0.15% (p=0.008 n=5)
_Merge/size1=1000000/size2=100-16       16.14Ki ± ∞ ¹    16.16Ki ± ∞ ¹   +0.15% (p=0.008 n=5)
_Merge/size1=1000000/size2=10000-16     16.14Ki ± ∞ ¹    16.16Ki ± ∞ ¹   +0.15% (p=0.008 n=5)
_Merge/size1=1000000/size2=1000000-16   16.14Ki ± ∞ ¹    16.16Ki ± ∞ ¹   +0.15% (p=0.008 n=5)
geomean                                 14.94Ki          16.26Ki         +8.78%
¹ need >= 6 samples for confidence interval at level 0.95

                                      │   old.txt   │              new.txt               │
                                      │  allocs/op  │  allocs/op   vs base               │
_Merge/size1=100/size2=100-16           32.00 ± ∞ ¹   20.00 ± ∞ ¹  -37.50% (p=0.008 n=5)
_Merge/size1=100/size2=10000-16         26.00 ± ∞ ¹   20.00 ± ∞ ¹  -23.08% (p=0.008 n=5)
_Merge/size1=100/size2=1000000-16       26.00 ± ∞ ¹   20.00 ± ∞ ¹  -23.08% (p=0.008 n=5)
_Merge/size1=10000/size2=100-16         4.000 ± ∞ ¹   6.000 ± ∞ ¹  +50.00% (p=0.008 n=5)
_Merge/size1=10000/size2=10000-16       4.000 ± ∞ ¹   6.000 ± ∞ ¹  +50.00% (p=0.008 n=5)
_Merge/size1=10000/size2=1000000-16     4.000 ± ∞ ¹   6.000 ± ∞ ¹  +50.00% (p=0.008 n=5)
_Merge/size1=1000000/size2=100-16       4.000 ± ∞ ¹   6.000 ± ∞ ¹  +50.00% (p=0.008 n=5)
_Merge/size1=1000000/size2=10000-16     4.000 ± ∞ ¹   6.000 ± ∞ ¹  +50.00% (p=0.008 n=5)
_Merge/size1=1000000/size2=1000000-16   4.000 ± ∞ ¹   6.000 ± ∞ ¹  +50.00% (p=0.008 n=5)
geomean                                 7.639         8.963        +17.33%
¹ need >= 6 samples for confidence interval at level 0.95

@seiflotfy seiflotfy merged commit 0cc8976 into main Dec 5, 2024
@ilyam8
Copy link

ilyam8 commented Dec 8, 2024

@seiflotfy, hey. This PR introduces a dependency on github.com/kamstrup/intmap which breaks builds on 32-bit systems. I had to revert to v0.2.0 for now.

/home/cm/go/pkg/mod/github.com/kamstrup/intmap@v0.5.0/map64.go:22:11: 0x9E3779B9 (untyped int constant 2654435769) overflows int

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants