-
Notifications
You must be signed in to change notification settings - Fork 3
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
The current version of the cache uses `string` as the key and `interface{}` as the value. That fits the use case for which it was designed, but it is not as flexible as it could be. Go 1.18 generics created an opportunity to do change that. The [container/heap](https://pkg.go.dev/container/heap) in the standard library doesn't support generics. I'm sure it will at some point, but for now, the source code was copied from the standard libary and made generic. The `pq` and `lazylru` components were copied into a subpackage, `generic`. While the internals of the library (and especially the tests) are littered with type annotations, the external interface is pretty clean. Previously, the cache would be used like so: ```go // import "github.com/TriggerMail/lazylru" lru := lazylru.New(maxItems, ttl) lru.Set("key", "value") if v, ok := lru.Get("key"); ok { vstr, ok := v.(string) if !ok { panic("something terrible has happened") } fmt.Println(vstr) } ``` The new version is a bit cleaner: ```go // import "github.com/TriggerMail/lazylru/generic" lru := lazylru.NewT[string, string](maxItems, ttl) lru.Set("key", "value") if v, ok := lru.Get("key"); ok { fmt.Println(v) } ``` It's expected that the cache is going to be created at the start of a program and accessed many times, so the real win is the lack of casting on the `Get`. It is easy to put in a value when you mean a pointer or a pointer when you mean a value, but the generic version prevents that problem. The `panic` in the sample code above is a maybe overkill, but the caller is likely to do _something_ to deal with type. There is a performance impact to the casting, but it doesn't appear to be huge. In terms of caching performance, there was an improvement in all cases. I tested the original, interface-based implementation as well as a generic implementation of [string, interface{}] to mimic the interface type as closely as possible and a generic implementation of `[string, int]` to see what the improvement would be. Tests were run on an Apple Macbook Pro M1. An excerpt of the benchmarch is listed below: ```text 1% W, 99% R 99% W, 1% R ------------------------- ------------ ------------ interface-based 60.94 ns/op 107.80 ns/op generic[string,interface] 54.21 ns/op 87.76 ns/op generic[string,int] 53.24 ns/op 93.80 ns/op ``` * Separate interface and generic versions to allow the consumer to select the generic as a version * Make testing work under go 1.18 * golang:rc-buster image * go fmt * installing go-junit-report properly * adding test-results to .gitignore * Building with Earthly to make life easier on myself * Using revive rather than golangci-lint because golangci-lint doesn't work with go 1.18 yet * Publishing coverage results to coveralls * On interface (top-level) only because goveralls doesn't like submodules * README badges for coverage * trying out coveralls flags * passing the build number from buildkite to Earthly * passing build number as jobname to coveralls * adding git above generic so coveralls figures its stuff out * fairly meaningless fuzz tests * benchmark * update readme
- Loading branch information
1 parent
c90e924
commit 2d7d9a8
Showing
41 changed files
with
2,631 additions
and
269 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
.buildkite/ | ||
.vscode/ | ||
vendor/ | ||
generic/vendor/ | ||
test-results/ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,2 +1,3 @@ | ||
.vscode/ | ||
.DS_Store | ||
test-results/ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,30 @@ | ||
run: | ||
go: "1.17" | ||
timeout: 5m | ||
issues-exit-code: 1 | ||
tests: true | ||
skip-dirs-use-default: true | ||
modules-download-mode: vendor | ||
|
||
linters: | ||
enable: | ||
- revive | ||
- gofmt | ||
- govet | ||
- gosec | ||
- unconvert | ||
- goconst | ||
- gocyclo | ||
- goimports | ||
|
||
linters-settings: | ||
govet: | ||
enable: | ||
- fieldalignment | ||
fieldalignment: | ||
fix: true | ||
|
||
issues: | ||
exclude: | ||
- EXC0002 # Annoying issue about not having a comment | ||
- EXC0012 # func should have comment or be unexported |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,267 @@ | ||
FROM golang:1.18 | ||
|
||
all-bench: | ||
BUILD +fmt-bench | ||
BUILD +lint-bench | ||
BUILD +vet-bench | ||
BUILD +test-bench | ||
|
||
all-interface: | ||
BUILD +fmt-interface | ||
BUILD +lint-interface | ||
BUILD +vet-interface | ||
BUILD +test-interface | ||
|
||
all-generic: | ||
BUILD +fmt-generic | ||
BUILD +lint-generic | ||
BUILD +vet-generic | ||
BUILD +test-generic | ||
|
||
all: | ||
BUILD +all-bench | ||
BUILD +all-interface | ||
BUILD +all-generic | ||
|
||
ci-bench: | ||
ARG --required BUILD_NUMBER | ||
BUILD +fmt-bench | ||
BUILD +lint-bench | ||
BUILD +vet-bench | ||
COPY --dir +test-bench/files ./test-results/bench | ||
SAVE ARTIFACT ./test-results/bench AS LOCAL test-results/bench | ||
|
||
BUILD +publish-coverage-bench --BUILD_NUMBER=$BUILD_NUMBER | ||
|
||
ci-interface: | ||
ARG --required BUILD_NUMBER | ||
BUILD +fmt-interface | ||
BUILD +lint-interface | ||
BUILD +vet-interface | ||
COPY --dir +test-interface/files ./test-results/interface | ||
SAVE ARTIFACT ./test-results/interface AS LOCAL test-results/interface | ||
BUILD +publish-coverage-interface --BUILD_NUMBER=$BUILD_NUMBER | ||
|
||
ci-generic: | ||
ARG --required BUILD_NUMBER | ||
BUILD +fmt-generic | ||
BUILD +lint-generic | ||
BUILD +vet-generic | ||
COPY --dir +test-generic/files ./test-results/generic | ||
SAVE ARTIFACT ./test-results/generic AS LOCAL test-results/generic | ||
BUILD +publish-coverage-generic --BUILD_NUMBER=$BUILD_NUMBER | ||
|
||
ci: | ||
ARG --required BUILD_NUMBER | ||
BUILD +ci-bench --BUILD_NUMBER=$BUILD_NUMBER | ||
BUILD +ci-interface --BUILD_NUMBER=$BUILD_NUMBER | ||
BUILD +ci-generic --BUILD_NUMBER=$BUILD_NUMBER | ||
|
||
go-mod-bench: | ||
WORKDIR /bench | ||
RUN git config --global url."git@github.com:".insteadOf "https://github.com/" | ||
RUN mkdir -p -m 0600 ~/.ssh && ssh-keyscan github.com >> ~/.ssh/known_hosts | ||
COPY go.mod go.sum . | ||
RUN --ssh go mod download | ||
|
||
go-mod-interface: | ||
WORKDIR /app | ||
RUN git config --global url."git@github.com:".insteadOf "https://github.com/" | ||
RUN mkdir -p -m 0600 ~/.ssh && ssh-keyscan github.com >> ~/.ssh/known_hosts | ||
COPY go.mod go.sum . | ||
RUN --ssh go mod download | ||
|
||
go-mod-generic: | ||
WORKDIR /generic | ||
RUN git config --global url."git@github.com:".insteadOf "https://github.com/" | ||
RUN mkdir -p -m 0600 ~/.ssh && ssh-keyscan github.com >> ~/.ssh/known_hosts | ||
COPY go.mod go.sum . | ||
RUN --ssh go mod download | ||
|
||
go-mod: | ||
BUILD +go-mod-bench | ||
BUILD +go-mod-interface | ||
BUILD +go-mod-generic | ||
|
||
fmt-bench: | ||
COPY --dir ./bench /bench | ||
WORKDIR /bench | ||
RUN find . -type d -path "./vendor" -prune -o -name "*.go" -exec gofmt -d -e {} \; | tee /tmp/gofmt.out | ||
RUN bash -c 'if [[ -s /tmp/gofmt.out ]]; then exit 1; fi' | ||
|
||
fmt-interface: | ||
COPY --dir . /app | ||
RUN rm -rf generic | ||
RUN rm -rf bench | ||
RUN find . -type d -path "./vendor" -prune -o -name "*.go" -exec gofmt -d -e {} \; | tee /tmp/gofmt.out | ||
RUN bash -c 'if [[ -s /tmp/gofmt.out ]]; then exit 1; fi' | ||
|
||
fmt-generic: | ||
COPY --dir ./generic /generic | ||
WORKDIR /generic | ||
RUN find . -type d -path "./vendor" -prune -o -name "*.go" -exec gofmt -d -e {} \; | tee /tmp/gofmt.out | ||
RUN bash -c 'if [[ -s /tmp/gofmt.out ]]; then exit 1; fi' | ||
|
||
fmt: | ||
BUILD +fmt-bench | ||
BUILD +fmt-interface | ||
BUILD +fmt-generic | ||
|
||
vendor-bench: | ||
FROM +go-mod-bench | ||
WORKDIR /app | ||
COPY --dir . . | ||
WORKDIR /app/bench | ||
RUN --ssh go mod vendor | ||
SAVE ARTIFACT . files | ||
|
||
vendor-interface: | ||
FROM +go-mod-interface | ||
WORKDIR /app | ||
COPY --dir . . | ||
RUN rm -rf generic | ||
RUN rm -rf bench | ||
RUN --ssh go mod vendor | ||
SAVE ARTIFACT . files | ||
|
||
vendor-generic: | ||
FROM +go-mod-generic | ||
WORKDIR /app | ||
COPY --dir .git . | ||
COPY ./generic ./generic | ||
WORKDIR /app/generic | ||
RUN --ssh go mod vendor | ||
SAVE ARTIFACT . files | ||
|
||
vendor: | ||
BUILD +vendor-bench | ||
BUILD +vendor-interface | ||
BUILD +vendor-generic | ||
|
||
lint-bench: | ||
FROM +vendor-bench | ||
COPY +golangci-lint/go/bin/golangci-lint /go/bin/golangci-lint | ||
RUN golangci-lint run | ||
|
||
lint-interface: | ||
FROM +vendor-interface | ||
COPY +golangci-lint/go/bin/golangci-lint /go/bin/golangci-lint | ||
RUN golangci-lint run | ||
|
||
lint-generic: | ||
FROM +vendor-generic | ||
COPY +golangci-lint/go/bin/golangci-lint /go/bin/golangci-lint | ||
RUN golangci-lint run | ||
|
||
lint: | ||
BUILD +lint-bench | ||
BUILD +lint-interface | ||
BUILD +lint-generic | ||
|
||
vet-bench: | ||
FROM +vendor-bench | ||
RUN go vet ./... | ||
|
||
vet-interface: | ||
FROM +vendor-interface | ||
RUN go vet ./... | ||
|
||
vet-generic: | ||
FROM +vendor-generic | ||
RUN go vet ./... | ||
|
||
vet: | ||
BUILD +vet-bench | ||
BUILD +vet-interface | ||
BUILD +vet-generic | ||
|
||
test-bench: | ||
FROM +vendor-bench | ||
COPY +junit-report/go/bin/go-junit-report /go/bin/go-junit-report | ||
RUN go version | ||
RUN mkdir -p test-results | ||
# To both see the output in the console AND convert into junit-style results | ||
# to send to the plug-in, we need to run the tests, writing to a file, then | ||
# send that file to go-junit-report | ||
RUN 2>&1 go test -race -v ./... -cover -coverprofile=test-results/cover.out | tee test-results/go-test-bench.out | ||
RUN cat test-results/go-test-bench.out | $GOPATH/bin/go-junit-report > test-results/go-test-bench-report.xml | ||
SAVE ARTIFACT test-results files | ||
|
||
test-interface: | ||
FROM +vendor-interface | ||
COPY +junit-report/go/bin/go-junit-report /go/bin/go-junit-report | ||
RUN go version | ||
RUN mkdir -p test-results | ||
# To both see the output in the console AND convert into junit-style results | ||
# to send to the plug-in, we need to run the tests, writing to a file, then | ||
# send that file to go-junit-report | ||
RUN 2>&1 go test -race -v ./... -cover -coverprofile=test-results/cover.out | tee test-results/go-test-interface.out | ||
RUN cat test-results/go-test-interface.out | $GOPATH/bin/go-junit-report > test-results/go-test-interface-report.xml | ||
SAVE ARTIFACT test-results files | ||
|
||
test-generic: | ||
FROM +vendor-generic | ||
COPY +junit-report/go/bin/go-junit-report /go/bin/go-junit-report | ||
RUN go version | ||
RUN mkdir -p test-results | ||
# To both see the output in the console AND convert into junit-style results | ||
# to send to the plug-in, we need to run the tests, writing to a file, then | ||
# send that file to go-junit-report | ||
RUN 2>&1 go test -race -v ./... -cover -coverprofile=test-results/cover.out | tee test-results/go-test-generic.out | ||
RUN cat test-results/go-test-generic.out | $GOPATH/bin/go-junit-report > test-results/go-test-generic-report.xml | ||
SAVE ARTIFACT test-results files | ||
|
||
test: | ||
COPY --dir +test-bench/files ./test-results/bench | ||
COPY --dir +test-interface/files ./test-results/interface | ||
COPY --dir +test-generic/files ./test-results/generic | ||
SAVE ARTIFACT ./test-results AS LOCAL test-results | ||
|
||
publish-coverage-interface: | ||
ARG --required BUILD_NUMBER | ||
FROM +test-interface | ||
COPY +goveralls/go/bin/goveralls /go/bin/goveralls | ||
RUN --no-cache --secret COVERALLS_TOKEN=+secrets/COVERALLS_TOKEN \ | ||
goveralls \ | ||
-jobnumber="$BUILD_NUMBER" \ | ||
-flagname=interface \ | ||
-service=buildkite \ | ||
-coverprofile=test-results/cover.out | ||
|
||
publish-coverage-bench: | ||
ARG --required BUILD_NUMBER | ||
FROM +test-bench | ||
COPY +goveralls/go/bin/goveralls /go/bin/goveralls | ||
RUN --no-cache --secret COVERALLS_TOKEN=+secrets/COVERALLS_TOKEN \ | ||
goveralls \ | ||
-jobnumber="$BUILD_NUMBER" \ | ||
-flagname=bench \ | ||
-service=buildkite \ | ||
-coverprofile=test-results/cover.out | ||
|
||
publish-coverage-generic: | ||
ARG --required BUILD_NUMBER | ||
FROM +test-generic | ||
COPY +goveralls/go/bin/goveralls /go/bin/goveralls | ||
RUN --no-cache --secret COVERALLS_TOKEN=+secrets/COVERALLS_TOKEN \ | ||
goveralls \ | ||
-jobnumber="$BUILD_NUMBER" \ | ||
-flagname=generic \ | ||
-service=buildkite \ | ||
-coverprofile=test-results/cover.out | ||
|
||
# These are tools that are used in the targets above | ||
goveralls: | ||
RUN echo Installing goveralls | ||
RUN go install github.com/mattn/goveralls@latest | ||
SAVE ARTIFACT /go/bin/goveralls /go/bin/goveralls | ||
|
||
golangci-lint: | ||
RUN echo Installing golangci-lint... | ||
# see https://golangci-lint.run/usage/install/#other-ci | ||
RUN curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b /go/bin v1.45.0 | ||
SAVE ARTIFACT /go/bin/golangci-lint /go/bin/golangci-lint | ||
|
||
junit-report: | ||
RUN go install github.com/jstemmer/go-junit-report@latest | ||
SAVE ARTIFACT /go/bin/go-junit-report /go/bin/go-junit-report |
Oops, something went wrong.