Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix skewed middleware benchmarks #156

Merged
merged 3 commits into from
Aug 2, 2023
Merged

Fix skewed middleware benchmarks #156

merged 3 commits into from
Aug 2, 2023

Conversation

jub0bs
Copy link
Contributor

@jub0bs jub0bs commented Aug 2, 2023

I noticed two distinct problems in bench_test.go.

Missing Origin header in preflight requests

The Origin header is missing from the preflight request in two benchmarks: BenchmarkPreflight and BenchmarkPreflightHeader. As a result, those benchmarks don't really exercise the middleware's preflight logic.

Re-use of the response object skews benchmarks

In a given benchmark, a FakeResponse object gets re-used across iterations. Accordingly, at the end of a benchmark, the response's headers contain a "Vary" key associated to a huge slice of strings (with "Origin" repeated many times); this is evident if you add the following statement right after the benchmark loop:

fmt.Println(res.header)

Of course, because net/http does not re-use response objects from one handler invocation to the next, those benchmarks are not realistic. More specifically, re-using a http.Header map across benchmark iterations yields skewed results, because the cost of adding an element to the slice of strings associated to key "Vary" is amortised by append (which http.Header.Add uses under the hood). Therefore, the benchmark results look better than they realistically should.

A simple solution is to clear the FakeResponse.header map at the beginning of each benchmark iteration. I've added a small helper to do that, while we wait for the new clear builtin promised by Go 1.21.

New benchmark results

I have not updated the Benchmarks section of the README (you may want to do that yourself after running the benchmarks on your own machine), but here are the results I typically get on my old Macbook Pro:

$ go test -benchmem -run '^$' -bench '^(BenchmarkWithout|BenchmarkDefault|BenchmarkAllowedOrigin|BenchmarkPreflight|BenchmarkPreflightHeader)$'
goos: darwin
goarch: amd64
pkg: github.com/rs/cors
cpu: Intel(R) Core(TM) i7-6700HQ CPU @ 2.60GHz
BenchmarkWithout-8            67202618         16.70 ns/op         3 B/op        1 allocs/op
BenchmarkDefault-8             3527163         340.9 ns/op        51 B/op        4 allocs/op
BenchmarkAllowedOrigin-8       3140804         383.6 ns/op        51 B/op        4 allocs/op
BenchmarkPreflight-8           1291208         966.7 ns/op       160 B/op        6 allocs/op
BenchmarkPreflightHeader-8      944126          1299 ns/op       208 B/op       10 allocs/op

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants