-
Notifications
You must be signed in to change notification settings - Fork 28
Deterministic benchmark order; Keybook interface benchmarks #43
Conversation
License: MIT Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
License: MIT Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
License: MIT Signed-off-by: Adrian Lanzafame <adrianlanzafame92@gmail.com>
f74a212
to
c04ff2d
Compare
Sorry, totally missed this as it was submitted during devcon4. On my plate for today @lanzafame. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, sorry for the delay! Just minor comments that you can apply or ignore – as you wish ;-)
t.Run(name, func(t *testing.T) { | ||
t.Run("Cacheful", func(t *testing.T) { | ||
t.Parallel() | ||
t.Run(name+" Cacheful", func(t *testing.T) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we keep the hierarchical [datastore]/{Cacheful,Cacheless}/[testcase]
format in the full test name, or is the slash a reserved char?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry, didn't see this review. I will be adding some more benchmarks soon, so I will add that in then.
@@ -38,7 +39,15 @@ func BenchmarkPeerstore(b *testing.B, factory PeerstoreFactory, variant string) | |||
go addressProducer(ctx, b, p.ch, p.n) | |||
} | |||
|
|||
for name, bench := range peerstoreBenchmarks { | |||
// So tests are always run in the same order. | |||
ordernames := make([]string, 0, len(peerstoreBenchmarks)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Makes sense, although we can also change the type of peerstoreBenchmarks
to be an array of struct { name string, testFunc func(...) }
so the order is deterministic (I think I had it like this early on!).
Then could do away with the sorting here and in the keybook benchmarks, and apply non-lexicographical orders manually.
Not worth changing, but just wanted to point out an alternative.
No description provided.