Go 1.18 generics for lazylru

[dangermike]

The current version of the cache uses string as the key and interface{} as the value. That fits the use case for which it was designed, but it is not as flexible as it could be. Go 1.18 generics created an opportunity to do change that.

The container/heap in the standard library doesn't support generics. I'm sure it will at some point, but for now, the source code was copied from the standard libary and made generic. The pq and lazylru components were copied into a subpackage, generic. While the internals of the library (and especially the tests) are littered with type annotations, the external interface is pretty clean. Previously, the cache would be used like so:

// import "github.com/TriggerMail/lazylru"

lru := lazylru.New(maxItems, ttl)
lru.Set("key", "value")
if v, ok := lru.Get("key"); ok {
	vstr, ok := v.(string)
	if !ok {
		panic("something terrible has happened")
	}
	fmt.Println(vstr)
}

The new version is a bit cleaner:

// import "github.com/TriggerMail/lazylru/generic"

lru := lazylru.New[string, string](maxItems, ttl)
lru.Set("key", "value")
if v, ok := lru.Get("key"); ok {
	fmt.Println(v)
}

It's expected that the cache is going to be created at the start of a program and accessed many times, so the real win is the lack of casting on the Get. It is easy to put in a value when you mean a pointer or a pointer when you mean a value, but the generic version prevents that problem. The panic in the sample code above is a maybe overkill, but the caller is likely to do something to deal with type. There is a performance impact to the casting, but it doesn't appear to be huge.

In terms of caching performance, there was an improvement in all cases. I tested the original, interface-based implementation as well as a generic implementation of [string, interface{}] to mimic the interface type as closely as possible and a generic implementation of [string, int] to see what the improvement would be. Tests were run on an Apple Macbook Pro M1. An excerpt of the benchmarch is listed below:

                           1% W, 99% R 99% W, 1% R
------------------------- ------------ ------------
interface-based            60.94 ns/op 107.80 ns/op
generic[string,interface]  54.21 ns/op  87.76 ns/op
generic[string,int]        53.24 ns/op  93.80 ns/op