Replace time.Now with lower precision time source

[bartle-stripe]

time.Now() will sometimes result in a full system call (this is the default on AWS EC2 instances). I think for an LRU you can probably get away with second precision. One option would be to have a background goroutine that called time.Now() every second and cached the result globally.

[coocood]

Do you know any cache library have this optimization?

[pheepi]

I have proposed to replace wall time time.Now() by a custom timer in #69. Then you could create you own custom time structure with field seconds and update it periodically with a Go timer.

[bartle-stripe]

Seems reasonable to me.

[thesilentg]

@pheepi @bartle-stripe Did you implement the background goroutine time.Now() timer? If so, would you be willing to share the code here?

@coocood For reference, this is what the pprof data looks like when trying to set tens of millions of entries in an AWS Fargate task. Roughly 60% of the CPU for setting items is being spent in time.now()

[bartle-stripe]

I don't believe I ever did.

[thesilentg]

This resulted in ~50% speedup for calls to .Set()

type lowResClock struct {
	now uint32
}

func newLosResClock() lowResClock {
	clock := lowResClock{
		now: uint32(time.Now().Unix()),
	}
	ticker := time.NewTicker(1 * time.Second)
	go func() {
		for {
			select {
			case _ = <-ticker.C:
				clock.now = uint32(time.Now().Unix())
			}
		}
	}()
	return clock
}

func (clock lowResClock) Now() uint32 {
	return clock.now
}

[thesilentg]

@coocood Do you think the performance improvement from this is worth replacing the existing default implementation?

[coocood]

@thesilentg

The lowResClock starts a goroutine and never exit, if cache are created and discarded too many times, there would be a lot of goroutines leaked.

And clock.now needs atomic access to be race-free.

We could create a well-implemented builtin lowResClock, but I think the default should still be time.Now()

[pheepi]

Optimized timer with cached value is now part of the code passed as a custom parameter of NewCacheCustomTimer in #88. Create new stoppable timer with NewCachedTimer and call Stop() method when your cache is no more in use. The method stops the background routine that periodically updates the time.