Increase poller history LRU cache size

[mantas-sidlauskas]

What changed?
LRU poller cache size is increased from 1000 to 5000

Why?
Task list poller count metric is representing entries in this cache. When there are more than 1000 workers, this metric becomes misleading.

How did you test it?
Unit tests

Potential risks

[meiliang86]

Unrelated to this diff, why do we pass in historyUpdatedFunc? IMO we should just pass in metrics scope and emit it inside the cache.
Also, why not calling the cache.Size() method directly to get the size? Calling iterator to just get the size does not sound efficient.

[coveralls]

Coverage decreased (-0.08%) to 66.634% when pulling b8968f6 on mantas-sidlauskas:history_size into f2f10d4 on uber:master.

[meiliang86]

So this is per tasklist right? 5000 should be a sufficient large value to use.

[mantas-sidlauskas]

Yes, this is per tasklist. 5000 should be enough for most cases, if we will see 5000 pollers per tasklist, it might be a good indicator to turn on tasklist sharding

[mantas-sidlauskas]

Unrelated to this diff, why do we pass in historyUpdatedFunc? IMO we should just pass in metrics scope and emit it inside the cache.
Also, why not calling the cache.Size() method directly to get the size? Calling iterator to just get the size does not sound efficient.

Yes, this sounds reasonable. Unfortunately current LRU cache implementation evicts outdated entries only when iterating over items (i.e. get list of pollers)

This can be addressed in a separate PR