notifications-controller memory usage

[rrey]

Hello,

I have the notifications-controller pods that slowly keeps on consuming memory.
~3-5 MB per day.

That's a linear increase that ultimately leads to getting close to the container allowed limits.

Is the controller keeping refs in memory when some notifications fails that could explain that behaviour ?
I would expect the memory to increase when a lot of notifications are sent and go down when the spike ends, but here we only see increase in the metrics and memory never goes back to a stable value.

Any hint ?