Memory consumption increases heavily over time

[constcast]

When VERMONT keeps running in packet processing mode, memory consumption will significantly increase over time.

Probable causes:

• The InstanceManager will allocate new Packets at peak times which will never be freed (check by disabling the Manager)
• FlowHashtable filling up

[nickbroon]

I wonder if these patches address this issue?

felixe@328f38c
felixe@b793f2b
felixe@6c1c56c
felixe@e5fdf1d

Seen as a sequence here: https://github.com/felixe/ccsVermont/commits/http-aggregation?after=074b80f778f4a5f4b7707778195529190188903a+69

Perhaps @felixe can comment.

[felixe]

Hi Nick,
They definitely do address a memory issue in high speed scenarios.
Vermont originally preallocated a static amount of packet instances and additional instances were added when needed. The problem was that these instances were not freed anymore.
With our changes the preallocated amount of packet instances is configurable (they will never bee freed), and additionally added instances are freed again after use.
The idea is that now the memory usage will always shrink back to the initial size (of preallocated packets instances).
We found that, especially in high speed scenarios, it is advantageous to allocated a higher number of instances at startup, so no allocation operations have to be done during peak situations.

hope that helps,
best

felix