FAQ_Timing

FAQ about timing and timeouts:

This FAQ about timing and timeouts will try to answer the most common questions about timing and timeouts.

Background

Threads:

When acquiring memory from a FPGA hardware device only one thread may access the device at the very same time. Additional threads will be queued and will resume when the thread locking the FPGA is finished. This behavior may lead to short spikes in latency in multi-threaded environments.

Internal Refresh:

MemProcFS, by default, performs various refreshes internally which requires reading from the FPGA. If refreshes are disabled by the user new processes are no longer be picked up, new memory allocations inside existing processes would be missing and so on. The problem would grow worse over time if the internal refresh is disabled. When an internal refresh is in progress DMA reads may be delayed and latency spikes would be seen due to the Threads point above.

Default Refresh Settings:

The default refresh settings are set to execute on three different time intervals.

Refresh Type	Refresh Interval	Actions taken
Memory	0.3 seconds	1/3 of internal memory cache is dropped (instant operation). Memory will be re-read and inserted anew into the cache at first access.
TLB	2 seconds	1/3 of internal page table translation cache is dropped (instant operation). Page tables will be re-read and inserted anew into the cache at first access.
FAST	5 seconds	Process list is refreshed with new processes (small timing spike).
MEDIUM	15 seconds	Process list is totally refreshed (noticeable timing spike). VADs, Network connections, Objects, Heaps etc. are cleared (instant) but on first access they will take a short while to re-generate (causing noticeable timing spikes).
SLOW	5 minutes	Same as MEDIUM. Also Registry information, Services, Pool, Memory Map etc. are cleared (instant) but on first access they will take a longer time to re-generate (causing very noticeable timing spikes)

User options

It's possible for the user to customize the background refresh behavior with custom intervals.

It's possible to completely disable the background refresh with the -norefresh initialization command line option. If the refresh is completely disabled it is highly recommended to do manual refreshes at healthy time intervals to avoid memory drift, lost memory allocations and processes if running MemProcFS for a longer time.

API: Modifying default refresh times

It's possible to modify the default refresh times given in the above table and still make use of the internal refresh engine. This requires that the internal refreshes have not been disabled by the user at initialization time with the -norefresh command line option.

The following C-API constants may be used together with the VMMDLL_ConfigGet() and VMMDLL_ConfigSet() functions.

#define VMMDLL_OPT_CONFIG_IS_REFRESH_ENABLED      0x2000000300000000 // R - 1/0
#define VMMDLL_OPT_CONFIG_TICK_PERIOD             0x2000000400000000 // RW - base tick period in ms
#define VMMDLL_OPT_CONFIG_READCACHE_TICKS         0x2000000500000000 // RW - memory cache validity period (in ticks)
#define VMMDLL_OPT_CONFIG_TLBCACHE_TICKS          0x2000000600000000 // RW - page table (tlb) cache validity period (in ticks)
#define VMMDLL_OPT_CONFIG_PROCCACHE_TICKS_PARTIAL 0x2000000700000000 // RW - process refresh (partial) period (in ticks)
#define VMMDLL_OPT_CONFIG_PROCCACHE_TICKS_TOTAL   0x2000000800000000 // RW - process refresh (full) period (in ticks)

The corresponding functionality exists in the other supported programming languages as well.

First the user will have to set a "tick period" in milliseconds. The other periods for memory, tlb (page tables), fast (VMMDLL_OPT_CONFIG_PROCCACHE_TICKS_PARTIAL) and medium (VMMDLL_OPT_CONFIG_PROCCACHE_TICKS_TOTAL) are given in multiples of ticks.

Example: Double the base tick period from the default 100ms to 200ms, effectively doubling the interval between all refreshes.

VMMDLL_ConfigSet(hVMM, VMMDLL_OPT_CONFIG_TICK_PERIOD, 200);

API: Manually refreshing

If the internal refresh sub-system have been disabled at start-up by the user by providing the -norefresh initialization command line option it is highly recommended to perform manual refreshes if running for an extended period of time. Not doing this will lead to degraded memory analysis result as time passes by.

The following API constants allows the user to perform manual refreshes with the VMMDLL_ConfigGet() and VMMDLL_ConfigSet() functions.

#define VMMDLL_OPT_REFRESH_ALL              0x2001ffff00000000 // W - refresh all caches
#define VMMDLL_OPT_REFRESH_FREQ_MEM         0x2001100000000000 // W - refresh memory cache (excl. TLB) [fully]
#define VMMDLL_OPT_REFRESH_FREQ_MEM_PARTIAL 0x2001000200000000 // W - refresh memory cache (excl. TLB) [partial 33%/call]
#define VMMDLL_OPT_REFRESH_FREQ_TLB         0x2001080000000000 // W - refresh page table (TLB) cache [fully]
#define VMMDLL_OPT_REFRESH_FREQ_TLB_PARTIAL 0x2001000400000000 // W - refresh page table (TLB) cache [partial 33%/call]
#define VMMDLL_OPT_REFRESH_FREQ_FAST        0x2001040000000000 // W - refresh fast frequency - incl. partial process refresh
#define VMMDLL_OPT_REFRESH_FREQ_MEDIUM      0x2001000100000000 // W - refresh medium frequency - incl. full process refresh
#define VMMDLL_OPT_REFRESH_FREQ_SLOW        0x2001001000000000 // W - refresh slow frequency.

Corresponding functionality exists in all MemProcFS supported programming languages. Also note that even though the API constant says FREQ this has nothing to do with setting a frequency or interval.

Example: Refresh everything. This may have a considerable latency impact.

VMMDLL_ConfigSet(hVMM, VMMDLL_OPT_REFRESH_ALL, 1);

Example: Refresh MEDIUM (total process list refresh + some additional refreshes). This may have some latency impact.

VMMDLL_ConfigSet(hVMM, VMMDLL_OPT_REFRESH_FREQ_MEDIUM, 1);

Example: Flush all internal memory caches and memory translation caches. This will be instant, but there may be some impact on first use before the internal cache is re-populated again.

VMMDLL_ConfigSet(hVMM, VMMDLL_OPT_REFRESH_FREQ_MEM, 1);
VMMDLL_ConfigSet(hVMM, VMMDLL_OPT_REFRESH_FREQ_TLB, 1);

API: Manually refreshing a process fast

MemProcFS does not allow for refreshing an individual process only. This is not a problem since a refresh will mostly discard any cached data and then data will be re-read and re-generated on a first-use basis. If the other processes are never accessed they won't incur a performance penalty.

The fastest would be to discard the contents of cached physical memory and page tables.

Example: Refresh only memory caches

VMMDLL_ConfigSet(hVMM, VMMDLL_OPT_REFRESH_FREQ_MEM, 1);
VMMDLL_ConfigSet(hVMM, VMMDLL_OPT_REFRESH_FREQ_TLB, 1);

Refreshing only memory caches may not always be sufficient. Sometimes the process itself must be refreshed as well. Especially if there are new memory allocations related to the Virtual Address Descriptors (VADs), newly loaded modules, handles etc. The actual refresh would be fairly fast, but if the number of VADs are large they may take a substantial time to re-generate on first use.

Example: Refresh process and memory caches

VMMDLL_ConfigSet(hVMM, VMMDLL_OPT_REFRESH_FREQ_MEM, 1);
VMMDLL_ConfigSet(hVMM, VMMDLL_OPT_REFRESH_FREQ_TLB, 1);
VMMDLL_ConfigSet(hVMM, VMMDLL_OPT_REFRESH_FREQ_MEDIUM, 1);