Infinite loop in rate limit middleware

[jlucktay]

Branch/Environment/Version

• Branch/Version: tag v2.8.5
• Environment: on-premise, though likely to affect others also

Describe the bug
In certain circumstances, requests passing through the rate limit/quota check middleware may fall into an infinite loop, due to a recursive function call that is not properly guarded and as such does not have an adequate exit condition.

Reproduction steps
TBD

Actual behavior
Some requests entering the gateway get stuck on an infinite/recursive loop, never get past the rate limit/quota check middleware, and cause a gradual increase in CPU usage and allocated memory.

Expected behavior
All requests entering the gateway are limit-/quota-checked and throttled/retried/completed appropriately.

Screenshots/Video
pprof output from the inuse_space sample of two heap profiles of an affected gateway are below.

Profile 1 is still receiving new traffic and Profile 2 is after the point where new traffic is no longer entering the gateway.

Corresponding increases in CPU time spent were also identified on the CPU profile in pprof (not shown).

CPU usage of the instance running this affected gateway, in contrast to some other healthy instances/gateways:

Configuration (tyk config file):
TBD