Addressing the cold start issue with benchmarks

[HamidShojanazeri]

Issue

Recently, we are experiencing the cold start issue with many of running benchmarks where a jump in model p99 has been observed. So far, we have tried to address/ smooth it out by increasing the number of requests.

Request

We need to modify the benchmark routines to separate warmup from actual runs that are benchmarked.