Issue while running benchmarking tool in setting the rpm

[rajm3180]

The rpm value set in the command is not honoured and we get inconsistent rpm from the benchmarking tool using --rate paramater. eg, if we set --rate 10, we get RPM of over 20 as well.

python -m benchmark.bench load --temperature 0.0 --shape-profile custom --deployment gpt-4o-2024-05-13-managed --max-tokens 200 --context-tokens 20000 --api-version 2024-02-01 --rate 10 --duration 600 https://genai-stg-westus3-1.openai.azure.com/

[michaeltremeer]

Hey @rajm3180, can you share a screenshot showing the logs? And is this occurring throughout the test or only early on?