[Feature]: Benchmarks for audio models

[DarkLight1337]

🚀 The feature, motivation and pitch

• Add audio datasets to benchmarks/benchmark_dataset.py to so we can run performance benchmarks on audio models as well.
• Add a benchmark similar to MMMU ([DO NOT MERGE] VLM offline benchmark with MMMU-Pro vision #11196) but for audio models to evaluate their correctness.

Alternatives

No response

Additional context

cc @mgoin @ywang96

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[p88h]

FWIW, i'm trying to add VLLM support to the ASR leaderboard here: huggingface/open_asr_leaderboard#50

It's going slowly, but might be some starting point for more extensive benchmarking in VLLM.

[NickLucche]

On second thought, I think I might look into this issue as I've already done something similar to evaluate whisper for the transcription api.