Release v0.5.5

[simon-mo]

We will make a release later this week or early next week (Aug 16-Aug19) to address Gemma logits soft-caps bug, openai server metrics bug, and include more performance enhancements.

Please add blockers if needed.

[robertgshaw2-neuralmagic]

Will focus on getting these over the line tomorrow and thursday:

• [ Bugfix ] Fix Prometheus Metrics With zeromq Frontend #7279
• [Bugfix][Frontend] Fix Issues Under High Load With zeromq Frontend #7394

[njhill]

• [Misc] Deprecation Warning when setting --engine-use-ray #7424

[simon-mo]

• [Bugfix][Docs] Update list of mock imports #7493

[robertgshaw2-neuralmagic]

• [Bug][Frontend] Improve ZMQ client robustness #7443

[simon-mo]

• [Kernel] W8A16 Int8 inside FusedMoE  #7415

[mgoin]

• [Bugfix] Fix default weight loading for scalars #7534

[WoosukKwon]

• [Misc] Remove Gemma RoPE #7638
• [Misc] Use torch.compile for GemmaRMSNorm #7642

[youkaichao]

• [doc] fix doc build error caused by msgspec #7659
• [Ray backend] Better error when pg topology is bad.  #7584

[njhill]

• [BugFix] Avoid premature async generator exit and raise all exception variations #7698

[robertgshaw2-neuralmagic]

Not required but nice + easy:

• [Frontend] Improve Startup Failure UX #7716

[njhill]

• [BugFix] Fix server crash on empty prompt #7746 (DoS issue)

[sindhuvahinis]

• A fix for phi-v3.5 model [Model] Fix Phi-3.5-vision-instruct 'num_crops' issue #7710

[Jimmy-Newtron]

... a release later this week or early next week (Aug 16-Aug19) ...

When do you plan the release now?

[simon-mo]

Now