Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[v0.2.3] Release Tracker #1856

Closed
5 tasks done
WoosukKwon opened this issue Nov 30, 2023 · 6 comments
Closed
5 tasks done

[v0.2.3] Release Tracker #1856

WoosukKwon opened this issue Nov 30, 2023 · 6 comments
Labels
release Related to new version release

Comments

@WoosukKwon
Copy link
Collaborator

WoosukKwon commented Nov 30, 2023 •
edited
Loading

ETA: Nov 30th - Dec 2nd.

Major changes

  • Refactoring on Worker, InputMetadata, and Attention
  • Fix TP support for AWQ models
  • Support Prometheus metrics
  • Fix Baichuan & Baichuan 2

PRs to be merged before the release
@WoosukKwon WoosukKwon added the release Related to new version release label Nov 30, 2023
@WoosukKwon WoosukKwon pinned this issue Nov 30, 2023
@simon-mo
Copy link
Collaborator

I think we should absolutely include #1756 (and it's newer iteration). I'm also debugging #1707.

@simon-mo
Copy link
Collaborator

I will also add #1662

@WoosukKwon
Copy link
Collaborator Author

@simon-mo If I understand correctly, AWQ performance turns out to be orthogonal to the performance issue in #1707, is this right?

@simon-mo
Copy link
Collaborator

simon-mo commented Dec 2, 2023

AWQ's bad prefill performance exacerbated the issue. The root issue was non-chunked prefill.

@simon-mo
Copy link
Collaborator

simon-mo commented Dec 2, 2023

@WoosukKwon for the prometheus metrics, I made a new PR, please take a look.

@Tostino Tostino mentioned this issue Dec 2, 2023
Merged
@WoosukKwon
Copy link
Collaborator Author

Closed by #1903

@WoosukKwon WoosukKwon unpinned this issue Dec 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
release Related to new version release
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@simon-mo @WoosukKwon