Skip to content

[New Model] Support BertForTokenClassification / Named Entity Recognition (NER) task #24872

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 17 commits into from
Sep 18, 2025
Merged

[New Model] Support BertForTokenClassification / Named Entity Recognition (NER) task #24872

merged 17 commits into from
Sep 18, 2025
+257 −2

Conversation

@noooop
Copy link
Collaborator

@noooop noooop commented Sep 15, 2025 •
edited by github-actions bot
Loading

Purpose

Support BertForTokenClassification / Named Entity Recognition (NER) task

fix bert + Flex Attention + torch.comple by @Isotr0py
(see: #24872 (comment)

Fix #24752
Fix #25060

Test Plan

tests/models/language/pooling/test_token_classification.py

Test Result

pass

Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
  • (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

@mergify mergify bot added documentation Improvements or additions to documentation new-model Requests to new models labels Sep 15, 2025
@noooop
+ ner
d0078ec 
Signed-off-by: wang.yuqi <noooop@126.com>
@noooop
+ ner online
ed3dba4 
Signed-off-by: wang.yuqi <noooop@126.com>
@noooop
Copy link
Collaborator Author

noooop commented Sep 15, 2025 •
edited
Loading

@mlinmg

try: examples/online_serving/ner.py

But it's definitely not very user-friendly

Do you ok with that. or We still need to further optimize the API.

@mlinmg
Copy link
Contributor

mlinmg commented Sep 15, 2025

That's great! The implementation really depends on the popularity of the token classification API. For now, a basic implementation should be more than enough imo. If more people start using it, it might be worthwhile to create a complete end-to-end flow.

@mergify mergify bot added the frontend label Sep 16, 2025
@noooop
Polishing
c6d4cb7 
Signed-off-by: wang.yuqi <noooop@126.com>
@noooop
Polishing
5a5ef91 
Signed-off-by: wang.yuqi <noooop@126.com>
@noooop
fix
dd1d36f 
Signed-off-by: wang.yuqi <noooop@126.com>
noooop
noooop commented Sep 16, 2025
View reviewed changes
@noooop
fix
9775e5d 
Signed-off-by: wang.yuqi <noooop@126.com>
@noooop
fix
cda6dea 
Signed-off-by: wang.yuqi <noooop@126.com>
@noooop
fix
4e79662 
Signed-off-by: wang.yuqi <noooop@126.com>
@noooop
fix
0a61127 
Signed-off-by: wang.yuqi <noooop@126.com>
@noooop noooop marked this pull request as ready for review September 16, 2025 06:41
@noooop
Copy link
Collaborator Author

noooop commented Sep 16, 2025

@DarkLight1337

Ready for review

DarkLight1337
DarkLight1337 reviewed Sep 16, 2025
View reviewed changes
DarkLight1337
DarkLight1337 reviewed Sep 16, 2025
View reviewed changes
@noooop
fix
690aca0 
Signed-off-by: wang.yuqi <noooop@126.com>
@noooop noooop mentioned this pull request Sep 16, 2025
Merged
5 tasks
@Isotr0py
fix flex attn compile
3d86837 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
@mergify mergify bot added the v1 label Sep 16, 2025
@DarkLight1337 DarkLight1337 mentioned this pull request Sep 17, 2025
Closed
1 task
@noooop
Copy link
Collaborator Author

noooop commented Sep 18, 2025

@DarkLight1337
Could you please review this pr.

DarkLight1337
DarkLight1337 approved these changes Sep 18, 2025
View reviewed changes
Copy link
Member

@DarkLight1337 DarkLight1337 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM if the test can pass

@DarkLight1337 DarkLight1337 added the ready ONLY add when PR is ready to merge/full CI is needed label Sep 18, 2025
@noooop
Copy link
Collaborator Author

noooop commented Sep 18, 2025 •
edited
Loading

@DarkLight1337

You didn't using auto-merge..

@DarkLight1337 DarkLight1337 merged commit 5f696c3 into vllm-project:main Sep 18, 2025
66 checks passed
@noooop noooop deleted the fix_bert_seq_cls branch September 18, 2025 15:26
debroy-rh pushed a commit to debroy-rh/vllm that referenced this pull request Sep 19, 2025
@noooop @Isotr0py @debroy-rh
[New Model] Support BertForTokenClassification / Named Entity Recogni…
8ce7e1f 
…tion (NER) task (vllm-project#24872)

Signed-off-by: wang.yuqi <noooop@126.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
@noooop noooop mentioned this pull request Sep 22, 2025
Merged
5 tasks
FeiDaLI pushed a commit to FeiDaLI/vllm that referenced this pull request Sep 25, 2025
@noooop @Isotr0py @FeiDaLI
[New Model] Support BertForTokenClassification / Named Entity Recogni…
364d46b 
…tion (NER) task (vllm-project#24872)

Signed-off-by: wang.yuqi <noooop@126.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
charlifu pushed a commit to ROCm/vllm that referenced this pull request Sep 25, 2025
@noooop @Isotr0py @charlifu
[New Model] Support BertForTokenClassification / Named Entity Recogni…
aebda1b 
…tion (NER) task (vllm-project#24872)

Signed-off-by: wang.yuqi <noooop@126.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: charlifu <charlifu@amd.com>
@DarkLight1337 DarkLight1337 mentioned this pull request Sep 29, 2025
Open
1 task
@antrec antrec mentioned this pull request Oct 7, 2025
Merged
5 tasks
xuebwang-amd pushed a commit to xuebwang-amd/vllm that referenced this pull request Oct 10, 2025
@noooop @Isotr0py @xuebwang-amd
[New Model] Support BertForTokenClassification / Named Entity Recogni…
a4dc4c4 
…tion (NER) task (vllm-project#24872)

Signed-off-by: wang.yuqi <noooop@126.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: xuebwang-amd <xuebwang@amd.com>
choprahetarth pushed a commit to Tandemn-Labs/vllm that referenced this pull request Oct 11, 2025
@noooop @Isotr0py @choprahetarth
[New Model] Support BertForTokenClassification / Named Entity Recogni…
323a34b 
…tion (NER) task (vllm-project#24872)

Signed-off-by: wang.yuqi <noooop@126.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
lywa1998 pushed a commit to lywa1998/vllm that referenced this pull request Oct 20, 2025
@noooop @Isotr0py @lywa1998
[New Model] Support BertForTokenClassification / Named Entity Recogni…
37461a4 
…tion (NER) task (vllm-project#24872)

Signed-off-by: wang.yuqi <noooop@126.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
xuebwang-amd pushed a commit to xuebwang-amd/vllm that referenced this pull request Oct 24, 2025
@noooop @Isotr0py @xuebwang-amd
[New Model] Support BertForTokenClassification / Named Entity Recogni…
da9a745 
…tion (NER) task (vllm-project#24872)

Signed-off-by: wang.yuqi <noooop@126.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: xuebwang-amd <xuebwang@amd.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Isotr0py Isotr0py Isotr0py left review comments

@DarkLight1337 DarkLight1337 DarkLight1337 approved these changes

@hmellor hmellor Awaiting requested review from hmellor

@ywang96 ywang96 Awaiting requested review from ywang96 ywang96 is a code owner

@aarnphm aarnphm Awaiting requested review from aarnphm aarnphm is a code owner

@chaunceyjiang chaunceyjiang Awaiting requested review from chaunceyjiang chaunceyjiang is a code owner

@WoosukKwon WoosukKwon Awaiting requested review from WoosukKwon

@robertgshaw2-redhat robertgshaw2-redhat Awaiting requested review from robertgshaw2-redhat

@njhill njhill Awaiting requested review from njhill

@comaniac comaniac Awaiting requested review from comaniac

@alexm-redhat alexm-redhat Awaiting requested review from alexm-redhat

Assignees

No one assigned

Labels

documentation Improvements or additions to documentation frontend new-model Requests to new models ready ONLY add when PR is ready to merge/full CI is needed v1

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[Bug]: Does not run embedding model sergeyzh/rubert-tiny-turbo [Bug]: Some Bert models no longer compatible for BertForSequenceClassification

4 participants

@noooop @mlinmg @Isotr0py @DarkLight1337