Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[examples] better results on wenetspeech using revised transcripts #2371

Merged
merged 2 commits into from
Mar 1, 2024

Conversation

xingchensong
Copy link
Member

@xingchensong xingchensong commented Feb 27, 2024

⚠️quality is important than quantity⚠️

see wenet-e2e/WenetSpeech#54 for more infos

Model Dev Test_Net Test_Meeting blankpenalty
u2pp_conformer (original transcripts) 8.37 % N=328207 C=302993 S=12556 D=12658 I=2253 9.02 % N=414285 C=379271 S=23881 D=11133 I=2365 15.52 % N=220358 C=187746 S=14507 D=18105 I=1596 0.0
- 7.10 % N=328207 C=308457 S=13211 D=6539 I=3537 8.83 % N=414285 C=381933 S=24810 D=7542 I=4215 13.63 % N=220358 C=194446 S=16232 D=9680 I=4132 2.5
u2pp_conformer (revised transcripts) 5.90 % N=328207 C=311721 S=14597 D=1889 I=2888 8.96 % N=414092 C=380232 S=27606 D=6254 I=3222 11.99 % N=220358 C=195808 S=18243 D=6307 I=1878 0.0
zipformer-M 8.58 7.84 14.64 0.0
- 7.24 7.77 12.37 2.5

@xingchensong xingchensong changed the title [examples/wenetspeech] better results on u2pp_conformer using fixed text [examples] better results on wenetspeech using fixed text Feb 28, 2024
@xingchensong
Copy link
Member Author

xingchensong commented Feb 28, 2024

update whisper result:

Model Dev Test_Net Test_Meeting
whisper (original transcripts) 7.27 % N=328207 C=308016 S=11392 D=8799 I=3672 7.90 % N=414097 C=383382 S=18954 D=11761 I=2018 13.00 % N=220358 C=194417 S=11788 D=14153 I=2705
whisper (revised transcripts) 4.66 % N=328207 C=315591 S=10352 D=2264 I=2692 6.54 % N=414092 C=389523 S=19101 D=5468 I=2513 8.84 % N=220358 C=202722 S=11296 D=6340 I=1839
paraformer 3.38 % N=328207 C=319078 S=8045 D=1084 I=1959 6.74 % N=414285 C=388817 S=20119 D=5349 I=2444 6.95 % N=220358 C=206461 S=9090 D=4807 I=1422

As you can see, the overall results are already very close to those of Paraformer. I believe that by adding more open-source data, Whisper can definitely achieve better performance than Paraformer.

@xingchensong xingchensong changed the title [examples] better results on wenetspeech using fixed text [examples] better results on wenetspeech using revised transcripts Feb 29, 2024
@xingchensong xingchensong requested a review from robin1001 March 1, 2024 02:33
@whiteshirt0429 whiteshirt0429 self-requested a review March 1, 2024 03:20
@whiteshirt0429 whiteshirt0429 merged commit ad663fd into main Mar 1, 2024
2 of 4 checks passed
@xingchensong xingchensong deleted the xcsong-wenetspech branch March 1, 2024 03:21
@Ryuk17
Copy link

Ryuk17 commented Mar 6, 2024

Any plan to release this trained model?

@xingchensong
Copy link
Member Author

Any plan to release this trained model?

目前效果还没全面超越paraformer,没有release的意义,另外听闻阿里有意发布更好的多语种模型,后续在那个更强的base模型上去调可能会更好

@xingchensong
Copy link
Member Author

xingchensong commented Mar 6, 2024

@Ryuk17 另外,现在在wenet训练200M级别的模型已经很高效了,8卡3090,使用deepspeed 3~4天就能跑出来上面的u2pp_conformer结果

@Ryuk17
Copy link

Ryuk17 commented Mar 6, 2024

了解,谢谢

@xingchensong
Copy link
Member Author

Any plan to release this trained model?

传了一个whipser的

链接:https://pan.baidu.com/s/1R1mJyqc_MwKZy6Vn32Z3zA?pwd=171n
提取码:171n
--来自百度网盘超级会员V6的分享

@wwfcnu
Copy link

wwfcnu commented May 24, 2024

Any plan to release this trained model?

传了一个whipser的

链接:https://pan.baidu.com/s/1R1mJyqc_MwKZy6Vn32Z3zA?pwd=171n 提取码:171n --来自百度网盘超级会员V6的分享

这个微调后的模型能重新转回到openai style的ckpt吗

@Zth9730
Copy link
Contributor

Zth9730 commented May 24, 2024

Any plan to release this trained model?

传了一个whipser的
链接:https://pan.baidu.com/s/1R1mJyqc_MwKZy6Vn32Z3zA?pwd=171n 提取码:171n --来自百度网盘超级会员V6的分享

这个微调后的模型能重新转回到openai style的ckpt吗

https://github.com/wenet-e2e/wenet/blob/main/wenet/whisper/convert_whisper_to_wenet_config_and_ckpt.py#L199 参考这里是可以转回去的

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants