New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

长文本通过切分增大batch size后并行处理，与串行相比总耗时几乎不变 #1029

Open

araloak opened this issue Mar 3, 2025 · 1 comment

araloak commented Mar 3, 2025

我修改了下代码，基于句号对长文本切分以后把多个文本放到一个batch里（扩大了batch_size，原始代码默认为1），但测试发现并行相比串行耗时几乎没有减少。我没太清楚是GPU算子在串行下已达到极限的原因吗？还是其他原因，请问还有什么优化空间吗？

GPU 4070
Windows 10 操作系统
使用300MB-SFT模型

Collaborator

aluminumbox commented Mar 6, 2025

没研究过加速相关，可以再群里问问

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment