Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

您好，请问这篇论文至少要什么配置，一张80GB的A100可以吗 #1

Open
wmm228 opened this issue Feb 3, 2024 · 1 comment
Open

您好，请问这篇论文至少要什么配置，一张80GB的A100可以吗 #1

wmm228 opened this issue Feb 3, 2024 · 1 comment

Comments

@wmm228
Copy link

wmm228 commented Feb 3, 2024

No description provided.

@Linear95
Copy link
Owner

Linear95 commented Feb 4, 2024

我们实验是在8张40G的A100上跑的，micro_batch_size是1。一张80G的A100应该可以跑，因为micro_batch_size可以开大，你可以试一下micro_batch_size开到4或者8，然后gradient_accumulation对应算一下就好，保证global_batch_size是64就行

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@Linear95 @wmm228