-
Notifications
You must be signed in to change notification settings - Fork 7.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
训练检测模型时出现以下错误 #84
Comments
问下,你是在自己的数据集上训练的吗,还是用的readme中提到的数据集。方便的话,提供下
初步怀疑是数据读取有问题 |
训练模型很小了,轻量的模型只有4M多,你的GPU有多少内存,用的是哪一个算法 |
watch nvidia-smi 看一下GPU内存使用情况,如果有其他程序占用了内存,但是GPU没有利用率,可以kill掉 |
这个程序没有利用率,可能是python程序非正常关闭,但是进程依然存在,导致一直再占着显存 |
那试试减小batchsize |
减少test_batch_size_per_card,train_batch_size_per_card可以解决问题,非常感谢 |
好的 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
The text was updated successfully, but these errors were encountered: