Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

生成说话人视频问题 #51

Closed
Free-D opened this issue Mar 10, 2023 · 3 comments
Closed

生成说话人视频问题 #51

Free-D opened this issue Mar 10, 2023 · 3 comments

Comments

@Free-D
Copy link

Free-D commented Mar 10, 2023

你好,我根据流程到最后生成视频,生成的视频中没有人形,我检查了/infer_out/May/pred_video/tmp_imgs/zozo_infer_torso_nerf下的所有图片,参考图一
后面我参考这个issue检查了checkpoints/May/lm3d_nerf_torso/validation_results/validation_400000_/imgs/frame_9181.jpg,参考图二
然后我又检查了同路径下的图三frame_9181_gt.jpg
请问这是哪个步骤出现了问题?
图一:
00000
图二:
frame_9181
图三:
frame_9181_gt

@yerfor
Copy link
Owner

yerfor commented Mar 10, 2023

你好,看起来似乎在训练head_nerf的阶段,就没有能够成功收敛。你可能需要重新训练head nerf。因为原始nerf非常依赖于初始化,你可能要执行数次训练命令行,直到观察到loss能在1000steps内能够正常下降到0.001左右,才能够正常训练。开始正常训练后,你会看到/validation_results/文件夹里生成的图片是有正常的脑袋的。训练torso nerf的技巧也是类似。

我承认目前基于AD-NeRF框架的渲染器训练比较麻烦(需要初始化数次),我计划在近期释出基于RAD-NeRF的渲染器,那个更新将完全解决这个问题。

@Free-D
Copy link
Author

Free-D commented Mar 10, 2023

好的,感谢你的回复,我会等更新之后再重新训练,感谢开源项目!

@Free-D Free-D closed this as completed Mar 10, 2023
@yerfor
Copy link
Owner

yerfor commented Mar 10, 2023

好的,我会尽快更新

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants