Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Accelerate the training process #6

Open
JZArray opened this issue May 24, 2024 · 5 comments
Open

Accelerate the training process #6

JZArray opened this issue May 24, 2024 · 5 comments

Comments

@JZArray
Copy link

JZArray commented May 24, 2024

Hello guys, a quick question, when increasing the number of IDs, losses converge slowly , comparing with the a small dataset(contains 4/50 Ids), how can I boost the training speed?
@flyingshan
Copy link

flyingshan commented May 27, 2024 •
edited
Loading

I am also trying to reimplement the paper, can you help me with some of my questions? Thank you very much!

  1. What is the resolution of the 3d warping flow ws->/w->d? According to the paper, if the input of W* is a 1-d vector, the resolution of the output ws->/w->d will be 16x16x16, which I think is too small to model the facial mimics.
  2. How to set the parameter "group num" of the GroupNorm layer? Does it make any difference?
  3. What is the Adaptive GroupNorm mentioned in the paper, which implementation do you adopt?
  4. The jump layer in G3d seems buggy(resolutions don't match), how do you solve this problem?

@flyingshan
Copy link

BTW, may I ask how many GPUs do you use to train this, and how much time does it take for you to converge in large ids?

@johndpope
Copy link

in the follow up paper EmoPortraits that is going to be released in July with models / and new video dataset - EmoPortraits - https://github.com/neeek2303/EMOPortraits they say the drop the features from 512 -> 128. it may help - though - we need more compute.

WRT -1 - I think we can assume it's 64x64

WRT 2.
https://github.com/johndpope/MegaPortrait-hack/blob/main/model.py#L234

  1. WRT 3
    https://github.com/johndpope/MegaPortrait-hack/blob/main/model.py#L269

  2. https://github.com/johndpope/MegaPortrait-hack/blob/main/model.py#L536 (haven't started training the upres model )
    I think we just cherry pick this https://github.com/Meta-Portrait/MetaPortrait/tree/main/sr_model for super resolution.

https://github.com/johndpope/MegaPortrait-hack/blob/main/model.py#L866

I put in a debug flag in models.py - that will show all the dimensions
I also comment the code with ambiguities using this emoji 🤷

2024-05-27 16:47:49,730 - DEBUG - image x: torch.Size([1, 3, 256, 256])
2024-05-27 16:47:49,846 - DEBUG - After conv: torch.Size([1, 64, 256, 256])
2024-05-27 16:47:49,846 - DEBUG - ResBlock_Custom > x.shape:  torch.Size([1, 64, 256, 256])
2024-05-27 16:47:49,877 - DEBUG - After resblock_128: torch.Size([1, 128, 256, 256])
2024-05-27 16:47:49,878 - DEBUG - After avgpool: torch.Size([1, 128, 128, 128])
2024-05-27 16:47:49,878 - DEBUG - ResBlock_Custom > x.shape:  torch.Size([1, 128, 128, 128])
2024-05-27 16:47:49,885 - DEBUG - After resblock_256: torch.Size([1, 256, 128, 128])
2024-05-27 16:47:49,885 - DEBUG - After avgpool: torch.Size([1, 256, 64, 64])
2024-05-27 16:47:49,885 - DEBUG - ResBlock_Custom > x.shape:  torch.Size([1, 256, 64, 64])
2024-05-27 16:47:49,889 - DEBUG - After resblock_512: torch.Size([1, 512, 64, 64])
2024-05-27 16:47:49,891 - DEBUG - After conv_1: torch.Size([1, 1536, 64, 64])
2024-05-27 16:47:49,891 - DEBUG - reshape 1546 -> C96 x D16 : torch.Size([1, 96, 16, 64, 64])
2024-05-27 16:47:49,891 - DEBUG -    🍒 ResBlock3D x.shape:torch.Size([1, 96, 16, 64, 64])
2024-05-27 16:47:49,893 - DEBUG -    conv1 > out.shape:torch.Size([1, 96, 16, 64, 64])
2024-05-27 16:47:49,895 - DEBUG -    norm1 > out.shape:torch.Size([1, 96, 16, 64, 64])
2024-05-27 16:47:49,895 - DEBUG -    F.relu(out) > out.shape:torch.Size([1, 96, 16, 64, 64])
2024-05-27 16:47:49,895 - DEBUG -    conv2 > out.shape:torch.Size([1, 96, 16, 64, 64])
2024-05-27 16:47:49,895 - DEBUG -    norm2 > out.shape:torch.Size([1, 96, 16, 64, 64])
2024-05-27 16:47:49,895 - DEBUG -    residual > residual.shape:torch.Size([1, 96, 16, 64, 64])
2024-05-27 16:47:49,895 - DEBUG - After resblock3D_96: torch.Size([1, 96, 16, 64, 64])
2024-05-27 16:47:49,895 - DEBUG -    🍒 ResBlock3D x.shape:torch.Size([1, 96, 16, 64, 64])
2024-05-27 16:47:49,896 - DEBUG -    conv1 > out.shape:torch.Size([1, 96, 16, 64, 64])
2024-05-27 16:47:49,896 - DEBUG -    norm1 > out.shape:torch.Size([1, 96, 16, 64, 64])
2024-05-27 16:47:49,896 - DEBUG -    F.relu(out) > out.shape:torch.Size([1, 96, 16, 64, 64])
2024-05-27 16:47:49,896 - DEBUG -    conv2 > out.shape:torch.Size([1, 96, 16, 64, 64])
2024-05-27 16:47:49,897 - DEBUG -    norm2 > out.shape:torch.Size([1, 96, 16, 64, 64])
2024-05-27 16:47:49,897 - DEBUG -    residual > residual.shape:torch.Size([1, 96, 16, 64, 64])
2024-05-27 16:47:49,897 - DEBUG - After resblock3D_96_2: torch.Size([1, 96, 16, 64, 64])
2024-05-27 16:47:49,897 - DEBUG -    🍒 ResBlock3D x.shape:torch.Size([1, 96, 16, 64, 64])
2024-05-27 16:47:49,897 - DEBUG -    conv1 > out.shape:torch.Size([1, 96, 16, 64, 64])
2024-05-27 16:47:49,898 - DEBUG -    norm1 > out.shape:torch.Size([1, 96, 16, 64, 64])
2024-05-27 16:47:49,898 - DEBUG -    F.relu(out) > out.shape:torch.Size([1, 96, 16, 64, 64])
2024-05-27 16:47:49,898 - DEBUG -    conv2 > out.shape:torch.Size([1, 96, 16, 64, 64])
2024-05-27 16:47:49,898 - DEBUG -    norm2 > out.shape:torch.Size([1, 96, 16, 64, 64])
2024-05-27 16:47:49,898 - DEBUG -    residual > residual.shape:torch.Size([1, 96, 16, 64, 64])
2024-05-27 16:47:49,898 - DEBUG - After resblock3D_96_1: torch.Size([1, 96, 16, 64, 64])
2024-05-27 16:47:49,898 - DEBUG -    🍒 ResBlock3D x.shape:torch.Size([1, 96, 16, 64, 64])
2024-05-27 16:47:49,899 - DEBUG -    conv1 > out.shape:torch.Size([1, 96, 16, 64, 64])
2024-05-27 16:47:49,899 - DEBUG -    norm1 > out.shape:torch.Size([1, 96, 16, 64, 64])
2024-05-27 16:47:49,899 - DEBUG -    F.relu(out) > out.shape:torch.Size([1, 96, 16, 64, 64])
2024-05-27 16:47:49,899 - DEBUG -    conv2 > out.shape:torch.Size([1, 96, 16, 64, 64])
2024-05-27 16:47:49,900 - DEBUG -    norm2 > out.shape:torch.Size([1, 96, 16, 64, 64])
2024-05-27 16:47:49,900 - DEBUG -    residual > residual.shape:torch.Size([1, 96, 16, 64, 64])
2024-05-27 16:47:49,900 - DEBUG - After resblock3D_96_1_2: torch.Size([1, 96, 16, 64, 64])
2024-05-27 16:47:49,900 - DEBUG -    🍒 ResBlock3D x.shape:torch.Size([1, 96, 16, 64, 64])
2024-05-27 16:47:49,900 - DEBUG -    conv1 > out.shape:torch.Size([1, 96, 16, 64, 64])
2024-05-27 16:47:49,900 - DEBUG -    norm1 > out.shape:torch.Size([1, 96, 16, 64, 64])
2024-05-27 16:47:49,900 - DEBUG -    F.relu(out) > out.shape:torch.Size([1, 96, 16, 64, 64])
2024-05-27 16:47:49,901 - DEBUG -    conv2 > out.shape:torch.Size([1, 96, 16, 64, 64])
2024-05-27 16:47:49,901 - DEBUG -    norm2 > out.shape:torch.Size([1, 96, 16, 64, 64])
2024-05-27 16:47:49,901 - DEBUG -    residual > residual.shape:torch.Size([1, 96, 16, 64, 64])
2024-05-27 16:47:49,901 - DEBUG - After resblock3D_96_2: torch.Size([1, 96, 16, 64, 64])
2024-05-27 16:47:49,901 - DEBUG -    🍒 ResBlock3D x.shape:torch.Size([1, 96, 16, 64, 64])
2024-05-27 16:47:49,901 - DEBUG -    conv1 > out.shape:torch.Size([1, 96, 16, 64, 64])
2024-05-27 16:47:49,902 - DEBUG -    norm1 > out.shape:torch.Size([1, 96, 16, 64, 64])
2024-05-27 16:47:49,902 - DEBUG -    F.relu(out) > out.shape:torch.Size([1, 96, 16, 64, 64])
2024-05-27 16:47:49,902 - DEBUG -    conv2 > out.shape:torch.Size([1, 96, 16, 64, 64])
2024-05-27 16:47:49,902 - DEBUG -    norm2 > out.shape:torch.Size([1, 96, 16, 64, 64])
2024-05-27 16:47:49,902 - DEBUG -    residual > residual.shape:torch.Size([1, 96, 16, 64, 64])
2024-05-27 16:47:49,902 - DEBUG - After resblock3D_96_2_2: torch.Size([1, 96, 16, 64, 64])
2024-05-27 16:47:50,005 - DEBUG - 👤 head_pose shape Should print: torch.Size([1, 6]):torch.Size([1, 6])
2024-05-27 16:47:50,005 - DEBUG - 📐 rotation shape Should print: torch.Size([1, 3]):torch.Size([1, 3])
2024-05-27 16:47:50,005 - DEBUG - 📷 translation shape Should print: torch.Size([1, 3]):torch.Size([1, 3])
2024-05-27 16:47:50,039 - DEBUG -    pitch:tensor([0.1282], device='cuda:0', grad_fn=<SelectBackward0>)
2024-05-27 16:47:50,040 - DEBUG -    yaw:tensor([0.0704], device='cuda:0', grad_fn=<SelectBackward0>)
2024-05-27 16:47:50,041 - DEBUG -    roll:tensor([0.0335], device='cuda:0', grad_fn=<SelectBackward0>)
2024-05-27 16:47:50,054 - DEBUG - 👤 head_pose shape Should print: torch.Size([1, 6]):torch.Size([1, 6])
2024-05-27 16:47:50,054 - DEBUG - 📐 rotation shape Should print: torch.Size([1, 3]):torch.Size([1, 3])
2024-05-27 16:47:50,054 - DEBUG - 📷 translation shape Should print: torch.Size([1, 3]):torch.Size([1, 3])
2024-05-27 16:47:50,056 - DEBUG -    pitch:tensor([0.1268], device='cuda:0', grad_fn=<SelectBackward0>)
2024-05-27 16:47:50,060 - DEBUG -    yaw:tensor([0.0713], device='cuda:0', grad_fn=<SelectBackward0>)
2024-05-27 16:47:50,061 - DEBUG -    roll:tensor([0.0258], device='cuda:0', grad_fn=<SelectBackward0>)
2024-05-27 16:47:50,066 - DEBUG - es shape:torch.Size([1, 512, 4, 4])
2024-05-27 16:47:50,066 - DEBUG - zs shape:torch.Size([1, 512, 4, 4])
2024-05-27 16:47:50,066 - DEBUG - FlowField > zs sum.shape:torch.Size([1, 512, 4, 4])
2024-05-27 16:47:50,068 - DEBUG -       conv1x1 > x.shape:torch.Size([1, 2048, 4, 4])
2024-05-27 16:47:50,068 - DEBUG -       reshape_layer > x.shape:torch.Size([1, 512, 4, 4, 4])
2024-05-27 16:47:50,068 - DEBUG -    🍒 ResBlock3D x.shape:torch.Size([1, 512, 4, 4, 4])
2024-05-27 16:47:50,070 - DEBUG -    conv1 > out.shape:torch.Size([1, 256, 4, 4, 4])
2024-05-27 16:47:50,070 - DEBUG -    norm1 > out.shape:torch.Size([1, 256, 4, 4, 4])
2024-05-27 16:47:50,070 - DEBUG -    F.relu(out) > out.shape:torch.Size([1, 256, 4, 4, 4])
2024-05-27 16:47:50,071 - DEBUG -    conv2 > out.shape:torch.Size([1, 256, 4, 4, 4])
2024-05-27 16:47:50,072 - DEBUG -    norm2 > out.shape:torch.Size([1, 256, 4, 4, 4])
2024-05-27 16:47:50,073 - DEBUG -    residual > residual.shape:torch.Size([1, 256, 4, 4, 4])
2024-05-27 16:47:50,075 - DEBUG -       upsample1 > x.shape:torch.Size([1, 256, 8, 8, 8])
2024-05-27 16:47:50,075 - DEBUG -    🍒 ResBlock3D x.shape:torch.Size([1, 256, 8, 8, 8])
2024-05-27 16:47:50,076 - DEBUG -    conv1 > out.shape:torch.Size([1, 128, 8, 8, 8])
2024-05-27 16:47:50,077 - DEBUG -    norm1 > out.shape:torch.Size([1, 128, 8, 8, 8])
2024-05-27 16:47:50,077 - DEBUG -    F.relu(out) > out.shape:torch.Size([1, 128, 8, 8, 8])
2024-05-27 16:47:50,078 - DEBUG -    conv2 > out.shape:torch.Size([1, 128, 8, 8, 8])
2024-05-27 16:47:50,079 - DEBUG -    norm2 > out.shape:torch.Size([1, 128, 8, 8, 8])
2024-05-27 16:47:50,080 - DEBUG -    residual > residual.shape:torch.Size([1, 128, 8, 8, 8])
2024-05-27 16:47:50,081 - DEBUG -       upsample2 > x.shape:torch.Size([1, 128, 16, 16, 16])
2024-05-27 16:47:50,081 - DEBUG -    🍒 ResBlock3D x.shape:torch.Size([1, 128, 16, 16, 16])
2024-05-27 16:47:50,082 - DEBUG -    conv1 > out.shape:torch.Size([1, 64, 16, 16, 16])
2024-05-27 16:47:50,083 - DEBUG -    norm1 > out.shape:torch.Size([1, 64, 16, 16, 16])
2024-05-27 16:47:50,083 - DEBUG -    F.relu(out) > out.shape:torch.Size([1, 64, 16, 16, 16])
2024-05-27 16:47:50,084 - DEBUG -    conv2 > out.shape:torch.Size([1, 64, 16, 16, 16])
2024-05-27 16:47:50,084 - DEBUG -    norm2 > out.shape:torch.Size([1, 64, 16, 16, 16])
2024-05-27 16:47:50,087 - DEBUG -    residual > residual.shape:torch.Size([1, 64, 16, 16, 16])
2024-05-27 16:47:50,087 - DEBUG -       upsample3 > x.shape:torch.Size([1, 64, 16, 32, 32])
2024-05-27 16:47:50,087 - DEBUG -    🍒 ResBlock3D x.shape:torch.Size([1, 64, 16, 32, 32])
2024-05-27 16:47:50,088 - DEBUG -    conv1 > out.shape:torch.Size([1, 32, 16, 32, 32])
2024-05-27 16:47:50,089 - DEBUG -    norm1 > out.shape:torch.Size([1, 32, 16, 32, 32])
2024-05-27 16:47:50,089 - DEBUG -    F.relu(out) > out.shape:torch.Size([1, 32, 16, 32, 32])
2024-05-27 16:47:50,091 - DEBUG -    conv2 > out.shape:torch.Size([1, 32, 16, 32, 32])
2024-05-27 16:47:50,091 - DEBUG -    norm2 > out.shape:torch.Size([1, 32, 16, 32, 32])
2024-05-27 16:47:50,093 - DEBUG -    residual > residual.shape:torch.Size([1, 32, 16, 32, 32])
2024-05-27 16:47:50,093 - DEBUG -       upsample4 > x.shape:torch.Size([1, 32, 16, 64, 64])
2024-05-27 16:47:50,094 - DEBUG -       conv3x3x3 > x.shape:torch.Size([1, 3, 16, 64, 64])
2024-05-27 16:47:50,094 - DEBUG -       gn > x.shape:torch.Size([1, 3, 16, 64, 64])
2024-05-27 16:47:50,094 - DEBUG -       F.relu > x.shape:torch.Size([1, 3, 16, 64, 64])
2024-05-27 16:47:50,096 - DEBUG -       tanh > x.shape:torch.Size([1, 3, 16, 64, 64])
2024-05-27 16:47:50,096 - DEBUG - w_em_s2c:  :torch.Size([1, 3, 16, 64, 64])
2024-05-27 16:47:50,127 - DEBUG - w_rt_s2c: :torch.Size([1, 3, 64, 64, 64])
2024-05-27 16:47:50,129 - DEBUG - w_em_s2c_resized: torch.Size([1, 3, 64, 64, 64])
2024-05-27 16:47:50,129 - DEBUG - vs shape:torch.Size([1, 96, 16, 64, 64])
2024-05-27 16:47:50,129 - DEBUG - 🍏 apply_warping_field v:torch.Size([1, 96, 16, 64, 64])
2024-05-27 16:47:50,129 - DEBUG - warp_field:torch.Size([1, 3, 64, 64, 64])
2024-05-27 16:47:50,130 - DEBUG - Resized warp_field:torch.Size([1, 3, 16, 64, 64])
2024-05-27 16:47:50,130 - DEBUG - Canonical grid:torch.Size([16, 64, 64, 3])
2024-05-27 16:47:50,130 - DEBUG - Batch grid:torch.Size([1, 16, 64, 64, 3])
2024-05-27 16:47:50,130 - DEBUG - Warped grid:torch.Size([1, 16, 64, 64, 3])
2024-05-27 16:47:50,131 - DEBUG - Normalization factors:tensor([63, 63, 15], device='cuda:0')
2024-05-27 16:47:50,132 - DEBUG - Normalized warped grid:torch.Size([1, 16, 64, 64, 3])
2024-05-27 16:47:50,133 - DEBUG - v_canonical:torch.Size([1, 96, 16, 64, 64])
2024-05-27 16:47:50,164 - DEBUG - FlowField > zs sum.shape:torch.Size([1, 512, 4, 4])
2024-05-27 16:47:50,164 - DEBUG -       conv1x1 > x.shape:torch.Size([1, 2048, 4, 4])
2024-05-27 16:47:50,164 - DEBUG -       reshape_layer > x.shape:torch.Size([1, 512, 4, 4, 4])
2024-05-27 16:47:50,164 - DEBUG -    🍒 ResBlock3D x.shape:torch.Size([1, 512, 4, 4, 4])
2024-05-27 16:47:50,165 - DEBUG -    conv1 > out.shape:torch.Size([1, 256, 4, 4, 4])
2024-05-27 16:47:50,165 - DEBUG -    norm1 > out.shape:torch.Size([1, 256, 4, 4, 4])
2024-05-27 16:47:50,165 - DEBUG -    F.relu(out) > out.shape:torch.Size([1, 256, 4, 4, 4])
2024-05-27 16:47:50,165 - DEBUG -    conv2 > out.shape:torch.Size([1, 256, 4, 4, 4])
2024-05-27 16:47:50,166 - DEBUG -    norm2 > out.shape:torch.Size([1, 256, 4, 4, 4])
2024-05-27 16:47:50,166 - DEBUG -    residual > residual.shape:torch.Size([1, 256, 4, 4, 4])
2024-05-27 16:47:50,166 - DEBUG -       upsample1 > x.shape:torch.Size([1, 256, 8, 8, 8])
2024-05-27 16:47:50,166 - DEBUG -    🍒 ResBlock3D x.shape:torch.Size([1, 256, 8, 8, 8])
2024-05-27 16:47:50,166 - DEBUG -    conv1 > out.shape:torch.Size([1, 128, 8, 8, 8])
2024-05-27 16:47:50,166 - DEBUG -    norm1 > out.shape:torch.Size([1, 128, 8, 8, 8])
2024-05-27 16:47:50,166 - DEBUG -    F.relu(out) > out.shape:torch.Size([1, 128, 8, 8, 8])
2024-05-27 16:47:50,167 - DEBUG -    conv2 > out.shape:torch.Size([1, 128, 8, 8, 8])
2024-05-27 16:47:50,167 - DEBUG -    norm2 > out.shape:torch.Size([1, 128, 8, 8, 8])
2024-05-27 16:47:50,167 - DEBUG -    residual > residual.shape:torch.Size([1, 128, 8, 8, 8])
2024-05-27 16:47:50,167 - DEBUG -       upsample2 > x.shape:torch.Size([1, 128, 16, 16, 16])
2024-05-27 16:47:50,167 - DEBUG -    🍒 ResBlock3D x.shape:torch.Size([1, 128, 16, 16, 16])
2024-05-27 16:47:50,167 - DEBUG -    conv1 > out.shape:torch.Size([1, 64, 16, 16, 16])
2024-05-27 16:47:50,168 - DEBUG -    norm1 > out.shape:torch.Size([1, 64, 16, 16, 16])
2024-05-27 16:47:50,168 - DEBUG -    F.relu(out) > out.shape:torch.Size([1, 64, 16, 16, 16])
2024-05-27 16:47:50,168 - DEBUG -    conv2 > out.shape:torch.Size([1, 64, 16, 16, 16])
2024-05-27 16:47:50,169 - DEBUG -    norm2 > out.shape:torch.Size([1, 64, 16, 16, 16])
2024-05-27 16:47:50,169 - DEBUG -    residual > residual.shape:torch.Size([1, 64, 16, 16, 16])
2024-05-27 16:47:50,169 - DEBUG -       upsample3 > x.shape:torch.Size([1, 64, 16, 32, 32])
2024-05-27 16:47:50,169 - DEBUG -    🍒 ResBlock3D x.shape:torch.Size([1, 64, 16, 32, 32])
2024-05-27 16:47:50,169 - DEBUG -    conv1 > out.shape:torch.Size([1, 32, 16, 32, 32])
2024-05-27 16:47:50,170 - DEBUG -    norm1 > out.shape:torch.Size([1, 32, 16, 32, 32])
2024-05-27 16:47:50,170 - DEBUG -    F.relu(out) > out.shape:torch.Size([1, 32, 16, 32, 32])
2024-05-27 16:47:50,170 - DEBUG -    conv2 > out.shape:torch.Size([1, 32, 16, 32, 32])
2024-05-27 16:47:50,170 - DEBUG -    norm2 > out.shape:torch.Size([1, 32, 16, 32, 32])
2024-05-27 16:47:50,170 - DEBUG -    residual > residual.shape:torch.Size([1, 32, 16, 32, 32])
2024-05-27 16:47:50,170 - DEBUG -       upsample4 > x.shape:torch.Size([1, 32, 16, 64, 64])
2024-05-27 16:47:50,171 - DEBUG -       conv3x3x3 > x.shape:torch.Size([1, 3, 16, 64, 64])
2024-05-27 16:47:50,171 - DEBUG -       gn > x.shape:torch.Size([1, 3, 16, 64, 64])
2024-05-27 16:47:50,171 - DEBUG -       F.relu > x.shape:torch.Size([1, 3, 16, 64, 64])
2024-05-27 16:47:50,171 - DEBUG -       tanh > x.shape:torch.Size([1, 3, 16, 64, 64])
2024-05-27 16:47:50,173 - DEBUG - w_em_c2d_resized:torch.Size([1, 3, 64, 64, 64])
2024-05-27 16:47:50,173 - DEBUG - w_c2d shape:torch.Size([1, 3, 64, 64, 64])
2024-05-27 16:47:50,173 - DEBUG - 🍏 apply_warping_field v:torch.Size([1, 96, 16, 64, 64])
2024-05-27 16:47:50,173 - DEBUG - warp_field:torch.Size([1, 3, 64, 64, 64])
2024-05-27 16:47:50,173 - DEBUG - Resized warp_field:torch.Size([1, 3, 16, 64, 64])
2024-05-27 16:47:50,173 - DEBUG - Canonical grid:torch.Size([16, 64, 64, 3])
2024-05-27 16:47:50,173 - DEBUG - Batch grid:torch.Size([1, 16, 64, 64, 3])
2024-05-27 16:47:50,173 - DEBUG - Warped grid:torch.Size([1, 16, 64, 64, 3])
2024-05-27 16:47:50,174 - DEBUG - Normalization factors:tensor([63, 63, 15], device='cuda:0')
2024-05-27 16:47:50,174 - DEBUG - Normalized warped grid:torch.Size([1, 16, 64, 64, 3])
2024-05-27 16:47:50,174 - DEBUG - v_canonical:torch.Size([1, 96, 16, 64, 64])
2024-05-27 16:47:50,177 - DEBUG - G2d > x:torch.Size([1, 96, 64, 64])

@JZArray
Copy link
Author

JZArray commented May 27, 2024

@flyingshan Regarding to your questions:

  1. just follow dimensions used in their paper, at least it can work. If you think it is too small, you can increase the size, but no gurantee to resutls;
    2&3. I didn't use GN or Adaptive GN, BN can also generate good visual results (Maybe GN gives better quantitative results)
  2. Not sure whether this is a typo in paper, or totally a mistake in G3D, in their new paper EMOPortrait, the structure of G3D is same. However, follwing their steps, G3D cannot work, so I implemented this part by myself.

@johndpope
Copy link

@flyingshan - maybe this helps - johndpope/MegaPortrait-hack#18

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@johndpope @JZArray @flyingshan