About 3D warping operation

[gorgeousdays]

Hi, thanks for your awesome work.

I am trying to reproduce the paper, but I have meet some problems with 3D warping operation.Can you provide a detailed explanation of the 3D warping operation.

Thanks.

[johndpope]

old ticket - but can you review this
https://real3dportrait.github.io/
their code has these modules
from modules.real3d.facev2v_warp.network
I believe that this builds from this paper - let me know otherwise.

[johndpope]

@gorgeousdays - i completely rebuild some code (using Claude Opus)
https://github.com/johndpope/MegaPortrait-hack

I got stuck on warping too (but I'm close)
johndpope/MegaPortrait-hack#4

@neeek2303 is the paper diagram actually accurate?

https://github.com/johndpope/MegaPortrait-hack/blob/main/model.py#L286

why do you need 2048 channels ?

when I concatenate the values - I get these dimensions - but then it's supposed to magically become 2048 - I don't understand.

        vs, es = self.Eapp(xs) # Appearance encoder source
        Rs, ts, zs = self.Emtn(xs) # motion encoder source
        Rd, td, zd = self.Emtn(xd) # motion encoder driving

        # es shape: torch.Size([1, 512])
        # Rs shape: torch.Size([1, 3])
        # ts shape: torch.Size([1, 3])
        # zs shape: torch.Size([1, 50])

I've got 512 + 3 + 3 + 50?

is 50 dimensions for expression accurate?

UPDATE - with some help - this is working
johndpope/MegaPortrait-hack#36