Implementation Details

[Yuan1221]

Hi there,
I have used 2 T4 GPU to retrain the model using the train_con.py script. The issue(cleanup function) persists across 2 different datasets, CARLA and CelebA. I would like to ask if this happened to you while training the model or if it's some error while executing the script.

Lots of thanks!

[primecai]

Hi,

The "cleanup" function is specific to a cluster I am using. Please feel free to comment it out! It has no effect on training.

Best regards,
Shengqu

[Yuan1221]

Thanks for your reply.

I have attempted to comment it out, however, the training process will be interrupted after 1 stage, which it won't progress to the next stage (0% in 'Progress to next stage: ), and subsequently exit the process completely.

[primecai]

Are you sure the other parts of the code are unchanged, and that you are loading the datasets correctly? The time it takes to reach 6 epochs is definitely not right.

[Yuan1221]

Right, I have modified the parameters inside train.py and curriculum,py, magically it works normally now.

May I know if Pix2NeRF is able to be trained on using own dataset? The dataset format I'm trying is in 1280x720 resolution with noisy background .

[primecai]

Hi,

I do not think training on 1280x720 images with noisy backgrounds will work without any tuning. There are two reasons:

1. Our backbone pi-GAN is very memory consuming to train, and the generative geometry prior need a relatively large batch size to work. To fit in 1280x720 images, this will be very computationally expensive. To enable these high resolution training, please check EG3D.
2. Our model requires a strong canonical pose to work. With noisy background, I doubt it will work as well as mere faces/cars/chairs.

Best regards,
Shengqu

[Yuan1221]

Thanks for the reply,

In terms of EG3D, may I know what you recommend is to replace the pi-GAN backbone with it?

If Pix2NeRF is retrained with own custom dataset, wouldn't the previous prior (faces/cars/chairs) not constraining the images as it's not pretrained using these datasets.

Thank you.

[primecai]

Hi,

Yes, it will be very interesting to train a feed-forward framework for EG3D by combining the ideas.
And I may not fully understand your second question, do you mean to finetune the pretrained models on other datasets?

Best,
Shengqu