Generalizable Human Gaussians (GHG) for Sparse View Synthesis (ECCV 2024)

Youngjoong Kwon; · Baole Fang* · Yixing Lu* · Haeoye Dong · Cheng Zhang · Francisco Vicente Carrasco · Albert Mosella-Montoro · Jianjin Xu · Shingo Takagi · Daeil Kim · Aayush Prakash · Fernando de la Torre

^*Equal contribution.

Project Page | Video | Paper

News

• 08/26/2024 To make the comparison with our GHG easier, we provide the evaluation results in this link.
• 08/26/2024 The evaluation code and pretrained model of GHG are now released!

---

Installation

Instructions on downloading the dataset and pretrained model weights, and installing the dependencies can be found in INSTALL.md.

---

Custom dataset

If you want to try GHG on your own dataset, please refer to the CUSTOM_DATASET.md.

---

Training

```
CUDA_VISIBLE_DEVICES=0 python train_nightly_ver.py
```

Evaluation

We provide detailed information about the evaluation protocol in PROTOCOL.md. To make the comparison with our Generalizable Human Gaussians easier, we provide the evaluation results in this link.

1. Please download the pretrained weights following the instructions in INSTALL.md.
2. Generate the predictions.

   CUDA_VISIBLE_DEVICES=0 python eval.py --test_data_root datasets/THuman/val --regressor_path weights/model_gaussian.pth --inpaintor_path weights/model_inpaint.pth
   

   The results will be saved at $ROOT/outputs/eval/{$exp_name}.
3. Compute the metrics.

   python metrics/compute_metrics.py
   

Citation

If you find this code useful for your research, please cite it using the following BibTeX entry.

@article{kwon2024ghg,
  title={Generalizable Human Gaussians for Sparse View Synthesis},
  author={Youngjoong Kwon, Baole Fang, Yixing Lu, Haoye Dong, Cheng Zhang, Francisco Vicente Carrasco, Albert Mosella-Montoro, Jianjin Xu, Shingo Takagi, Daeil Kim, Aayush Prakash, Fernando De la Torre},
  journal={European Conference on Computer Vision},
  year={2024}
}