NeuroFluid

Code repository for this paper:
NeuroFluid: Fluid Dynamics Grounding with Particle-Driven Neural Radiance Fields.
Shanyan Guan, Huayu Deng, Yunbo Wang^†, Xiaokang Yang
ICML 2022
[Paper] [Project Page]

Please cite our paper if you find this code useful:

@inproceedings{guan2022neurofluid,
  title={NeuroFluid: Fluid Dynamics Grounding with Particle-Driven Neural Radiance Fields},
  author={Guan, Shanyan and Deng, Huayu and Wang, Yunbo and Yang, Xiaokang},
  booktitle={ICML},
  year={2022}
}

Dependencies

NeuroFluid is implemented and tested on Ubuntu 18.04 with python == 3.7 and cuda == 11.1. To run NeuroFluid, please install dependencies as follows:

1. Create an environment

   conda create -n fluid-env python=3.7
   conda activate fluid-env

2. Install Open3D.

   git clone https://github.com/isl-org/Open3D-ML.git && cd Open3D-ML && git checkout c461790869257e851ae7f035585b878df73bc093
   pip install open3d==0.15.2
   pip install -r requirements.txt
   pip install -r requirements-torch-cuda.txt

3. Install Pytorch3D

   conda install -c fvcore -c iopath -c conda-forge fvcore iopath
   conda install -c bottler nvidiacub
   wget -c https://github.com/facebookresearch/pytorch3d/archive/refs/tags/v0.6.1.tar.gz
   tar -xf v0.6.1.tar.gz && cd pytorch3d-0.6.1 && pip install -e .

4. install other dependencies

   pip install -r requirements.txt

Generate data

See this guide to generate fluid data.

Running the pretrained model

• Evaluate NeuroFluid:

python eval_e2e.py --resume_from $MODEL_PATH --dataset DATASET_NAME

DATASET_NAME is one element of [bunny, watercube, watersphere, honeycone]. To compute PSNR/SSIM/LPIPS results, run utils/evaluate_images.ipynb

• Evaluate the transition model

python eval_transmodel.py --resume_from $MODEL_PATH

• Evaluate the renderer

python eval_renderer.py --resume_from $MODEL_PATH --dataset DATASET_NAME

DATASET_NAME is one element of [bunny, watercube, watersphere, honeycone].

Run the training script

1. Warm-up stage

CUDA_VISIBLE_DEVICES=3 python train_renderer.py --expdir exps/watercube --expname scale-1.0/warmup --dataset watercube --config configs/watercube_warmup.yaml 

2. End2end stage

CUDA_VISIBLE_DEVICES=0 python train_e2e.py --expdir exps/watercube --expname scale-1.0/e2e --dataset watercube --config configs/watercube_e2e.yaml 

Fetch data

Download dataset and models from this link.

Acknowledgement

The implementation of transition model is borrowed from DeepLagrangianFluids. Please consider cite their paper if you use their code snippet:

@inproceedings{Ummenhofer2020Lagrangian,
        title     = {Lagrangian Fluid Simulation with Continuous Convolutions},
        author    = {Benjamin Ummenhofer and Lukas Prantl and Nils Thuerey and Vladlen Koltun},
        booktitle = {International Conference on Learning Representations},
        year      = {2020},
}

We refer to nerf_pl to implement our renderer. Thank D-NeRF for providing the script of computing PSNR/SSIM/LPIPS.