A differentiable point-based rendering library.
Document🏠 | Paper📄 (Comming soon) | DPRT Backend🌐

Pointrix is a differentiable point-based rendering library which has following properties:

• Powerful Backend:

  • Support "Render Anything"(depth, normal, optical flow, anything you want) and "Backward Anything" (Intrinsics & Extrinsics).
  • Modular design and easy to modify, support open-gl and opencv camera.

• Rich Feature:

  • Static Scene Reconstruction:
    • Vanilla 3DGS (2023 Siggraph Best Paper)
  • Dynamic Scene Reconstruction:
    • Deformable 3DGS (2024 CVPR)
    • Gaussian-Flow (2024 CVPR)
  • Text to 3D generation:
    • MVDream (2023 Arxiv)

• Highly Extensible and Designed for Research:

  • Pointrix adopts a modular design, with clear structure and easy extensibility.
  • Only few codes need to be modified if you want to add a new method.

contributors

Made with contrib.rocks.

Prerequisites

Installations

1. Install the following package:

First,create a new conda environment and activate it:

conda create -n pointrix python=3.9
conda activate pointrix

Then, you need to install pytorch

conda install pytorch==2.1.1 torchvision==0.16.1 pytorch-cuda=12.1 -c pytorch -c nvidia

Other dependencies:

pip install -r requirements.txt

Finally, install our DPTR rendering kernel:

# Install official diff-gaussian-rasterization
# clone the code from github
git clone https://github.com/pointrix-project/dptr.git --recursive
cd dptr
# install dptr
pip install .

# Install simple knn
git clone https://gitlab.inria.fr/bkerbl/simple-knn.git
cd simple-knn
python setup.py install
pip install .

Note: we support both gaussian original kernel and DPTR kernel.

Running our example

1. Lego

1. Download the lego data and put it in your folder:

wget http://cseweb.ucsd.edu/\~viscomp/projects/LF/papers/ECCV20/nerf/nerf_example_data.zip

2. Run the following command to train the model (...data path in the config file...):

cd Pointrix
pip install -e .
cd projects/gaussian_splatting
python launch.py --config ./configs/nerf_dptr.yaml

# you can also run this if you have installed gaussian original kernel
python launch.py --config ./configs/nerf.yaml

2. Mip-nerf 360 or other colmap dataset

1. Download the data and put it in your folder:

http://storage.googleapis.com/gresearch/refraw360/360_v2.zip

2. Run the following command to train the model (...data path in the config file...):

cd Pointrix
pip install -e .
cd projects/gaussian_splatting
python launch.py --config ./configs/colmap_dptr.yaml

# you can also run this if you have install gaussian original kernel
python launch.py --config ./configs/colmap.yaml

Try other methods

1. Dynamic Gaussian

1. Download the iphone dataset and put it in your folder: https://drive.google.com/drive/folders/1cBw3CUKu2sWQfc_1LbFZGbpdQyTFzDEX

2. Run the following command to train the model:

you need to modify the data path in the config file to the path of the data you downloaded.

cd Pointrix
pip install -e .
cd projects/deformable_gaussian
python launch.py --config deform.yaml

2. Generation (WIP)

WIP

• Introduction video
• Add GUI for visualization (this week).
• Implementataion of Gaussian-Flow (CVPR 2024) (this week).
• Implementataion of MVDream (this week).
• Implementataion of Relightable Gaussian (arXiv 2023).
• Support camera optimization (this week).

Welcome to join us or submit PR if you have any idea or methods.