Unraveling the Effects of Synthetic Data on
End-to-End Autonomous Driving Humanoid Robots

Junhao Ge*¹ , Zuhong Liu*¹ , Longteng Fan*¹ Yifan Jiang*¹
Jiaqi Su¹ , Yiming Li² , Zhejun Zhang³ , Siheng Chen^1,4
1Shanghai Jiao Tong University ²New York University
³ETH Zurich ⁴Shanghai Artificial Intelligence Laboratory

This is the official project repository of the paper Unraveling the Effects of Synthetic Data on End-to-End Autonomous Driving.

TODOs

• Arxiv Link
• Code Release
  • Data Processing
  • Data Generation
  • Data Rendering
  • Closed-Loop Evaluation
• Demo Data Relase
• Installation of this repo
• Demo script for demo
• Comment for Code

SceneCrafter

A comprehensive scene generation and simulation framework for autonomous driving research.

Overview

SceneCrafter is a powerful toolkit for generating realistic driving scenarios, simulating traffic behaviors, and rendering high-quality scene data for autonomous driving research and development.

Installation

Prerequisites

• Python 3.8+
• CUDA-enabled GPU (for rendering)
• pytorch 2.4.1+cu121

Set up the conda environment

# git clone the repository
git clone https://github.com/cancaries/SceneCrafter.git
cd SceneCrafter

# Set conda environment
conda create -n scenecrafter python=3.8
conda activate scenecrafter

# Install torch (corresponding to your CUDA version)
pip install torch==2.4.1+cu121 torchvision==0.19.1+cu121 torchaudio==2.4.1+cu121 --index-url https://download.pytorch.org/whl/cu121

# Install requirements
pip install -r requirements.txt

# Install SceneRenderer submodules(Street Gaussian)
pip install ./SceneRenderer/street-gaussian/submodules/diff-gaussian-rasterization
pip install ./SceneRenderer/street-gaussian/submodules/simple-knn
pip install ./SceneRenderer/street-gaussian/submodules/simple-waymo-open-dataset-reader

Demo Data

We provide a demo data in demo link, please unzip it in the root folder. You can try the demo by running the following command:

python ./scripts/generate_data.sh

License

This project is licensed under the MIT License - see the LICENSE file for details.

Citation

If you use SceneCrafter in your research, please cite our work:

@misc{ge2025unravelingeffectssyntheticdata,
      title={Unraveling the Effects of Synthetic Data on End-to-End Autonomous Driving}, 
      author={Junhao Ge and Zuhong Liu and Longteng Fan and Yifan Jiang and Jiaqi Su and Yiming Li and Zhejun Zhang and Siheng Chen},
      year={2025},
      eprint={2503.18108},
      archivePrefix={arXiv},
      primaryClass={cs.RO},
      url={https://arxiv.org/abs/2503.18108}, 
}

Acknowledgements

SceneRenderer is built based on Street Gaussian. More details can be found in Street Gaussian.