Word-to-World

Speech to 3D scene generation using LLMs and RAG. Built for VR using Unity.

About

A VR experience that allows the user to generate a scene with different assets through speech.

• Uses Langchain for RAG to pick from over 500 generic assets
• Uses OpenAI's GPT to transform textual descriptions of a scene into 3D configurations

Credits

The default database of assets is populated with models from: https://kenney.nl/assets