Goals as Reward-Producing Programs

A cleaned up version of the long-running development repository for our paper Goals as Reward-Producing Programs.

Quick links:

• Project webpage
• Paper
• Raw participant responses in our behavioral experiment
• Participant-created games as programs in our DSL
• Our DSL in an eBNF
• Code for our game creation behavioral experiment
• Code for our game human evaluation experiment

Reproduction

To reproduce analyses reported in the paper, see the notebooks in the reproduction_notebooks folder. We provide three notebooks:

• behavioral_analyses.ipynb reproduces anaylses of our behavioral data, reported in Figure 2.
• human_evaluation_summary.ipynb reproduces summary analyses of our human evaluations, reported in Tables 1 and SI-2.
• human_evaluations_mixed_models.ipynb reproduces the mixed effect model analyses of our human evaluations, reported in Tables ED-1, SI-3, and SI-4.

Viewing games

The easiest way to view games including in the human evaluation, including their back-translations to natural language, is to use the project webpage.

Otherwise, you can also use the game_viewer.ipynb to view any game from our participant-created dataset or model productions.

Setup

Create a conda environment, e.g. conda create -n game-gen python=3.10 anaconda (we worked with Python 3.10 on OS X and Ubuntu, though other versions should work, too).

Activate the environment with conda activate game-gen.

Install the requirements with pip install -r requirements.txt.

Running the R notebook human_evaluations_mixed_models.ipynb requires installing the R kernel for Jupyter, we use VS Code's integration, which requires following a few additional setup steps (if these instructions fail, try the ones here).

All in all, the setup should not take particularly long if you already have VS Code and conda installed.

Running the model

Running the model includes several steps, depending on what exactly you choose to modify from our current model. As a result, we provide a list of steps in a separate file, MODEL_README.md.