This is the GitHub Repo for paper "ReasonAgain: Using Extractable Symbolic Programs to Evaluate Mathematical Reasoning".
The perturbation data we generate in the paper is in the "data" folder.
To generate your own perturbations, please first run
python generate_program.py
which will generate all the programs of questions. Then, please run
python run_program.py
to generate the perturbations.
Feel free to leave any questions about the code or the implementation of the pipeline. Thanks!