BERT_for_GAP-coreference

This project was realised in the context of the INF8225 AI course. In this project, we aim to reduce gender bias in pronoun resolution by creating a coreference resolver that performs well on a gender-balanced pronoun dataset, the Gendered Ambiguous Pronouns (GAP) dataset. We leverage BERT's strong pre-training tasks on large unsupervised datasets and transfer these contextual representations to the fine-tuning stage. The fine-tuning stage was trained in a SWAG-like manner on the GAP supervised dataset.

We have submitted our best performing model to the Gendered Pronoun Resolution Kaggle competition.

Setting up

git clone --recursive git@github.com:isabellebouchard/BERT_for_GAP-coreference.git

Make sure the submodules are properly initialized.

First steps

To run the code, first install Docker to be able to build and run a docker container with all the proper dependencies installed

docker build -t IMAGE_NAME .
nvidia-docker run --rm -it -v /path/to/your/code/:/project IMAGE_NAME

If you don't have access to GPU, change nvidia-docker for docker. It is highly recommended to run the training on (multiple) GPUs.

Once inside the container you should be able run the training script:

python run_GAP.py --data_dir gap-coreference \
                  --bert_model bert-base-cased \
                  --output_dir results \

This will run the training script and save checkpoints of the best model in the output directory.