Codebase for the paper "Exploring and Exploiting Latent Commonsense Knowledge in Pretrained Masked Language Models".
Note: under maintenance, will be complete soon.
- DistilBERT-base
- BERT(base, large, etc.)
- RoBERTa(base, large, etc.)
- MPNet
git clone https://github.com/DRSY/LAMP.git && cd LAMP
pip install -r requirements.txt
Specify parameters about probing experiments in a separate params file, then run:
make -f Makefile probe
detailed hyperparameters can be found in probe.sh.
Specify parameters about GLUE experiments in a separate params file, then run:
make -f Makefile glue
make -f Makefile clean
detailed hyperparameters can be found in glue.sh.