Official code and data release for Unlocking Anticipatory Text Generation: A Constrained Approach for Large Language Models Decoding, accepted by EMNLP 2024.
The following command is used to install dependencies:
bash install.sh
- Lexical-Constraint Satisfaction Evaluation
test_model_constraints.sh
- Toxicity-Constraint Satisfaction Evaluation
test_model_constraints_toxity.sh
- Factual-Correctness-Satisfaction Evaluation
test_model_constraints_true.sh
test_model_constraints_true_YoN.sh
Go to the folder InstructionFollowing
- NEUROLOGIC (adapted version for LLM)
NEUROLOGIC.sh
- Greedy, Beam-Search, Our
beam_search_large.sh
In the above script, different settings can be used.
Greddy
beam_size=1
Beam-Search
beam_size=20
Our:
beam_size=20
choose alpha from [0.1, 10]
- Evaluation
Evaluation script is from CommonGen for automatic metrics.
Go to the folder Toxicity
toxity_script.sh
- Evaluation
Perspective API is used for toxicity evaluation.
Go to the folder FactaulQA
- main file
beam_search_eli5_large.py
- Claims as Constraints
beam_search_eli5_large_restricted_claims.sh
- Evaluation
Evaluation script is from ALCE. We did not truncate the answer by the first newline.
@inproceedings{
2024unlocking,
title={Unlocking Anticipatory Text Generation: A Constrained Approach for Large Language Models Decoding},
author={Lifu Tu and Semih Yavuz and Jin Qu and Jiacheng Xu and Rui Meng and Caiming Xiong and Yingbo Zhou},
booktitle={The 2024 Conference on Empirical Methods in Natural Language Processing},
year={2024},
url={https://openreview.net/forum?id=oeMJredL7n}
}