This folder contains the source code for Long-Range Arena benchmark in Transformer-LS paper.
It is built on this repository
The implementation of long-short term attention is here.
To run the experiments, first, run the following to create the train, val and test data
cd datasets
python text.py
python listops.py
python retrieval.py
To download the datasets, one may refer to the instructions within Long-Range Arena repository
Then, create the path for the checkpoints: mkdir LRA_chks. Finally, simply execute
bash run_text.sh
bash run_listops.sh
bash run_retrieval.sh