Short-term memory and in-context retrieval in language models

Repository hosting code used for reports on retrieval in language models.

Citations

---

Armeni et al, 2024

@inproceedings{armeni-etal-2024-transformer,
    title = "Transformer verbatim in-context retrieval across time and scale",
    author = "Armeni, Kristijan  and
      Pranji{\'c}, Marko  and
      Pollak, Senja",
    editor = "Barak, Libby  and
      Alikhani, Malihe",
    booktitle = "Proceedings of the 28th Conference on Computational Natural Language Learning",
    month = nov,
    year = "2024",
    address = "Miami, FL, USA",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2024.conll-1.6",
    pages = "56--68",
}

Armeni et al, 2022

@inproceedings{armeni-etal-2022-characterizing,
    title = "Characterizing Verbatim Short-Term Memory in Neural Language Models",
    author = "Armeni, Kristijan  and
      Honey, Christopher  and
      Linzen, Tal",
    editor = "Fokkens, Antske  and
      Srikumar, Vivek",
    booktitle = "Proceedings of the 26th Conference on Computational Natural Language Learning (CoNLL)",
    month = dec,
    year = "2022",
    address = "Abu Dhabi, United Arab Emirates (Hybrid)",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2022.conll-1.28",
    doi = "10.18653/v1/2022.conll-1.28",
    pages = "405--424",
}

Licence

---

This work is licensed under the Creative Commons Attribution 4.0 International License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/ or send a letter to Creative Commons, PO Box 1866, Mountain View, CA 94042, USA.