README.md

A Survey of Large Language Models Attribution [ArXiv preprint]

---

🌟 Introduction

Open-domain dialogue systems, driven by large language models, have changed the way we use conversational AI. However, these systems often produce content that might not be reliable. In traditional open-domain settings, the focus is mostly on the answer’s relevance or accuracy rather than evaluating whether the answer is attributed to the retrieved documents. A QA model with high accuracy may not necessarily achieve high attribution.

Attribution refers to the capacity of a model, such as an LLM, to generate and provide evidence, often in the form of references or citations, that substantiates the claims or statements it produces. This evidence is derived from identifiable sources, ensuring that the claims can be logically inferred from a foundational corpus, making them comprehensible and verifiable by a general audience. The primary purposes of attribution include enabling users to validate the claims made by the model, promoting the generation of text that closely aligns with the cited sources to enhance accuracy and reduce misinformation or hallucination, and establishing a structured framework for evaluating the completeness and relevance of the supporting evidence in relation to the presented claims.

In this repository, we focus on unraveling the sources that these systems tap into for attribution or citation. We delve into the origins of these facts, their utilization by the models, the efficacy of these attribution methodologies, and grapple with challenges tied to ambiguous knowledge reservoirs, inherent biases, and the pitfalls of excessive attribution.

✨ Work in progress. We would like to appreciate any contributions via PRs, issues from NLP community.

---

1. Attribution Definition & Position Paper

• [2021/07] Rethinking Search: Making Domain Experts out of Dilettantes Donald Metzler et al. arXiv. [paper]

• [2021/12] Measuring Attribution in Natural Language Generation Models. H Rashkin et al. CL. [paper]

• [2022/11] The attribution problem with generative AI Anna Rogers [blog]

• [2023/07] Citation: A Key to Building Responsible and Accountable Large Language Models Jie Huang et al. arXiv. [paper]

• [2023/10] Establishing Trustworthiness: Rethinking Tasks and Model Evaluation Robert Litschko et al. arXiv. [paper]

• [2023/11] Unifying Corroborative and Contributive Attributions in Large Language Models Theodora Worledge et al. NeurIPS ATTRIB Workshop 2023 [paper]

• [2024/03] Reliable, Adaptable, and Attributable Language Models with Retrieval Akari Asai et al. arXiv. [paper]

2. Attribution Paper Before the Era of Large Language Models and Related Task

2.1 Fact Checking & Claim Verificication & Natural Language Inference

• [2021/11] The Fact Extraction and VERification (FEVER) Shared Task James Thorne et al. EMNLP'18 [paper]

• [2021/08] A Survey on Automated Fact-Checking Zhijiang Guo et al. TACL'22 [paper]

• [2021/10] Truthful AI: Developing and governing AI that does not lie Owain Evans et al. arXiv [paper]

• [2021/05] Evaluating Attribution in Dialogue Systems: The BEGIN Benchmark Nouha Dziri et al. TACL'22 [paper] [code]

• [2023/10] Explainable Claim Verification via Knowledge-Grounded Reasoning with Large Language Models Haoran Wang et al. Findings of EMNLP'23 [paper]

• [2022/07] Improving Wikipedia Verifiability with AI Fabio Petroni et al. arXiv. [paper] [code]

2.2 Feature Attribution and Interpretability of Models for NLP

• [2022/12] Foveate, Attribute, and Rationalize: Towards Physically Safe and Trustworthy AI Alex Mei et al. findings of ACL'22 [paper]

• [2023/07] Inseq: An Interpretability Toolkit for Sequence Generation Models Gabriele Sarti et al. ACL Demo'23 [paper] [library]

• [2023/10] Quantifying the Plausibility of Context Reliance in Neural Machine Translation Gabriele Sarti et al. arXiv. [paper]

2.3 Attribution in Mutli-modal Systems

• [2017/06] A unified view of gradient-based attribution methods for Deep Neural Networks. Marco Ancona et al. arXiv. [paper]

• [2021/03] Towards multi-modal causability with Graph Neural Networks enabling information fusion for explainable AI. Andreas Holzinger et al. arXiv. [paper]

• [2023/03] Retrieving Multimodal Information for Augmented Generation: A Survey. Ruochen Zhao et al. arXiv. [paper]

• [2023/07] Improving Explainability of Disentangled Representations using Multipath-Attribution Mappings. Lukas Klein et al. arXiv. [paper]

• [2023/07] Visual Explanations of Image-Text Representations via Mult-Modal Information Bottleneck Attribution. Ying Wang et al. arXiv. [paper]

• [2023/07] MAEA: Multimodal Attribution for Embodied AI. Vidhi Jain et al. arXiv. [paper]

• [2023/10] Rephrase, Augment, Reason: Visual Grounding of Questions for Vision-Language Models. Archiki Prasad et al. arXiv. [paper] [code]

2.4 Wiki

• [2019/11] Transforming Wikipedia into Augmented Data for Query-Focused Summarization. Haichao Zhu et al. TASLP. [paper]

• [2023/04] WebBrain: Learning to Generate Factually Correct Articles for Queries by Grounding on Large Web Corpus. Hongjin Qian et al. arXiv. [paper]

2.5 Model based Information Retrieval

• [2022/02] Transformer Memory as a Differentiable Search Index Yi Tay et al. NeurIPS'22 [paper]

2.6 Small Language Model

• [2023/08] Optimizing Factual Accuracy in Text Generation through Dynamic Knowledge Selection Hongjin Qian et al. arxiv. [paper]

3. Sources of Attribution

3.1 Pre-training Data

• [2023/02] The ROOTS Search Tool: Data Transparency for LLMs Aleksandra Piktus et al. arXiv. [paper]

• [2022/05] ORCA: Interpreting Prompted Language Models via Locating Supporting Data Evidence in the Ocean of Pretraining Data Xiaochuang Han et al. arXiv. [paper]

• [2022/05] Understanding In-Context Learning via Supportive Pretraining Data Xiaochuang Han et al. arXiv. [paper]

• [2022/07] [link the fine-tuned LLM to its pre-trained base model] Matching Pairs: Attributing Fine-Tuned Models to their Pre-Trained Large Language Models Myles Foley et al. ACL 2023. [paper]

3.2 Out-of-model Knowledge and Retrieval-based Question Answering & Knowledge-Grounded Dialogue

• [2021/04] Retrieval augmentation reduces hallucination in conversation Kurt Shuster et al. arXiv. [paper]

• [2020/07] Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering Gautier Izacard et al. arXiv. [paper]

• [2021/12] Improving language models by retrieving from trillions of tokens Sebastian Borgeaud et al. arXiv. [paper]

• [2022/12] Rethinking with Retrieval: Faithful Large Language Model Inference Hangfeng He et al. arXiv. [paper]

4. Datasets for Attribution

• [2022/12] CiteBench: A benchmark for Scientific Citation Text Generation Martin Funkquist et al. arXiv. [paper]

• [2023/04] WebBrain: Learning to Generate Factually Correct Articles for Queries by Grounding on Large Web Corpus Hongjing Qian et al. arXiv. [paper] [code]

• [2023/05] Enabling Large Language Models to Generate Text with Citations Tianyu Gao et al. arXiv. [paper] [code]

• [2023/07] HAGRID: A Human-LLM Collaborative Dataset for Generative Information-Seeking with Attribution Ehsan Kamalloo et al. arXiv. [paper] [code]

• [2023/09] EXPERTQA : Expert-Curated Questions and Attributed Answers Chaitanya Malaviya et al. arXiv. [paper] [code]

• [2023/11] SEMQA: Semi-Extractive Multi-Source Question Answering Tal Schuster et al. arXiv. [paper] [code]

• [2024/01] Benchmarking Large Language Models in Complex Question Answering Attribution using Knowledge Graphs Nan Hu et al. arXiv. [paper]

• [2024/05] WebCiteS: Attributed Query-Focused Summarization on Chinese Web Search Results with Citations Haolin Deng et al. ACL'24 [paper]

5. Approaches to Attribution

5.1 Direct Generated Attribution

• [2023/05] "According to ..." Prompting Language Models Improves Quoting from Pre-Training Data Orion Weller et al. arXiv. [paper]

• [2023/07] Credible Without Credit: Domain Experts Assess Generative Language Models Denis Peskoff et al. ACL 2023. [paper]

• [2023/09] ChatGPT Hallucinates when Attributing Answers Guido Zuccon et al. arXiv. [paper]

• [2023/09] Towards Reliable and Fluent Large Language Models: Incorporating Feedback Learning Loops in QA Systems Dongyub Lee et al. arXiv. [paper]

• [2023/09] Retrieving Evidence from EHRs with LLMs: Possibilities and Challenges Hiba Ahsan et al. arXiv. [paper]

• [2023/10] Learning to Plan and Generate Text with Citations Annoymous et al. OpenReview, ICLR 2024 [paper]

• [2023/10] 1-PAGER: One Pass Answer Generation and Evidence Retrieval Palak Jain et al. arxiv [paper]

• [2024/2] How well do LLMs cite relevant medical references? An evaluation framework and analyses Kevin Wu et al. arXiv. [paper]

• [2024/4] Source-Aware Training Enables Knowledge Attribution in Language Models. Muhammad Khalifa et al. arXiv. [paper]

• [2024/4] CoTAR: Chain-of-Thought Attribution Reasoning with Multi-level Granularity Moshe Berchansky et al. arXiv. [paper]

• [2024/7] Improving Retrieval Augmented Language Model with Self-Reasoning Xia et al. arXiv. [paper]

5.2 Retrieval-then-Answering

• [2023/04] Search-in-the-Chain: Towards the Accurate, Credible and Traceable Content Generation for Complex Knowledge-intensive Tasks Shicheng Xu et al. arXiv. [paper]

• [2023/05] Mitigating Language Model Hallucination with Interactive Question-Knowledge Alignment Shuo Zhang et al. arXiv. [paper]

• [2023/03] SmartBook: AI-Assisted Situation Report Generation Revanth Gangi Reddy et al. arXiv. [paper]

• [2023/10] Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection Akari Asai et al. arXiv. [paper] [homepage]

• [2023/11] LLatrieval: LLM-Verified Retrieval for Verifiable Generation Xiaonan Li et al. arXiv. [paper] [code]

• [2023/11] Effective Large Language Model Adaptation for Improved Grounding Xi Ye et al. arXiv. [paper]

• [2024/01] Towards Verifiable Text Generation with Evolving Memory and Self-Reflection Hao Sun et al. arXiv. [paper]

• [2024/02] Training Language Models to Generate Text with Citations via Fine-grained Rewards Chengyu Huang et al. arXiv. [paper]

• [2024/03] Improving Attributed Text Generation of Large Language Models via Preference Learning Dongfang Li et al. arXiv. [paper]

5.3 Post-Generation Attribution

• [2022/10] RARR: Researching and Revising What Language Models Say, Using Language Models Luyu Gao et al. arXiv. [paper]

• [2023/04] The Internal State of an LLM Knows When its Lying Amos Azaria et al. arXiv. [paper]

• [2023/05] Do Language Models Know When They're Hallucinating References? Ayush Agrawal et al. arXiv. [paper]

• [2023/05] Complex Claim Verification with Evidence Retrieved in the Wild Jifan Chen et al. arXiv. [paper][code]

• [2023/06] Retrieving Supporting Evidence for LLMs Generated Answers Siqing Huo et al. arXiv. [paper]

• [2024/06] CaLM: Contrasting Large and Small Language Models to Verify Grounded Generation I-Hung Hsu1 et al. arXiv. [paper]

5.4 Attribution Systems & End-to-End Attribution Models

• [2022/03] LaMDA: Language Models for Dialog Applications. Romal Thoppilan et al. arXiv. [paper]

• [2022/03] WebGPT: Browser-assisted question-answering with human feedback. Reiichiro Nakano, Jacob Hilton, Suchir Balaji et al. arXiv.[paper]

• [2022/03] GopherCite - Teaching language models to support answers with verified quotes. Jacob Menick et al. arXiv. [paper]

• [2022/09] Improving alignment of dialogue agents via targeted human judgements. Amelia Glaese et al. arXiv. [paper]

• [2023/05] WebCPM: Interactive Web Search for Chinese Long-form Question Answering. Yujia Qin et al. arXiv. [paper]

6. Attribution Evaluation

• [2022/07] Improving Wikipedia Verifiability with AI Fabio Petroni et al. arXiv. [paper]

• [2022/12] Attributed Question Answering: Evaluation and Modeling for Attributed Large Language Models. B Bohnet et al. arXiv. [paper] [code]

• [2023/04] Evaluating Verifiability in Generative Search Engines Nelson F. Liu et al. arXiv. [paper] [annonated data]

• [2023/05] WICE: Real-World Entailment for Claims in Wikipedia Ryo Kamoi et al. arXiv. [paper]

• [2023/05] Evaluating and Modeling Attribution for Cross-Lingual Question Answering Benjamin Muller et al. arXiv. [paper]

• [2023/05] FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation Sewon Min et al. arXiv. [paper] [code]

• [2023/05] Automatic Evaluation of Attribution by Large Language Models. X Yue et al. arXiv. [paper] [code]

• [2023/07] FacTool: Factuality Detection in Generative AI -- A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarios I-Chun Chern et al. arXiv. [paper][code]

• [2023/09] Quantifying and Attributing the Hallucination of Large Language Models via Association Analysis Li Du et al. arXiv. [paper]

• [2023/10] Towards Verifiable Generation: A Benchmark for Knowledge-aware Language Model Attribution Xinze Li et al. arXiv. [paper]

• [2023/10] Understanding Retrieval Augmentation for Long-Form Question Answering Hung-Ting Chen et al. arXiv. [paper]

• [2023/11] Enhancing Medical Text Evaluation with GPT-4 Yiqing Xie et al. arXiv. [paper]

7. Limitations, Future Directions and Challenges in Attribution

  a. hallucination of attribution i.e. does attribution faithfully to its content?
  b. Inability to attribute parameter knowledge of model self.
  c. Validity of the knowledge source - source trustworthiness. Faithfulness ≠ Factuality
  d. Bias in attribution method
  e. Over-attribution & under-attribution
  f. Knowledge conflict

---

Cite

@misc{li2023llmattribution,
      title={A Survey of Large Language Models Attribution}, 
      author={Dongfang Li and Zetian Sun and Xinshuo Hu and Zhenyu Liu and Ziyang Chen and Baotian Hu and Aiguo Wu and Min Zhang},
      year={2023},
      eprint={2311.03731},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      howpublished={\url{https://github.com/HITsz-TMG/awesome-llm-attributions}},
}

For finding survey of hallucination please refer to:

• Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models
• Cognitive Mirage: A Review of Hallucinations in Large Language Models
• A Survey of Hallucination in Large Foundation Models

Project Maintainers & Contributors

• Dongfang Li
• Zetian Sun
• Xinshuo Hu
• Zhenyu Liu