π [ACM Computing Surveys 2025] Awesome Lifelong Learning Methods for Large Language Models (Updated Regularly; Latest Papers from NIPS2024, EMNLP2024, COLING2025, AAAI2025, and ICLR2025)
- 2025.02 β Our survey "Towards Lifelong Learning of Large Language Models: A Survey" is accepted by ACM Computing Surveys! π
- 2025.01 β The latest research from NIPS 2024, EMNLP 2024, COLING 2025, AAAI 2025, and ICLR 2025 is now available! Stay updated with the newest advancements in the field.
- 2024.12 β Our survey "Towards Lifelong Learning of Large Language Models: A Survey" has been featured on ζΊε¨δΉεΏ and η₯δΉ! Check out these platforms for an in-depth interpretation.
- 2024.06 β We have officially released our survey paper "Towards Lifelong Learning of Large Language Models: A Survey"! This repository is now live alongside the paperβfeel free to contribute by opening pull requests to add your papers! π
This repository collects awesome surveys, resources, and papers for Lifelong Learning with Large Language Models.
We define 12 lifelong learning scenarios as follows. Please refer to this survey for a detailed introduction.
Additionally, you can refer to this repository for lifelong learning of LLM agents.
- Lifelong Learning of Large Language Model-based Agents: A Roadmap (arXiv 2025.01)
- Towards Lifelong Learning of Large Language Models: A Survey (arXiv 2024.06)
- Recent Advances of Foundation Language Models-based Continual Learning: A Survey (arXiv 2024.05)
- Continual Learning of Large Language Models: A Comprehensive Survey (arXiv 2024.04)
- Towards Incremental Learning in Large Language Models: A Critical Review (arXiv 2024.04)
- Continual Learning for Large Language Models: A Survey (arXiv 2024.02)
- Continual Learning with Pre-Trained Models: A Survey (arXiv 2024.01)
- A Comprehensive Survey of Continual Learning: Theory, Method and Application (TPAMI 2024)
- How Do Large Language Models Capture the Ever-changing World Knowledge? A Review of Recent Advances (EMNLP 2023)
- Revisiting Class-Incremental Learning with Pre-Trained Models: Generalizability and Adaptivity are All You Need (arXiv 2023.03)
- Deep Class-Incremental Learning: A Survey (arXiv 2023.03)
- A Comprehensive Survey of Continual Learning: Theory, Method and Application (arXiv 2023.02)
- Continual Learning of Natural Language Processing Tasks: A Survey (arXiv 2022.11)
- Continual Lifelong Learning in Natural Language Processing: A Survey (COLING 2020)
- Codebase for Incremental Learning with LLMs
- UIC-Liu-Lab ContinualLM
- Mammoth
- RevisitingCIL
- PyCIL
- PyContinual
Coming soon! β³
-
Spurious Forgetting in Continual Learning of Language Models, ICLR 2025.
-
Knowledge And Capability Transfer Through Large Language Models' Parameters Fusing, ICLR 2025.
-
CollabEdit: Towards Non-destructive Collaborative Knowledge Editing, ICLR 2025.
-
MathCoder2: Better Math Reasoning from Continued Pretraining on Model-translated Mathematical Code, ICLR 2025.
-
Mix-CPT: A Domain Adaptation Framework via Decoupling Knowledge Learning and Format Alignment, ICLR 2025.
-
Self-Updatable Large Language Models with Parameter Integration, ICLR 2025.
-
Synthetic continued pretraining, ICLR 2025.
-
Last Iterate Convergence of Incremental Methods as a Model of Forgetting, ICLR 2025.
-
Unlocking the Power of Function Vectors for Characterizing and Mitigating Catastrophic Forgetting in Continual Instruction Tuning, ICLR 2025.
-
Perturbation-Restrained Sequential Model Editing, ICLR 2025.
-
In-Context Editing: Learning Knowledge from Self-Induced Distributions, ICLR 2025.
-
Adaptive Prompting for Continual Relation Extraction: A Within-Task Variance Perspective, AAAI 2025.
-
CMT: A Memory Compression Method for Continual Knowledge Learning of Large Language Models, AAAI 2025.
-
Continual Learning Using a Kernel-Based Method Over Foundation Models, AAAI 2025.
-
CareBot: A Pioneering Full-Process Open-Source Medical Language Model, AAAI 2025.
-
Domain-adaptative Continual Learning for Low-resource Tasks: Evaluation on Nepali, COLING 2025.
-
In-context Continual Learning Assisted by an External Continual Learner, COLING 2025.
-
Continual Learning Using Only Large Language Model Prompting, COLING 2025.
-
Rethinking Long Context Generation from the Continual Learning Perspective, COLING 2025.
-
Dynamic-prototype Contrastive Fine-tuning for Continual Few-shot Relation Extraction with Unseen Relation Detection, COLING 2025.
-
Discarding the Crutches: Adaptive Parameter-Efficient Expert Meta-Learning for Continual Semantic Parsing, COLING 2025. (Semantic Parsing)
-
TL-CL: Task And Language Incremental Continual Learning, EMNLP 2024.
-
SEEKR: Selective Attention-Guided Knowledge Retention for Continual Learning of Large Language Models, EMNLP 2024.
-
Mitigating Catastrophic Forgetting in Language Transfer via Model Merging, Findings of EMNLP 2024.
-
Gradient Localization Improves Lifelong Pretraining of Language Models, Findings of EMNLP 2024.
-
Learn more, but bother less: parameter efficient continual learning, NIPS 2024.
-
Continual Learning with Global Alignment, NIPS 2024.
-
Continual Audio-Visual Sound Separation, NIPS 2024. (Audio)
-
Should We Really Edit Language Models? On the Evaluation of Edited Language Models, NIPS 2024.
-
Continual Learning with Embedding Layer Surgery and Task-wise Beam Search using Whisper, IEEE Spoken Language Technology Workshop. (Speech)
-
Chained Tuning Leads to Biased Forgetting, ICML 2024 Workshop.
-
LOIRE: LifelOng learning on Incremental data via pre-trained language model gRowth Efficiently, OpenReview Preprint.
-
Learning Beyond the Surface: How Far Can Continual Pre-Training with LoRA Enhance LLMs' Domain-Specific Insight Learning?, Preprint.
-
Boosting Tool Use of Large Language Models via Iterative Reinforced Fine-Tuning, Preprint.
-
Continual Learning for Encoder-only Language Models via a Discrete Key-Value Bottleneck, Preprint.
-
CPRM: A LLM-based Continual Pre-training Framework for Relevance Modeling in Commercial Search, Preprint.
-
Velocitune: A Velocity-based Dynamic Domain Reweighting Method for Continual Pre-training, Preprint.
-
Federated Incremental Named Entity Recognition, Preprint.
-
Reviving Dormant Memories: Investigating Catastrophic Forgetting in Language Models through Rationale-Guidance Difficulty, Preprint.
-
Exploring Forgetting in Large Language Model Pre-Training, Preprint.
-
Efficient Continual Pre-training of LLMs for Low-resource Languages, Preprint.
-
EvoWiki: Evaluating LLMs on Evolving Knowledge, Preprint.
-
WISE: Rethinking the Knowledge Memory for Lifelong Model Editing of Large Language Models, NIPS 2024.
-
Train-Attention: Meta-Learning Where to Focus in Continual Knowledge Learning, NIPS 2024.
-
D-CPT Law: Domain-specific Continual Pre-Training Scaling Law for Large Language Models, NIPS 2024.
-
TiC-LM: A Multi-Year Benchmark for Continual Pretraining of Language Models, NIPS 2024 Workshop.
-
Stress-Testing Long-Context Language Models with Lifelong ICL and Task Haystack, NIPS 2024 (Database&Benchmark Track)
-
Does RoBERTa Perform Better than BERT in Continual Learn- ing: An Attention Sink Perspective, CoLM.
-
LEMoE: Advanced Mixture of Experts Adaptor for Lifelong Model Editing of Large Language Models, EMNLP 2024.
-
Lifelong Event Detection via Optimal Transport, EMNLP 2024.
-
Should We Really Edit Language Models? On the Evaluation of Edited Language Models, Preprint 2024.
-
Exploring Continual Fine-Tuning for Enhancing Language Ability in Large Language Model, Preprint 2024.
-
ToW: Thoughts of Words Improve Reasoning in Large Language Models, Preprint 2024.
-
SeQuiFi: Mitigating Catastrophic Forgetting in Speech Emotion Recognition with Sequential Class-Finetuning, Preprint 2024.(Speech)
-
In-context KV-Cache Eviction for LLMs via Attention-Gate, Preprint Openreview 2024.
-
In Praise of Stubbornness: The Case for Cognitive-Dissonance Aware Continual Update of Knowledge in LLMs, Preprint Openreview 2024.
-
Structure-aware Domain Knowledge Injection for Large Language Models, Preprint Openreview 2024.
-
Programming Every Example: Lifting Pre-training Data Quality like Experts at Scale, Preprint Openreview 2024.
-
Data Mixing Laws: Optimizing Data Mixtures by Predicting Language Modeling Performance, Preprint.
-
Amuro and Char: Analyzing the Relationship between Pre-Training and Fine-Tuning of Large Language Models, Preprint Openreview 2024.
-
ToEdit: How to Synthesize Text Data to Avoid Model Collapse?, Preprint Openreview 2024.
-
Math for AI: On the Generalization of Learning Mathematical Problem Solving, Preprint Openreview 2024.
-
EMMA-500: Enhancing Massively Multilingual Adaptation of Large Language Models, Preprint Openreview 2024.
-
Locate-then-Unlearn: An Effective Method of Multi-Task Continuous Learning for Large Language Models, Preprint Openreview 2024.
-
GE-PEFT: Gated Expandable Parameter-Efficient Fine-Tuning for Continual Learning, Preprint Openreview 2024.
-
Language Models as Feature Extractors for Accurate Continual Learning, Preprint Openreview 2024.
-
Towards Efficient and No Forgetting Domain Continual Pretraining by Mitigating the Stability Gap, Preprint Openreview 2024.
-
Data Efficient Continual Learning of Large Language Model, Preprint Openreview 2024.
-
NIRANTAR: Continual Learning with New Languages and Domains on Real-world Speech Data, Preprint Openreview 2024.
-
Continual Memorization of Factoids in Large Language Models, Preprint Openreview 2024.
-
Contextual Experience Replay for Continual Learning of Language Agents, Preprint Openreview 2024.
-
Balancing Continuous Pre-Training and Instruction Fine-Tuning: Optimizing Instruction-Following in LLMs, Preprint 2024.
-
Is Parameter Collision Hindering Continual Learning in LLMs?, Preprint 2024.
-
SLIM: Let LLM Learn More and Forget Less with Soft LoRA and Identity Mixture, Preprint 2024.
-
A Closer Look at Machine Unlearning for Large Language Models, Preprint 2024.
-
LeanAgent: Lifelong Learning for Formal Theorem Proving, Preprint 2024.
-
NusaMT-7B: Machine Translation for Low-Resource Indonesian Languages with Large Language Models, Preprint 2024.
-
Learning Attentional Mixture of LoRAs for Language Model Continual Learning, Preprint 2024.
-
Beyond Prompts: Dynamic Conversational Benchmarking of Large Language Models, Preprint 2024.
-
Synthetic continued pretraining, Preprint 2024.
-
Alleviating Hallucinations in Large Language Models with Scepticism Modeling, Preprint 2024.
-
A Practice of Post-Training on Llama-3 70B with Optimal Selection of Additional Language Mixture Ratio, Preprint 2024.
-
RedWhale: An Adapted Korean LLM Through Efficient Continual Pretraining, Preprint 2024.
-
PMoE: Progressive Mixture of Experts with Asymmetric Transformer for Continual Learning, Preprint 2024.
-
Enhance Lifelong Model Editing with Continuous Data-Adapter Association, Preprint 2024.
-
LoRAMoE: Alleviating World Knowledge Forgetting in Large Language Models via MoE-Style Plugin, ACL 2024.
-
TaSL: Task Skill Localization and Consolidation for Language Model Continual Learning, ACL 2024.
-
Forgetting before Learning: Utilizing Parametric Arithmetic for Knowledge Updating in Large Language Models, ACL 2024.
-
Continual Dialogue State Tracking via Reason-of-Select Distillation, Findings of ACL 2024.
-
Overcoming Catastrophic Forgetting by Exemplar Selection in Task-oriented Dialogue System, Findings of ACL 2024.
-
Unlocking Continual Learning Abilities in Language Models, Preprint 2024.
[pdf] [Code]
-
Large Language Model Can Continue Evolving From Mistakes, Preprint 2024.
-
COPAL: Continual Pruning in Large Language Generative Models, ICML 2024.
[pdf]
-
Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention, Preprint 2024.
-
AdapterSwap: Continuous Training of LLMs with Data Removal and Access-Control Guarantees, Preprint 2024.
-
Efficient Continual Pre-training for Building Domain Specific Large Language Models, ACL (Findings) 2024.
-
Mitigating Catastrophic Forgetting in Language Transfer via Model Merging, Preprint 2024.
-
Reuse, Donβt Retrain: A Recipe for Continued Pretraining of Language Models, Preprint 2024.
[pdf]
-
Breaking Language Barriers: Cross-Lingual Continual Pre-Training at Scale, Preprint 2024.
-
MoE-CT: A Novel Approach For Large Language Models Training With Resistance To Catastrophic Forgetting, Preprint 2024.
-
Leitner-Guided Memory Replay for Cross-lingual Continual Learning, NAACL 2024.
-
BAMBINO-LM: (Bilingual-)Human-Inspired Continual Pretraining of BabyLM, Preprint 2024.
-
Improving Language Models Trained with Translated Data via Continual Pre-Training and Dictionary Learning Analysis, Preprint 2024.
-
Boosting Large Language Models with Continual Learning for Aspect-based Sentiment Analysis, Preprint 2024.
[pdf]
-
ChronosLex: Time-aware Incremental Training for Temporal Generalization of Legal Classification Tasks, ACL 2024.
-
XMC-Agent : Dynamic Navigation over Scalable Hierarchical Index for Incremental Extreme Multi-label Classification, ACL (Findings) 2024.
-
Learn or Recall? Revisiting Incremental Learning with Pre-trained Language Models, ACL 2024.
[pdf] [Code]
-
Flexible Weight Tuning and Weight Fusion Strategies for Continual Named Entity Recognition, ACL (Findings) 2024.
-
Distilling Causal Effect of Data in Continual Few-shot Relation Learning, LREC-COLING 2024.
-
Improving Continual Few-shot Relation Extraction through Relational Knowledge Distillation and Prototype Augmentation, LREC-COLING 2024.
-
Enhancing Contrastive Learning with Noise-Guided Attack: Towards Continual Relation Extraction in the Wild, ACL 2024.
-
An Ensemble-of-Experts Framework for Rehearsal-free Continual Relation Extraction, ACL (Findings) 2024.
-
Continual Few-shot Relation Extraction via Adaptive Gradient Correction and Knowledge Decomposition, ACL (Findings) 2024.
-
Continual Learning with Semi-supervised Contrastive Distillation for Incremental Neural Machine Translation, ACL 2024.
[pdf]
-
Continual Dialogue State Tracking via Reason-of-Select Distillation, ACL (Findings) 2024,
-
Interpretable Catastrophic Forgetting of Large Language Model Fine-tuning via Instruction Vector, Preprint 2024.
-
Learn it or Leave it: Module Composition and Pruning for Continual Learning, Preprint 2024.
-
Dirichlet Continual Learning: Tackling Catastrophic Forgetting in NLP, UAI 2024.
-
PromptDSI: Prompt-based Rehearsal-free Instance-wise Incremental Learning for Document Retrieval, Preprint 2024.
-
Revisiting Catastrophic Forgetting in Large Language Model Tuning, Preprint 2024.
-
HFT: Half Fine-Tuning for Large Language Models, Preprint 2024.
-
Overcoming Catastrophic Forgetting by Exemplar Selection in Task-oriented Dialogue System, ACL 2024.
-
SAPT: A Shared Attention Framework for Parameter-Efficient Continual Learning of Large Language Models, ACL 2024.
-
Self-Evolving GPT: A Lifelong Autonomous Experiential Learner, ACL 2024.
-
LoRAMoE: Alleviating World Knowledge Forgetting in Large Language Models via MoE-Style Plugin, ACL 2024.
-
Mitigate Negative Transfer with Similarity Heuristic Lifelong Prompt Tuning, ACL (Findings) 2024.
-
Q-Tuning: Queue-based Prompt Tuning for Lifelong Few-shot Language Learning, NAACL(Findings) 2024.
-
Sequential Editing for Lifelong Training of Speech Recognition Models, Preprint 2024.
[pdf]
-
Lifelong Knowledge Editing for LLMs with Retrieval-Augmented Continuous Prompt Learning, Preprint 2024.
-
Detoxifying Large Language Models via Knowledge Editing, ACL 2024.
-
Model Editing at Scale leads to Gradual and Catastrophic Forgetting, ACL (Findings) 2024.
-
Can We Continually Edit Language Models? On the Knowledge Attenuation in Sequential Model Editing, ACL (Findings) 2024.
-
Incremental Sequential Labeling: A Tale of Two Shifts, ACL 2024 (Findings).
[pdf]
-
HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models, Preprint 2024.
[pdf] [Code]
-
Towards Practical Tool Usage for Continually Learning LLMs, Preprint 2024.
[pdf]
-
LoRA Learns Less and Forgets Less, Preprint 2024.
[pdf]
-
CorpusBrain++: A Continual Generative Pre-Training Framework for Knowledge-Intensive Language Tasks, Preprint 2024.
-
Examining Forgetting in Continual Pre-training of Aligned Large Language Models, Preprint 2024.
-
Rho-1: Not All Tokens Are What You Need, Preprint 2024.
-
SOLAR 10.7B: Scaling Large Language Models with Simple yet Effective Depth Up-Scaling, NAACL 2024.
-
LLaMA Pro: Progressive LLaMA with Block Expansion, Preprint 2024.
-
Embracing Language Inclusivity and Diversity in CLIP through Continual Language Learning, AAAI 2024.
-
Simple and Scalable Strategies to Continually Pre-train Large Language Models, Preprint 2024.
-
Set the Clock: Temporal Alignment of Pretrained Language Models, ACL (Findings) 2024.
[pdf] [Code]
-
HOP to the Next Tasks and Domains for Continual Learning in NLP, AAAI 2024.
-
Bayesian Parameter-Efficient Fine-Tuning for Overcoming Catastrophic Forgetting, Preprint 2024.
-
Rehearsal-Free Modular and Compositional Continual Learning for Language Models, NAACL 2024.
-
Few-shot Incremental Event Detection, TALLIP 2024.
-
Self-generated Replay Memories for Continual Neural Machine Translation, Preprint 2024.
-
F-MALLOC: Feed-forward Memory Allocation for Continual Learning in Neural Machine Translation, NAACL 2024.
-
SAPT: A Shared Attention Framework for Parameter-Efficient Continual Learning of Large Language Models, Preprint 2024.
-
InsCL: A Data-efficient Continual Learning Paradigm for Fine-tuning Large Language Models with Instructions, NAACL 2024.
-
Analyzing and Reducing Catastrophic Forgetting in Parameter Efficient Tuning, Preprint 2024.
-
Mitigating Catastrophic Forgetting in Large Language Models with Self-Synthesized Rehearsal, ACL 2024.
-
Scalable Language Model with Generalized Continual Learning, ICLR 2024.
-
Q-Tuning: Queue-based Prompt Tuning for Lifelong Few-shot Language, NAACL (Findings) 2024.
-
MoRAL: MoE Augmented LoRA for LLMs' Lifelong Learning, Preprint 2024.
-
WilKE: Wise-Layer Knowledge Editor for Lifelong Knowledge Editing, Preprint 2024.
-
CPPO: Continual Learning for Reinforcement Learning with Human Feedback, ICLR 2024.
[pdf] [Code]
-
COPR: Continual Human Preference Learning via Optimal Policy Regularization, Preprint 2024.
-
Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection, ICLR 2024.
-
LLMs in the Imaginarium: Tool Learning through Simulated Trial and Error, Preprint 2024.
-
Towards Practical Tool Usage for Continually Learning LLMs, Preprint 2024.
-
Confucius: Iterative Tool Learning from Introspection Feedback by Easy-to-Difficult Curriculum, AAAI 2024.
-
Toolllm: Facilitating large language models to master 16000+ real-world apis, ICLR 2024.
-
Continual Pre-Training of Large Language Models: How to (re)warm your model?, ICML (Workshop) 2023.
-
EcomGPT-CT: Continual Pre-training of E-commerce Large Language Models with Semi-structured Data, Preprint 2023.
-
QUERT: Continual Pre-training of Language Model for Query Understanding in Travel Domain Search, KDD 2023.
-
Recyclable Tuning for Continual Pre-training, ACL (Findings) 2023.
-
Large Language Models Encode Clinical Knowledge, Nature 2023.
-
Continual Learning Under Language Shift, Preprint 2023.
-
Exploring Continual Learning for Code Generation Models, ACL 2023.
-
Lifelong Language Pretraining with Distribution-Specialized Experts, ICML 2023.
-
Mitigating Catastrophic Forgetting in Task-Incremental Continual Learning with Adaptive Classification Criterion, Preprint 2023.
-
Rehearsal-free Continual Language Learning via Efficient Parameter Isolation, ACL 2023.
-
Class-Incremental Learning based on Label Generation, ACL 2023.
-
Overcoming Catastrophic Forgetting in Massively Multilingual Continual Learning, ACL (Findings) 2023.
-
InfoCL: Alleviating Catastrophic Forgetting in Continual Text Classification from An Information Theoretic Perspective, EMNLP 2023.
-
ConPET: Continual Parameter-Efficient Tuning for Large Language Models, Preprint 2023.
-
A Neural Span-Based Continual Named Entity Recognition Model, AAAI 2023.
-
Learning βOβ Helps for Learning More: Handling the Unlabeled Entity Problem for Class-incremental NER, ACL 2023.
-
Teamwork Is Not Always Good: An Empirical Study of Classifier Drift in Class-incremental Information Extraction, ACL (Findings) 2023.
-
ProtoNER: Few Shot Incremental Learning for Named Entity Recognition Using Prototypical Networks, BPM 2023.
-
Task Relation Distillation and Prototypical Pseudo Label for Incremental Named Entity Recognition, CIKM 2023.
-
Continual Named Entity Recognition without Catastrophic Forgetting, CIKM 2023.
-
SKD-NER:Continual Named Entity Recognition via Span-based Knowledge Distillation with Reinforcement Learning, EMNLP 2023.
-
Novel Slot Detection With an Incremental Setting, EMNLP (Findings) 2023.
-
Incremental event detection via an improved knowledge distillation based model, Neurocomputing 2023.
-
Decomposing Logits Distillation for Incremental Named Entity Recognition, SIGIR 2023.
-
Consistent Prototype Learning for Few-Shot Continual Relation Extraction, ACL 2023.
-
Enhancing Continual Relation Extraction via Classifier Decomposition, ACL (Findings) 2023.
-
Improving Continual Relation Extraction by Distinguishing Analogous Semantics, ACL 2023.
-
Serial Contrastive Knowledge Distillation for Continual Few-shot Relation Extraction, ACL (Findings) 2023.
-
ICA-Proto: Iterative Cross Alignment Prototypical Network for Incremental Few-Shot Relation Classification, EACL (Findings) 2023.
-
Serial Contrastive Knowledge Distillation for Continual Few-shot Relation Extraction, ACL (Findings) 2023.
-
Continual Knowledge Distillation for Neural Machine Translation, ACL 2023.
-
Knowledge Transfer in Incremental Learning for Multilingual Neural Machine Translation, ACL 2023.
-
Continual Learning for Multilingual Neural Machine Translation via Dual Importance-based Model Division, EMNLP 2023.
-
Prompts Can Play Lottery Tickets Well: Achieving Lifelong Information Extraction via Lottery Prompt Tuning, ACL 2023.
-
Large-scale Lifelong Learning of In-context Instructions and How to Tackle It, ACL 2023.
-
Generative Replay Inspired by Hippocampal Memory Indexing for Continual Language Learning, EACL 2023.
-
Lifelong Sequence Generation with Dynamic Module Expansion and Adaptation, EMNLP 2023.
-
Orthogonal Subspace Learning for Language Model Continual Learning, EMNLP (Findings) 2023.
-
Sub-network Discovery and Soft-masking for Continual Learning of Mixed Tasks, EMNLP (Findings) 2023.
-
Progressive Prompts: Continual Learning for Language Models, ICLR 2023.
-
Methods for Measuring, Updating, and Visualizing Factual Beliefs in Language Models, EACL 2023.
-
Aging with GRACE: Lifelong Model Editing with Key-Value Adaptors, ICLR 2023.
-
Transformer-Patcher: One Mistake worth One Neuron, ICLR 2023.
-
Mitigating the Alignment Tax of RLHF, Preprint 2023.
-
Tree of Clarifications: Answering Ambiguous Questions with Retrieval-Augmented Large Language Models, EMNLP 2023.
-
Active Retrieval Augmented Generation, EMNLP 2023.
-
Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions, ACL 2023.
-
Toolalpaca: Generalized tool learning for language models with 3000 simulated cases, Preprint 2023.
-
Gorilla: Large language model connected with massive apis, Preprint 2023.
-
Chameleon: Plug-and-play compositional reasoning with large language models, NIPS 2023.
-
Toolformer: Language Models Can Teach Themselves to Use Tools, NIPS 2023.
-
GPT4Tools: Teaching Large Language Model to Use Tools via Self-instruction, NIPS 2023.
-
Continual Pre-Training Mitigates Forgetting in Language and Vision, Preprint 2022.
-
ELLE: Efficient Lifelong Pre-training for Emerging Data, ACL (Findings) 2022.
-
Lifelong Pretraining: Continually Adapting Language Models to Emerging Corpora, ACL (Workshop) 2022.
-
bert2BERT: Towards Reusable Pretrained Language Models, ACL 2022.
-
Time Waits for No One! Analysis and Challenges of Temporal Misalignment, NAACL 2022.
-
TimeLMs: Diachronic Language Models from Twitter, ACL 2022.
-
Continual Few-shot Intent Detection, COLING 2022.
-
Incremental Intent Detection for Medical Domain with Contrast Replay Networks, ACL (Findings) 2022.
-
Continual training of language models for few-shot learning, EMNLP 2022.
-
Parameter-efficient Continual Learning Framework in Industrial Real-time Text Classification System, NAACL 2022.
-
Prompt Augmented Generative Replay via Supervised Contrastive Learning for Lifelong Intent Detection, NAACL (Findings) 2022.
-
Memory Efficient Continual Learning with Transformers, NIPS 2022.
-
Few-Shot Class-Incremental Learning for Named Entity Recognition, ACL 2022.
-
Learn and Review: Enhancing Continual Named Entity Recognition via Reviewing Synthetic Samples, ACL (Findings) 2022.
-
Incremental Prompting: Episodic Memory Prompt for Lifelong Event Dectection, COLING 2022.
-
Distilling Causal Effect from Miscellaneous Other-Class for Continual Named Entity Recognition, EMNLP 2022.
-
BNU: A Balance-Normalization-Uncertainty Model for Incremental Event Detection, ICASSP 2022.
-
Similarity-Driven Adaptive Prototypical Network for Class-incremental Few-shot Named Entity Recognition, ICTAI 2022.
-
HEFT: A History-Enhanced Feature Transfer framework for incremental event detection, KBS 2022.
-
Consistent Representation Learning for Continual Relation Extraction, ACL (Findings) 2022.
-
Continual Few-shot Relation Learning via Embedding Space Regularization and Data Augmentation, ACL 2022.
-
Less is More: Rethinking State-of-the-art Continual Relation Extraction Models with a Frustratingly Easy but Effective Approach, Preprint 2022.
-
Improving continual relation extraction through prototypical contrastive learning, COLING 2022.
-
Learning Robust Representations for Continual Relation Extraction via Adversarial Class Augmentation, EMNLP 2022.
-
Prompt-based prototypical framework for continual relation extraction, TASLP 2022.
-
Overcoming Catastrophic Forgetting beyond Continual Learning: Balanced Training for Neural Machine Translation, ACL 2022.
-
CLLE: A Benchmark for Continual Language Learning Evaluation in Multilingual Machine Translation, EMNLP (Findings) 2022.
-
Continual Learning of Neural Machine Translation within Low Forgetting Risk Regions, EMNLP 2022.
-
Entropy-Based Vocabulary Substitution for Incremental Learning in Multilingual Neural Machine Translation, EMNLP 2022.
-
Continual Sequence Generation with Adaptive Compositional Modules, ACL 2022.
-
ConTinTin: Continual Learning from Task Instructions, ACL 2022.
-
Continual Prompt Tuning for Dialog State Tracking, ACL 2022.
-
Prompt Conditioned VAE: Enhancing Generative Replay for Lifelong Learning in Task-Oriented Dialogue, EMNLP 2022.
-
Fine-tuned Language Models are Continual Learners, EMNLP 2022.
-
LFPT5: A Unified Framework for Lifelong Few-shot Language Learning Based on Prompt Tuning of T5, ICLR 2022.
-
Plug-and-Play Adaptation for Continuously-updated QA, ACL (Findings) 2022.
-
Temporal Adaptation of BERT and Performance on Downstream Document Classification: Insights from Social Media, EMNLP (Findings) 2021.
-
ECONET: Effective Continual Pretraining of Language Models for Event Temporal Reasoning, EMNLP 2021.
-
Mind the Gap: Assessing Temporal Generalization in Neural Language Models, NIPS 2021.
-
Learning to Solve NLP Tasks in an Incremental Number of Languages, ACL 2021.
-
Adapting BERT for Continual Learning of a Sequence of Aspect Sentiment Classification Tasks, NAACL 2021.
-
CLASSIC: Continual and Contrastive Learning of Aspect Sentiment Classification Tasks, EMNLP 2021.
-
Continual Learning for Text Classification with Information Disentanglement Based Regularization, NAACL 2021.
-
Hyperparameter-free Continuous Learning for Domain Classification in Natural Language Understanding, NAACL 2021.
-
Incremental Few-shot Text Classification with Multi-round New Classes: Formulation, Dataset and System, NAACL 2021.
-
Achieving Forgetting Prevention and Knowledge Transfer in Continual Learning, NIPS 2021.
-
Iterative Network Pruning with Uncertainty Regularization for Lifelong Sentiment Classification, SIGIR 2021.
-
Lifelong Intent Detection via Multi-Strategy Rebalancing, Preprint 2021.
-
Lifelong Knowledge-Enriched Social Event Representation Learning, EACL 2021.
-
Lifelong Learning of Hate Speech Classification on Social Media, NAACL 2021.
-
Continual Learning for Named Entity Recognition, AAAI 2021.
-
Lifelong Event Detection with Knowledge Transfer, EMNLP 2021.
-
Curriculum-meta learning for order-robust continual relation extraction, AAAI 2021.
-
Refining Sample Embeddings with Relation Prototypes to Enhance Continual Relation Extraction, ACL 2021.
-
Continual learning in multilingual nmt via language-specific embeddings, WMT 2021.
-
Continual Learning for Neural Machine Translation, NAACL 2021.
-
Towards Continual Learning for Multilingual Machine Translation via Vocabulary Substitution, NAACL 2021.
-
Rational LAMOL: A rationale-based lifelong learning framework, ACL 2021.
-
Continual Learning for Task-oriented Dialogue System with Iterative Network Pruning, Expanding and Masking, ACL 2021.
-
Learn Continually, Generalize Rapidly: Lifelong Knowledge Accumulation for Few-shot Learning, EMNLP (Findings) 2021.
-
Continual Learning in Task-Oriented Dialogue Systems, EMNLP 2021.
-
Incremental Event Detection via Knowledge Consolidation Networks, EMNLP 2020.
-
Continual Relation Learning via Episodic Memory Activation and Reconsolidation, ACL 2020.
-
Findings of the First Shared Task on Lifelong Learning Machine Translation, WMT 2020.
-
Distill and Replay for Continual Language Learning, COLING 2020.
-
Continual Learning for Natural Language Generation in Task-oriented Dialog Systems, EMNLP (Findings) 2020.
-
LAMOL: LAnguage MOdeling for Lifelong Language Learning, ICLR 2020.
-
Dense Passage Retrieval for Open-Domain Question Answering, EMNLP 2020.
-
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, NIPS 2020.
-
A Progressive Model to Enable Continual Learning for Semantic Slot Filling, EMNLP 2019.
-
Meta-Learning Improves Lifelong Relation Extraction, RepL4NLP 2019.
-
Sentence Embedding Alignment for Lifelong Relation Extraction, NAACL 2019.
-
From Bilingual to Multilingual Neural Machine Translation by Incremental Training, JASIST 2019.
-
Incremental Learning from Scratch for Task-Oriented Dialogue Systems, ACL 2019.