Differences

This shows you the differences between two versions of the page.

--- ml:catastrophic_forgetting [2021/05/19 12:20] – jmflanig
+++ ml:catastrophic_forgetting [2025/06/14 00:32] (current) – [Papers] jmflanig
@@ Line 1: / Line 1: @@
 ====== Catastrophic Forgetting ======
-Catastrophic forgetting is when a neural network is trained to do a first task but loses that ability when trained to do a second task.
+//Catastrophic forgetting// is when a neural network is trained to do one task but loses that ability when trained to do a second task.
+===== Overviews =====
   * [[https://arxiv.org/pdf/1612.00796.pdf|Kirkpatrick et al 2016 - Overcoming catastrophic forgetting in neural networks]]
+  * [[https://arxiv.org/pdf/2307.09218|Wang et al 2023 - A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learning]]
+  * [[https://arxiv.org/pdf/2312.10549|2023 - Catastrophic Forgetting in Deep Learning: A Comprehensive Taxonomy]]
+===== Papers =====
+  * EWC: [[https://arxiv.org/pdf/1612.00796|Kirkpatrick et al 2016 - Overcoming catastrophic forgetting in neural networks]] A famous method
+  * [[https://arxiv.org/pdf/1711.05769|Mallya Lazebnik 2017 - PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning]]
+==== In NLP ====
+  * [[https://aclanthology.org/2022.naacl-main.398.pdf|Li et al 2022 - Overcoming Catastrophic Forgetting During Domain Adaptation of Seq2seq Language Generation]]
+  * [[https://arxiv.org/pdf/2309.10105|2023 - Understanding Catastrophic Forgetting in Language Models via Implicit Inference]]
+  * [[https://openreview.net/pdf?id=tmsqb6WpLz|Zhang & Wu 2024 - Dissecting learning and forgetting in language model finetuning]]
+  * [[https://arxiv.org/pdf/2501.13453|Zheng et al 2025 - Spurious Forgetting in Continual Learning of Language Models]] Argues that the drop in performance while doing continued training "often reflects a decline in task alignment rather than knowledge loss"
+===== Related Pages =====
+  * [[Continual Learning]]