Differences

This shows you the differences between two versions of the page.

--- ml:catastrophic_forgetting [2025/02/01 00:43] – [In NLP] jmflanig
+++ ml:catastrophic_forgetting [2025/06/14 00:32] (current) – [Papers] jmflanig
@@ Line 8: / Line 8: @@
 ===== Papers =====
+  * EWC: [[https://arxiv.org/pdf/1612.00796|Kirkpatrick et al 2016 - Overcoming catastrophic forgetting in neural networks]] A famous method
+  * [[https://arxiv.org/pdf/1711.05769|Mallya Lazebnik 2017 - PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning]]
 ==== In NLP ====
@@ Line 13: / Line 16: @@
   * [[https://arxiv.org/pdf/2309.10105|2023 - Understanding Catastrophic Forgetting in Language Models via Implicit Inference]]
   * [[https://openreview.net/pdf?id=tmsqb6WpLz|Zhang & Wu 2024 - Dissecting learning and forgetting in language model finetuning]]
-  * [[https://arxiv.org/pdf/2501.13453|Zheng et al 2025 - Spurious Forgetting in Continual Learning of Language Models]]
+  * [[https://arxiv.org/pdf/2501.13453|Zheng et al 2025 - Spurious Forgetting in Continual Learning of Language Models]] Argues that the drop in performance while doing continued training "often reflects a decline in task alignment rather than knowledge loss"
 ===== Related Pages =====
   * [[Continual Learning]]