Differences

This shows you the differences between two versions of the page.

--- nlp:retrieval-augmented_methods [2023/11/30 05:34] – [During Pre-Training or Fine-Tuning] jmflanig
+++ nlp:retrieval-augmented_methods [2025/05/13 19:39] (current) – [Papers] jmflanig
@@ Line 1: / Line 1: @@
-====== Retrieval-Augmented Methods ======
+====== Retrieval-Augmented Methods (RAG) ======
+===== Overviews =====
+  * [[https://arxiv.org/pdf/2312.10997|Gao et al 2023 - Retrieval-Augmented Generation for Large Language Models: A Survey]]
 ===== Papers =====
+  * **[[https://arxiv.org/pdf/2005.11401.pdf|Lewis et al 2020 - Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks]]**
   * [[https://arxiv.org/pdf/2101.06804.pdf|Liu et al 2021 - What Makes Good In-Context Examples for GPT-3?]] Cited by [[https://arxiv.org/pdf/2112.08633.pdf|Rubin 2021]]
   * [[https://arxiv.org/pdf/2112.08633.pdf|Rubin et al 2021 - Learning To Retrieve Prompts for In-Context Learning]] Is this the first retrieval-based in-context learning paper?
   * [[https://arxiv.org/pdf/2201.11227.pdf|Poesia et al 2022 - Synchromesh: Reliable code generation from pre-trained language models]] One of the first retrieval-based in-context learning papers
   * [[https://arxiv.org/pdf/2203.08773.pdf|Wang et al 2022 - Training Data is More Valuable than You Think: A Simple and Effective Method by Retrieving from Training Data]]
+  * [[https://arxiv.org/pdf/2305.06983.pdf|Jiang et al 2023 - Active Retrieval Augmented Generation]]
   * [[https://arxiv.org/pdf/2208.03299.pdf|Izacard et al 2022 - Few-shot Learning with Retrieval Augmented Language Models]] Learns the retrieval model
-  * **[[https://arxiv.org/pdf/2310.01352.pdf|Lin et al 2023 - RA-DIT: Retrieval-Augmented Dual Instruction Tuning]]** Fine-tunes a language model to it's better for retrieval augmented use.  Best performance occurs after a small number of fine-tuning steps (<500 steps!).
+  * **[[https://arxiv.org/pdf/2310.01352.pdf|Lin et al 2023 - RA-DIT: Retrieval-Augmented Dual Instruction Tuning]]** Fine-tunes a language model so it's better for retrieval augmented use.  Best performance occurs after a small number of fine-tuning steps (<500 steps!).
+  * [[https://arxiv.org/pdf/2401.14021.pdf|Zhang et al 2024 - Accelerating Retrieval-Augmented Language Model Serving with Speculation]]
+  * [[https://arxiv.org/pdf/2502.16101|Zheng et al 2025 - Worse than Zero-shot? A Fact-Checking Dataset for Evaluating the Robustness of RAG Against Misleading Retrievals]]
-==== During Pre-Training or Fine-Tuning ====
+==== During Pre-Training ====
+  * [[https://arxiv.org/pdf/2002.08909.pdf|Guu et al 2020 - REALM: Retrieval-Augmented Language Model Pre-Training]]
   * RETRO: [[https://arxiv.org/pdf/2112.04426.pdf|Borgeaud et al 2021 - Improving Language Models by Retrieving from Trillions of Tokens]]
   * [[https://arxiv.org/pdf/2304.06762.pdf|Wang et al 2023 - Shall We Pretrain Autoregressive Language Models with Retrieval? A Comprehensive Study]]
 ===== Related Pages =====
+  * [[Language Model]]
   * [[Prompting]]