====== Retrieval-Augmented Methods (RAG) ======

===== Overviews =====
  * [[https://arxiv.org/pdf/2312.10997|Gao et al 2023 - Retrieval-Augmented Generation for Large Language Models: A Survey]]

===== Papers =====
  * **[[https://arxiv.org/pdf/2005.11401.pdf|Lewis et al 2020 - Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks]]**
  * [[https://arxiv.org/pdf/2101.06804.pdf|Liu et al 2021 - What Makes Good In-Context Examples for GPT-3?]] Cited by [[https://arxiv.org/pdf/2112.08633.pdf|Rubin 2021]]
  * [[https://arxiv.org/pdf/2112.08633.pdf|Rubin et al 2021 - Learning To Retrieve Prompts for In-Context Learning]] Is this the first retrieval-based in-context learning paper?
  * [[https://arxiv.org/pdf/2201.11227.pdf|Poesia et al 2022 - Synchromesh: Reliable code generation from pre-trained language models]] One of the first retrieval-based in-context learning papers
  * [[https://arxiv.org/pdf/2203.08773.pdf|Wang et al 2022 - Training Data is More Valuable than You Think: A Simple and Effective Method by Retrieving from Training Data]]
  * [[https://arxiv.org/pdf/2305.06983.pdf|Jiang et al 2023 - Active Retrieval Augmented Generation]]
  * [[https://arxiv.org/pdf/2208.03299.pdf|Izacard et al 2022 - Few-shot Learning with Retrieval Augmented Language Models]] Learns the retrieval model
  * **[[https://arxiv.org/pdf/2310.01352.pdf|Lin et al 2023 - RA-DIT: Retrieval-Augmented Dual Instruction Tuning]]** Fine-tunes a language model so it's better for retrieval augmented use.  Best performance occurs after a small number of fine-tuning steps (<500 steps!).
  * [[https://arxiv.org/pdf/2401.14021.pdf|Zhang et al 2024 - Accelerating Retrieval-Augmented Language Model Serving with Speculation]]
  * [[https://arxiv.org/pdf/2502.16101|Zheng et al 2025 - Worse than Zero-shot? A Fact-Checking Dataset for Evaluating the Robustness of RAG Against Misleading Retrievals]]

==== During Pre-Training ====
  * [[https://arxiv.org/pdf/2002.08909.pdf|Guu et al 2020 - REALM: Retrieval-Augmented Language Model Pre-Training]]
  * RETRO: [[https://arxiv.org/pdf/2112.04426.pdf|Borgeaud et al 2021 - Improving Language Models by Retrieving from Trillions of Tokens]]
  * [[https://arxiv.org/pdf/2304.06762.pdf|Wang et al 2023 - Shall We Pretrain Autoregressive Language Models with Retrieval? A Comprehensive Study]]

===== Related Pages =====
  * [[Language Model]]
  * [[Prompting]]