Retrieval-Augmented Methods (RAG)

Gao et al 2023 - Retrieval-Augmented Generation for Large Language Models: A Survey

Lewis et al 2020 - Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks

Liu et al 2021 - What Makes Good In-Context Examples for GPT-3? Cited by Rubin 2021

Rubin et al 2021 - Learning To Retrieve Prompts for In-Context Learning Is this the first retrieval-based in-context learning paper?

Poesia et al 2022 - Synchromesh: Reliable code generation from pre-trained language models One of the first retrieval-based in-context learning papers

Wang et al 2022 - Training Data is More Valuable than You Think: A Simple and Effective Method by Retrieving from Training Data

Jiang et al 2023 - Active Retrieval Augmented Generation

Izacard et al 2022 - Few-shot Learning with Retrieval Augmented Language Models Learns the retrieval model

Lin et al 2023 - RA-DIT: Retrieval-Augmented Dual Instruction Tuning Fine-tunes a language model so it's better for retrieval augmented use. Best performance occurs after a small number of fine-tuning steps (<500 steps!).

Zhang et al 2024 - Accelerating Retrieval-Augmented Language Model Serving with Speculation

Zheng et al 2025 - Worse than Zero-shot? A Fact-Checking Dataset for Evaluating the Robustness of RAG Against Misleading Retrievals

Guu et al 2020 - REALM: Retrieval-Augmented Language Model Pre-Training

RETRO: Borgeaud et al 2021 - Improving Language Models by Retrieving from Trillions of Tokens

Wang et al 2023 - Shall We Pretrain Autoregressive Language Models with Retrieval? A Comprehensive Study

Language Model

Prompting