My understanding
RAG combines two kinds of memory. Its parametric memory is a pretrained sequence-to-sequence model, while its non-parametric memory is a dense index of Wikipedia passages accessed through a neural retriever. Given a question or prompt, the retriever proposes relevant passages and the generator uses them as additional context when producing an answer.
The paper studies two variants. RAG-Sequence uses the same retrieved document to condition the whole output sequence, whereas RAG-Token can draw on different documents for different generated tokens. Both components can be fine-tuned together for downstream tasks without retraining the retrieval index itself.
Across knowledge-intensive tasks, this approach improved open-domain question answering and produced language that was more specific, diverse, and factual than a parametric-only baseline.
What changed in my thinking
I used to think of retrieval mostly as a way to supply extra context to a model. This paper makes the stronger architectural point: retrieval is a separate form of memory. It reduces the pressure to encode every fact in model weights and creates a path for updating knowledge without training a new generator from scratch.
It also clarified that RAG is not only a search step followed by prompting. The retriever and generator form one probabilistic system, and the uncertainty over retrieved documents is part of the generation process.
Questions and limitations
- How much of the reported improvement comes from retrieval quality versus the generator learning to use retrieved evidence well?
- What happens when the corpus is incomplete, outdated, biased, or contains conflicting passages?
- Retrieved evidence makes provenance possible, but it does not guarantee that the final answer is supported by that evidence.
- Dense retrieval and a large external index add latency, infrastructure cost, and another failure surface compared with a parametric-only model.
Connections
This paper is the conceptual foundation for many modern RAG systems built around large language models. The same separation between reasoning and external memory is especially useful in domains such as medicine, research, and decision support, where sources need to be current, inspectable, and replaceable.