RAG when the model lacks knowledge: facts that change, are private, or need a citation. Fine-tuning when it lacks behaviour: a format, a tone, a narrow task. An index is cheaper to update than weights, so reach for RAG first.
Card 1 of 292. Answer: RAG when the model lacks knowledge: facts that change, are private, or need a citation. Fine-tuning when it lacks behaviour: a format, a tone, a narrow task. An index is cheaper to update than weights, so reach for RAG first.
About · Quizzes · History of AI · To suggest new questions or report an error send me a dm.
Use RAG when the model lacks knowledge, such as changing facts, private documents or answers that need a citation, and fine-tune when it lacks a behaviour, such as a format, tone or narrow task. Retrieval-augmented generation fetches relevant passages at query time and puts them in the prompt; fine-tuning trains on your examples and changes the weights.
Take an HR assistant over 2,000 policies, 50 of which change each month. With RAG, an update is re-embedding 50 documents, answers cite the section used, and documents a user may not see are filtered out at retrieval. With fine-tuning, each update means new training data and a retrain, the model may blend old and new rules, and a withdrawn policy cannot be reliably erased from the weights.
Interviewers then ask whether fine-tuning can teach facts. Poorly: in "Fine-Tuning or Retrieval?" (Ovadia et al., EMNLP 2024), RAG consistently beat unsupervised fine-tuning for both familiar and new facts. The two combine well: fine-tune for format, retrieve for facts. RAG's costs are latency, longer prompts and dependence on retrieval quality.
