RAG (Retrieval-Augmented Generation)

A language model on its own answers purely from patterns learned during training, which means it has no access to information created after its training cutoff and no visibility into a specific business's private documents, product catalog, or support history. RAG addresses this by adding a retrieval step before generation: when a question comes in, the system searches an external knowledge base — commonly a vector database holding chunked, embedded versions of documents — for the most relevant passages, and passes those passages to the model alongside the original question.

The model then generates its answer using that retrieved content as grounding context, rather than relying solely on what it memorized during training. This is the architecture behind most production AI chatbots that need to answer accurately from a specific company's documentation, policies, or product data, and it's also a major factor in why well-structured, clearly written source content — the kind organized around clean headings and factual, specific statements — tends to get retrieved and cited more reliably than vague or poorly organized pages.

RAG measurably reduces (though doesn't eliminate) hallucination compared to an ungrounded model, because the model has actual source material to draw from instead of generating a plausible-sounding guess. It doesn't replace the need for accurate underlying data, though — a retrieval system confidently pulling from outdated or wrong source documents will ground the answer just as confidently in that wrong information.