RAG (Retrieval-Augmented Generation)
RAG is a technique that fetches relevant facts from your own data at query time and feeds them into a language model before it answers, so responses are grounded in your content rather than the model's memory. It is the standard way to make an LLM answer accurately from private or up-to-date information.
A bare LLM only knows what was in its training data, frozen at a cutoff date and unaware of your documents. RAG fixes this by splitting your content into chunks, storing them as embeddings in a vector database, retrieving the closest matches to a user's question, and passing them to the model as context. The model then answers from that retrieved evidence instead of guessing, which sharply reduces hallucination and lets you cite sources.
When you are building a product, RAG is usually the first thing to reach for before anyone suggests fine-tuning: it is cheaper, your content stays current the moment you update it, and you can show users exactly which source backed each answer. The hard parts are not the model but the retrieval — good chunking, the right embedding model, and evals that catch when the system returns confident answers from the wrong passage. We treat retrieval quality as the real engineering work.
// faq
Frequently asked questions
- Is RAG better than fine-tuning?
- For answering from a body of facts that changes over time, usually yes. RAG keeps knowledge in a database you can edit instantly and lets you cite sources, while fine-tuning bakes knowledge into model weights and has to be redone when facts change. Fine-tuning is better for teaching style, format, or a narrow task — the two are often combined, not opposed.
- Why does my RAG system still give wrong answers?
- Almost always retrieval, not the model. If the right passage never makes it into the context, the model cannot use it. Common causes: chunks too large or too small, a weak embedding model, no reranking, or missing evals to measure retrieval hit-rate. Fix the retrieval pipeline first and measure it before touching the prompt.