RAG explained: how AI answers from your documents instead of making things up
The technique behind every 'chat with your documents' product, explained without the jargon, because if you buy or build one, you should know how it works and where it breaks.
Ask a bare language model about your business and it will answer fluently and, quite possibly, wrongly: it has never seen your contracts, prices or policies. RAG (retrieval-augmented generation) is the standard fix, and the idea is simple enough to explain in a paragraph.
How does RAG work, in one paragraph?
When you ask a question, the system doesn't send it straight to the model. First it retrieves: searches your documents (files, emails, records) for the passages most relevant to the question. Then it hands the model your question plus those passages and instructs it: answer from this material and cite it. The model becomes a reader of your business rather than a guesser about it. Ask "what's our cancellation policy for trade customers?" and the answer comes from your actual terms document, with a link to it.
Why should an AI answer always show its sources?
A RAG answer arrives with receipts: the documents it drew from. That changes the trust equation: you don't have to believe the AI, you can check it in one click. For anything commercial (quotes, policies, client history), answers without sources shouldn't be trusted, and any system that can't show its sources isn't fit for business use. This is the backbone of what M22 calls a Company Brain.
Where does RAG go wrong?
- Retrieval misses. If the search step fetches the wrong passages, the model answers correctly from irrelevant material: confident and useless. Retrieval quality, not model quality, is behind many bad answers.
- Stale documents. RAG answers from what's indexed. If the 2023 price list is in and the 2026 one isn't, you'll quote 2023 prices fluently. Freshness pipelines matter.
- Permissions ignored. Naive RAG searches everything for everyone, meaning anyone can ask their way into salaries or legal disputes. Permission-aware retrieval is non-negotiable and is real engineering.
- Contradictory sources. Businesses accumulate conflicting documents. Good systems surface the conflict; bad ones pick one silently.
What should you ask a RAG vendor?
"How does retrieval respect permissions?" · "How fresh is the index?" · "Show me an answer with its sources." · "What happens when it doesn't know?" The last one is telling: the right behaviour is saying so; an honest "I can't find this" beats a fluent invention every time.
How does M22 help with this?
The Company Brain M22 Consultancy builds is retrieval done properly: permission-aware, so people only see answers drawn from what they are allowed to see, and every answer links back to the document it came from. M22 treats retrieval quality, not model choice, as the engineering problem worth solving, which is why an M22 build starts with your sources and your permission model rather than the model itself. The AI Audit starts at £1,500, a fixed fee agreed before day one. Book a thirty-minute call, or see how a Company Brain compares to other providers.