RAG: the enterprise pattern

Retrieval-Augmented Generation means: find the relevant chunks, then generate an answer conditioned on them. The name is one line. The work is a pipeline. Ingest: load docs, clean junk, split, attach metadata. Index: embed and store. Retrieve: hybrid search + filters + maybe rerank. Generate: model writes using only (or mostly) those chunks. Cite: each claim points at a chunk. UI: the user can open the source. Evaluate: retrieval recall and answer groundedness, separately.

If you cannot draw those boxes with an owner on each, you do not have a RAG program. You have a vendor checkbox. A RACI that says 'AI team' on every box is a failed RACI. Data / knowledge owner on ingest. Engineer on index and client. You on eval questions, generator contract, and go-live. Security on ACL. Product on the UI and the 'not a decision' footer. Sponsor on corpus scope and the no-go.

Ingest is where most timelines actually go. Connectors, permissions, junk, OCR, the argument about which Confluence spaces are in. Treat ingest as a workstream with a sample-chunk review gate, not as 'we'll hook up the loader on Thursday.' Day 10's chunking spec lives here. If ingest is unbounded ('all of Confluence'), the rest of the pipeline is theatre.

Index is operations: which vector store, which embedder, how filters work, how upserts happen when a page changes. You do not need to pick the brand in the first workshop. You do need a re-index SLA and a permission story. 'We'll rebuild nightly' is a lag you must print in the UI. 'We'll rebuild when we remember' is not an SLA.

Retrieve is yesterday. Hybrid, filters, k, eval. The only new point today: retrieve is allowed to return empty. Empty is a success path. It must be passed to generate as empty, not replaced with 'the closest 5 anyway.' Closest-anyway is how you get salary-adjacent answers from a project wiki.

Generate and cite are the product the user sees. They are the smallest part of the code and the largest part of the trust. The next concepts are those boxes in slow motion. Do not skip to them and ignore ingest. Sponsors will try to. Your job is the whole pipe.

Evaluate is two numbers, always: recall@k on retrieval, groundedness / citation correctness on the answer. They fail independently. High recall and a wandering generator still ships lies. A loyal generator on the wrong chunks still ships the wrong SOP. If a vendor reports one blended 'quality' score, ask them to split it. If they cannot, they are selling a demo.

RAG is a pipeline with a name on each box or it is a vendor checkbox. Ingest has a knowledge owner and a sample-chunk gate. Index has a re-index SLA. Retrieve is allowed to return empty. Generate is a prisoner of the packet. Cite deep-links. UI shows the source. Evaluate splits recall from groundedness. A RACI that says 'AI team' on every box is a failed RACI. Draw the boxes until you can staff them. If you cannot, you are not ready to ingest, and you should not be in a workshop pretending you are. The generator is the part everyone wants to talk about. The pipeline is the part that actually fails. Spend your words on ingest and retrieve even when the sponsor is already naming models.

The system prompt must say: answer only from the provided sources; quote or cite; if sources don't cover it, say you don't know and suggest who to ask. Without that, RAG becomes search plus hallucination with a bibliography of unrelated files. The model is a prisoner. The packet is the cell. If the cell is empty, the prisoner does not get to wander the building.

Stuffing 20 chunks 'to be safe' dilutes the good ones and blows cost. 3–8 good chunks beat 20 mediocre. That is why retrieval eval comes first. If recall@5 is 40%, adding 15 more neighbors will not find a document that was never a neighbor. It will find more junk for the model to quote. k is a product knob you set from the eval, not from anxiety.

The user packet you send is a structured object, not a pile. Typical shape: the question, the retrieved chunks labeled [S1], [S2] with source metadata, the refuse rule, the citation rule, the output contract (short answer, then sources). Labeling matters. If chunks are unlabeled blobs, citations become 'handbook.pdf' and you have decoration. If labels exist, you can demand [S3] on every factual sentence and you can eval it.

Prior knowledge is the leak. Frontier models know a lot of generic HR, law, and vendor folklore. That knowledge will fill gaps unless you forbid it for the classes of claim you care about: policy, prices, owners, dates, SLAs. The prompt should name those classes. 'Do not use prior knowledge for policy, prices, or owners' is a line you paste. 'Be accurate' is not.

Refuse-when-empty is a successful behavior. Write the exact sentence: 'I don't have this in the indexed documents.' Optionally: who to ask (the knowledge owner, a queue). Optionally: what would need to be ingested. Do not let the model improvise a helpful guess. Helpful guesses are how RAG gets banned after one steering meeting.

Refuse-when-irrelevant is harder and still required. Retrieval returned five chunks, none of which answer the question (vector search always returns neighbors). The prompt needs: if the sources are not sufficient, refuse; do not stitch a plausible policy from nearby topics. Groundedness eval is how you test this. The empty-gold rows from yesterday plus 'near-miss' questions (asked the wiki about something in a different space) are your test kit.

Temperature 0, pinned model, prompt version id in the log. The generator is the least of your moving parts and still needs the same client spec as day 9. JSON or a strict markdown contract if you will parse citations. Streaming is fine for the prose if you buffer until the first citation exists, as you already decided for policy copilots.

The prisoner prompt is the product: answer only from SOURCES, cite [Sn], refuse if empty or irrelevant, no prior knowledge for policy, prices, dates, or owners. Label the packet. Use 3 to 8 chunks, not 20. Temperature 0. Prompt version in the log. Exact refuse sentence, not an improvised apology that turns into a guess. Prior knowledge is the tunnel out of the cell; name the claim classes you forbid. If this contract is not versioned, a bad answer has no parent. Paste it. Date it. Put the id on the request. That is how you debug, and that is how you eval. A generator without a cage is search plus hallucination with a bibliography of unrelated files. You already knew that. Write it so the model knows it too.

A citation that says 'Source: handbook.pdf' is decoration. A citation that opens heading 4.2 of the 2024 travel policy is a product. Store enough metadata at ingest to deep-link: path, heading trail, page, paragraph id if you have it. This is how legal, QA, and operators trust the thing. If you cannot highlight, you cannot claim citation. That sentence will return tomorrow as citation theatre. Plant it now.

The UI is the contract with the user. Answer text, inline marks or footnotes, clickable sources, a visible 'this is not a decision / not legal advice / not an HR determination' footer on v1. No write-tools. No 'add this to the wiki' button until you have a human gate and a knowledge owner. v1 is read-with-receipts.

Groundedness eval: given answer + retrieved chunks, did each claim appear in the packet? First version: you, by hand, on 20 rows. Mark claim-level, not whole-answer-level, or one true sentence will launder two invented ones. Later, an LLM-as-judge — with skepticism (day 16). Never let the judge be the only eval, and never let the same model grade its own homework without a frozen human set.

Citation correctness is a separate mark: the cited [S3] actually supports the sentence, not merely appeared in the packet. Models will cite a nearby chunk because it was there. That is theatre. Your eval sheet has two columns: grounded (claim in packet) and cited-right (the id they pointed at is the one that holds the claim).

When the packet supports a partial answer, the product should say so. 'The SOP covers contractors in-country; it does not mention contractors abroad. I don't have the abroad rule in the index.' Partial-plus-refuse is more useful than a confident whole. Teach the prompt that omitting is allowed. Fullness is not a virtue when the corpus is thin.

Operators need a 'see the packet' debug. Not in the customer UI on day one, but in an internal view: the chunks, scores, filters that fired. When someone says 'it cited the wrong SOP,' you must be able to tell retrieve-wrong from generate-wrong in five minutes. request id on the footer, as on day 9.

Do not auto-open the source in a way that bypasses ACL. The citation click goes through the same permission check as retrieve. A deep-link that dumps a document the user could not have retrieved is an ACL leak with extra steps. Tomorrow covers the leak in slow motion. The hook is: citations inherit ACL.

A citation is a deep-link to the heading that supports the claim, clickable, inheriting ACL, with a highlight if you can build it. 'handbook.pdf' in a footer is decoration, and decoration trains auditors to trust a footnote that does not earn it. Groundedness is claim-level: did this sentence appear in the packet. Citation correctness is separate: did they point at the chunk that holds it. Partial answers plus refuse on the rest beat a confident whole. Operators get a 'see the packet' debug with the request id. v1 UI: answer, citations, 'this is not a decision' footer, no writes. If you cannot highlight, you cannot claim citation. Plant that now; tomorrow it returns as theatre.

Need a live number from SAP? That is a tool call, not a PDF index. Need to file a ticket? Structured write with approval. Need yesterday's tickets clustered? Embeddings without a generator. Need a deterministic workflow with 14 systems? That is orchestration, maybe with a model in one step — not a chat over a wiki. RAG shines for 'what does our documentation / prior decisions say.' Keep it in that box and it looks like magic. Stretch it into an agent of record and it looks like a lawsuit.

Math, counts, and 'how many open risks in RAID' are not RAG. The model will approximate from a stale export if you let it. The right pattern is a query tool against the system of record, with an owner of that query. You may still RAG the definition of a risk. You do not RAG the inventory of risks.

Actions are not RAG. 'Open a follow-up' is a write. Writes are day 13, with caps and a human gate. A wiki copilot that quietly files Jira tickets because the prompt said 'be helpful' is an incident. Non-goal it in v1 even if the sponsor saw a demo that did it.

Personal data, HR cases, health, payroll, employee performance: usually not this corpus, often not this pattern at all without a much uglier compliance track. If the wiki contains them incidentally, they are out-of-scope and ACL-blocked, not 'we'll prompt it not to answer.' Prompting is not access control.

Conversational search over a corpus the user already has in an enterprise search product may not need a generator. Sometimes the right v1 is hybrid search plus snippets, no LLM. Cheaper, easier to cite (the snippet is the citation), easier to ACL. Add generation when operators ask for synthesis across multiple hits, not because the roadmap said 'AI.'

If the path is known and the input is a form, do not put a model in the loop. 'Submit an access request' is a form with a policy RAG on the side to explain the rules. Mixing them into an agent is how you get a system that sometimes files the request and sometimes writes an essay about the request.

Write the non-goals in the same document as the pipeline. 'We will RAG the policy corpus with citations and a retrieval eval. We will not RAG the ERP. Numbers come from a query tool with an owner.' Say it early. Saying it after the vendor workshop is how you spend a quarter unpicking a demo.

RAG is for 'what does our documentation or prior decisions say.' Live numbers are a query tool. Writes are a form or a gated tool. Clustering is embeddings without a generator. Known 14-step paths are orchestrators. Personal HR data is out. Sometimes v1 should not generate at all — hybrid search plus snippets is cheaper, easier to cite, easier to ACL. Add generation when operators ask for synthesis across hits, not because the roadmap said AI. Write the non-goals in the same document as the pipeline. Repeat them when the demo gets exciting. The sentence that saves a quarter is the one you said early, not the one you drafted after the CEO screenshot.

Sponsors hate refuse. Demos hate refuse. Production requires refuse. Align them in the design: we will measure refuse rate, we will show a useful next step (who to ask, what to ingest), and we will not optimize refuse away by stuffing junk. A refuse rate of zero on a real wiki is a smell. It means you always retrieve something and always talk.

The empty path has a UX. Not a blank bubble (day 9). A sentence, a link to search, a 'request this document be added' if you have a knowledge owner, a request id. That is a successful task outcome when the corpus does not contain the answer. Write it in the one-pager as a screen, not as an afterthought.

Weak retrieval should look closer to empty than to confident. If top score is below a calibrated threshold and hybrid also missed, refuse or ask a clarifying question. Clarifying ('which product: Atlas or Orion?') is allowed. Inventing the product is not. Put two examples in the prompt: empty, and underspecified.

Do not retry generate on empty with a 'try to be helpful' second prompt. That second prompt is how prior knowledge walks in. If you retry, retry retrieve with a rewritten query (hyphenated ID, expanded acronym), then generate once more with the new packet, still allowed to refuse. Query rewrite is a retrieve trick, not a license to ramble.

Eval the refuse path as first-class. Yesterday's empty-gold rows. Today's groundedness on near-misses. Report false-answer rate on those rows separately from recall. Steering should see 'we refused 8/10 out-of-scope asks and we answered 1 we should not have.' That 1 is a go-live blocker if the domain is sensitive.

Operators will try to 'fix' refuse by adding more spaces to the index. Sometimes that is right (the document exists and is in-scope). Sometimes that is how HR folders get ingested on a Friday. Every corpus expansion is a change request with ACL, owner, and eval rows. Refuse is the pressure valve that makes those expansions visible.

Your generator contract from c2 is incomplete if it does not include the exact refuse sentence and the ban on prior knowledge. Paste it. Version it. Put the version id in the log. When a bad answer ships, you need to know which contract produced it.

Refuse-when-empty is a successful task outcome. Give it UX: a sentence, a next step, a request id — not a blank bubble, not a second prompt that says 'try to be helpful.' Query rewrite is a retrieve trick you may retry once. It is not a license to ramble. Measure refuse rate; zero on a real wiki is a smell. Eval the path with empty-gold rows and near-misses. The one false answer on an out-of-scope ask is a go-live blocker if the domain is sensitive. Operators will try to 'fix' refuse by adding spaces. Sometimes that is right. Sometimes that is how HR folders get ingested on a Friday. Every expansion is a change request with ACL, owner, and eval rows. Refuse is the pressure valve that makes those expansions visible instead of silent.

The practice is a one-pager for 'ChatGPT on Confluence for the program' — a sentence you will not accept as a spec. You replace it with: users, job, corpus in/out, pipeline boxes with owners, generator contract, two evals, first-release UI, non-goals. You will fold this into day 14. Write it so a skeptical engineer and a skeptical sponsor both survive the read.

Users and job: 'PMs on Program Atlas can ask what we decided about vendor X and get a cited answer from the decision log and SOPs in the Atlas space.' Not 'employees can chat with knowledge.' If you cannot name the job, you cannot name the eval questions, and you will RAG everything.

Corpus in/out: named spaces, named types (SOP, decision log, RAID). Out: HR personal data, vendor pricing folders, personal drives, archive unless dated and flagged. If out-of-scope material lives inside an in-scope space, that is an ingest filter, not a prompt. Name the knowledge owner who will keep the boundary.

First-release UI: answer, citations that open the heading, refuse sentence, 'this is not a decision' footer, request id, no writes, no agents. Search-like, not intern-like. If the sponsor needs a chat personality, they can have a one-line tone instruction. They cannot have tools that mutate Jira in v1.

Two evals, named: recall@5 on the day-10 table (at least 10 questions, mix including empty), and groundedness on 20 answers including the refuse cases. Owners of each. Cadence for re-run (every corpus change, every prompt change, every model bump).

Cost envelope at expected volume and 10×, using day 5 math and day 9 usage logs. If you cannot estimate, you do not have a v1, you have a credit card. Put the number next to the non-goals so the sponsor sees what they are buying.

The eight-minute talk track starts to form here: problem (PMs cannot find decisions), slice (Atlas space, read-only), how it works (the six boxes), how we'll know (two evals), what we will not do (ERP, writes, all-of-Confluence). Day 14 is that talk with RAID and ACL added. Draft the bones now so lab day is assembly, not invention.

The one-pager replaces 'ChatGPT on Confluence' with users, job, corpus in and out, boxes with owners, generator contract, two evals, first-release UI, non-goals, cost envelope. That is the object you fold into day 14. Write it so a skeptical engineer and a skeptical sponsor both survive the read. If either would bounce, you wrote a mood board. The eight-minute talk track starts here: problem, slice, how it works, how we'll know, what we will not do. Lab day is assembly of objects you already have. If this page is thin, thicken it tonight. Do not invent a new architecture under the clock.