Lab: design a project-wiki copilot

Bad: 'RAG over corporate knowledge for employees.' Good: 'PMs on Program Atlas can ask what we decided about vendor X and get a cited answer from the decision log and SOPs in the Atlas Confluence space. No writes. No HR. No finance folders. Recusal if ACL misses.' The second sentence can be built, evaluated, and killed. The first can only be demoed.

Name the audience, the job, the corpus, the non-goals, the eval, the owner. That is a design. The rest is decoration. If you skip naming because the fictional program feels fake, you will write generic prose and the artifact will not survive an interview. Specific fake beats vague real. Atlas, SOP-14, decision log 2024-06-12, PMs, 200 pages — pick them and stay loyal for the rest of the hour.

The job-to-be-done is a verb a PM already does: find the decision, check the SOP exception, see who owns the RAID item. It is not 'chat.' Chat is a UI. If you cannot name the verb, you cannot name the eval questions, and you will scope a chatbot of everything. Write the verb in one sentence at the top of the pack and do not start the pipeline until that sentence is boring and clear.

Users are a role with a permission set, not 'the business.' PMs on Atlas, not all employees. The ACL story follows from that. If you say 'employees,' you inherited every leak in the tenant. Narrow users are how you ship. You can add a second role in v1.1 if the eval still holds. You cannot subtract a role after the intern found the CEO deck.

Corpus in is a list of spaces and types, with an approximate page count. Corpus out is a list too, not a vibe: HR, finance, personal drives, archive unless flagged, vendor pricing. If out-of-scope pages live inside the in-scope space, say how ingest drops them. Boundary without a mechanism is a prompt wish. A number helps: 'about 200 pages in three types' is a corpus. 'Corporate knowledge' is a speech. Interviews will ask how big it is because cost, ACL, and eval effort all scale with that number, and 'we'll see' is not an estimate.

The knowledge owner has a name-shaped blank you fill: 'Atlas PMO lead' is acceptable if you do not have a real human. 'The AI team' is not. Write what they owe: archive conflicts, sample 20 chunks, re-judge gold labels when SOPs move, take corpus errors routed from users. If that person would not agree to this, pick a thinner corpus.

Read your sentence out loud. If it needs a slide to be understood, it is not specific yet. Interviews will not give you the slide first. They will say 'tell me about a system you would ship' and you have 30 seconds to get Atlas, PMs, cited decisions, no writes, out of your mouth.

Specificity is the quality bar because vague enterprise RAG designs all sound the same and hire none of you. Name the audience, the job-to-be-done as a verb a PM already does, the corpus with a page count, the non-goals as sentences with reasons, the eval as fractions, the owner as a hat a human would accept. Specific fake beats vague real. Atlas, SOP-14, PMs, 200 pages — pick them and stay loyal for the hour. If the sentence needs a slide to be understood, it is not specific yet. Interviews will not give you the slide first. They will say 'tell me about a system you would ship' and you have 30 seconds. Write the sentence at the top of the pack and do not start the pipeline until it is boring and clear. Chat is a UI. The verb is the job. If you cannot name the verb, you will scope a chatbot of everything, which is the artifact this lab exists to prevent. Read it out loud. If you cringe, it is still too generic. Cut until a stranger could staff it.

The pipeline is day 11's boxes, named for Atlas. Ingest: Confluence Atlas space, SOP / decision log / RAID types, drop nav and archive, sample 20 chunks. Chunk: split on H2, max 800 tokens, 100 overlap, tables kept whole, metadata = path, heading, date, permission group. Index: embedder pinned, hybrid search, query-time ACL from SSO. Retrieve: k from eval, empty allowed. Generate: prisoner prompt, temperature 0, refuse sentence. Cite: deep-link to H2. UI: answer, citations, footer, request id. Eval: recall@5 and groundedness. No loop. No writes.

Owners on the arrows. Knowledge owner on ingest and conflicts. Engineer on index, client, ACL filter, highlight UX. You on eval, generator contract, RAID, go-live. Security on leak-test. Legal on logs and retention. Product on footer and degraded UX. If you are writing this as a one-person FDE slice, you still name the hats. Future-you will put a different human in the hat. Unnamed hats disappear.

Happy path: PM asks 'what did we decide about vendor X in Q2.' Query-time ACL filter. Hybrid retrieve. Gold decision log in top 5. Packet labeled [S1]… Generate with citations. UI opens heading. Trace logged with prompt version and usage. That path should be the 40-line pseudo-script if you write one: load is already done (index exists), loop is not needed (one question), call is retrieve then generate, print is the answer plus source ids.

Empty path: PM asks Jane's salary, or a policy in a different space. Retrieve returns nothing (or only out-of-scope neighbors you dropped). Generator emits the refuse sentence. UI offers 'ask the Atlas PMO' and a request id. This path is in the architecture, not in an appendix. Draw it. Staff it. If you cannot talk the empty path in the eight-minute walkthrough without notes, the pack is still a happy-path demo, and happy-path demos are what this week was built to stop you shipping.

Fail path: 429, timeout, empty 200, leak-test would fail so you do not ship. Client spec from day 9 is an appendix of this pack: defaults, error table, kill switch. Do not rewrite it. Attach it. The copilot is not allowed a different timeout philosophy because it is 'just a wiki.'

Change path: SOP-14 updates at 4pm. Connector upserts or the 24h lag banner is honest. Canary document in the runbook. Re-index SLA is a number. This is architecture, not operations-later. If you leave it out, you designed a demo.

What you are not drawing: a swarm, a write to Jira, a fine-tune, a multi-tenant platform, 'all of Confluence.' If a box wants to sneak onto the page, it goes in non-goals. The compare diagram exists so you have a place to put the energy that is not v1.

The architecture is day 11's boxes named for Atlas, staffed, with a happy path, an empty path, a fail path, and a change path. No loop. No writes. Client spec from day 9 is an appendix, not a rewrite. The optional 40-line script is retrieve then generate, four verbs you already know from day 8, user ACL on the search. If a box has no owner it will disappear. If a box wants to sneak on — swarm, Jira write, fine-tune, all of Confluence — it goes in non-goals. Draw until you can talk the empty path without notes. Empty is where this product is honest. Happy path is where demos live. You are not turning in a demo. You are turning in a slice a skeptical engineer could stand up in a week, and a skeptical sponsor could fund without pretending the rest of the tenant is in scope.

v1 is easy to agree with in a workshop and easy to betray by Friday. Write non-goals as complete sentences with a reason. 'No writes' is a slogan. 'No Jira writes in v1 because we have no human gate, no idempotency, and no owner of a wrong ticket' is a control. The compare diagram is the slide. The sentences are the pack.

No agents. The job is find-and-cite. Recall is not yet a number. A loop will paper over empty retrieve by searching again until it finds a neighbor. That neighbor will be wrong. Query rewrite once is allowed later; a free-form ReAct loop is not v1. Day 13 gave you the sentences. Paste them.

No all-of-Confluence. Day 12's memo is an appendix. The slice is Atlas, ~200 pages, three types. Expansion is a change request: owner, ACL, sample chunks, new eval rows. Without that sentence, success on Atlas will be read as permission to ingest HR.

No personal drives, no finance folders, no HR case data. If they appear in Atlas by accident, ingest drops them and security is told. Prompting is not the filter.

No live ERP numbers. 'What did we decide about vendor X' is in. 'What is the open PO' is a tool to finance SoR, out of v1, possibly forever for this copilot. Mixing them is how you get a confident wrong amount with a SOP citation.

No personality work, no avatar, no 'make it fun.' Tone can be one line in the system prompt. Budget does not go to the mascot while recall is unmeasured. A fun bot that cites the wrong SOP is still a wrong SOP. You can add tone in an afternoon once the evals exist. You cannot add evals in an afternoon once the mascot has been demoed to the CEO and is now the product.

v1.5 / v2 live in a short appendix so the sponsor sees a path: one read tool (ticket lookup) when recall@5 holds for two weeks; one draft-with-confirm when a knowledge owner asks for it. Dates only on v1. The path is how you stay in the room without signing up for a swarm.

v1 versus non-goals is how you protect the slice from Friday betrayal. No agents, because a loop will paper over empty retrieve. No all-of-Confluence, because success on Atlas will be read as permission to ingest HR. No personal drives, finance, HR cases. No live ERP numbers. No mascot budget while recall is unmeasured. Write them as complete sentences with reasons, not slogans. v1.5 lives in a short appendix: one read tool when recall holds; one draft-with-confirm when a knowledge owner asks. Dates only on v1. The compare diagram is the slide. The sentences are the pack. If it is not in the left column, it is not in the build. Anyone who wants an Atlas agent on the first ship is asking for the keynote. You have the sentences from yesterday. Paste them.

Paste the day 10 table. Ten questions, mix intact: ID, how-do-we, decision, table, synonym, region if you have it, stale trap, owner, should-be-empty, multi-hop-ish. Gold is a filename and heading, not 'the handbook.' Success: recall@5. The empty row's success is retrieve nothing / refuse. Do not drop the empty row because it 'isn't a demo.'

Groundedness plan: 20 answers, claim-level, two marks (grounded, cited-right), including refuses. First run by hand. Owner: you. Cadence: every prompt change, every model bump, every corpus expansion. LLM-as-judge is a later maybe (day 16), not a v1 shortcut that grades its own homework.

Leak-test: a user who should not see folder F, a question whose gold lives only in F. Success: refuse, no title leak. This is a go-live gate. Put it next to recall so it cannot be 'v2.'

ACL approach in one paragraph: ingest stores permission group from Confluence; query filters by SSO groups; citations click through the same check; POC runs with ACLs on or it is not a POC. God-account ingest is banned in production language even if the first load used one. Write the ban.

Cost envelope: expected questions / week, tokens in (packet size × k), tokens out, embed of 200 pages, re-embed on churn. Then 10×. Kill switch criteria. Day 5 math, day 9 usage logs. A missing envelope is how finance discovers you. Put the number even if it is ugly. Ugly and owned beats blank. If you cannot estimate, you do not have a v1 — you have a credit card and a wiki. Write the assumptions (volume, k, packet tokens, model price) so next quarter someone can refresh the envelope without you.

Re-index SLA: 24h lag banner, or event-driven if the connector is real. Canary page in the runbook. Connector-down: serve stale with banner vs go dark — pick one for Atlas (stale-with-banner is acceptable for decisions that are not safety-critical; say so).

Report these as fractions and dollars, not as 'good' and 'cheap.' 7/10 recall@5, 16/20 grounded, $X at 1× and $Y at 10×, leak-test pass. That block is the steering slide. The pipeline cartoon is backup.

Steering hears three numbers: recall@5 as a fraction, groundedness as a fraction, cost at 1x and 10x. Leak-test pass/fail sits beside them as a go-live gate, not as v2. Paste the day 10 table with the mix intact, including the empty row. ACL in one paragraph: query-time SSO filter, citations inherit it, POC runs with ACLs on. Re-index SLA as a number and a banner. Kill switch criteria next to the envelope. Report fractions and dollars, not 'good' and 'cheap.' That block is the steering slide. The pipeline cartoon is backup. If a heading in the design-pack table is empty, the artifact is not done. Fill the heading. Do not decorate the cartoon.

Paste at least six RAID rows from day 12, rewritten for Atlas: stale/conflict, ACL, junk ingest, demo eval, citation theatre, no owner, cost at 10×. Triggers observable. Owners named as hats. If you cannot name a knowledge owner, the row is still there and the conditions-to-proceed say the slice does not start.

The no-go memo becomes the 'if they push for all of Confluence' appendix. You may not send it this week. You will send it in a job, or you will wish you had. Six sentences, thinner slice, date. Keep it in the pack.

Talk track, timed: (1) problem — PMs cannot find decisions, they ask Slack, they get folklore. (2) slice — Atlas space, PMs, find-and-cite, no writes. (3) how it works — six boxes, prisoner generator, hybrid retrieve. (4) how we'll know — recall@5, groundedness, leak-test, cost. (5) risks — the RAID top three. (6) ask — named owner, two weeks to eval bar, no CEO demo until then. Eight minutes is enough if you do not tour the obvious.

Practice the talk once in your head before you tick the artifact. If you run long, cut architecture details, not the evals and non-goals. Interviewers who have shipped this will interrupt to ask about ACL and empty retrieve. Have those sentences on your tongue, not only in the doc.

Day 29 will reuse this walkthrough. Write the bullets in the pack, not on a sticky you will lose. The document is the source. The talk is a view.

Tone of the pack: delivery, not a pitch deck. Short sentences. Names. Numbers. No 'leverage' or 'empower.' A skeptical engineer should nod at the client spec and the leak-test. A skeptical sponsor should see the slice and the ask. If either audience would bounce, edit.

Done looks like headings filled, not a mood board. A pipeline cartoon without the table of questions is not done. A table of questions without ACL is not done. Tick the capstone artifact only when you would let a stranger read the pack without you in the room.

RAID, the no-go appendix, and the eight-minute talk track are how the pack ships without you narrating it. At least six rows with observable triggers. Six-sentence memo for the all-of-Confluence push. Talk: problem, slice, boxes, evals, ACL, cost, non-goals, ask. If you run long, cut architecture tourism, not evals and the empty path. Day 29 reuses this walkthrough; write the bullets in the pack. Tone is delivery, not a pitch. Tick the artifact only when you would let a stranger read it. A cartoon without the question table is not done. A table without ACL is not done. Practice the talk once in your head before you tick. The document is the source. The talk is a view.

You can read a script, specify an API client, design retrieval evals, draw RAG, catalog failures, and cap an agent. You are not a staff engineer. You are now dangerous in the useful way: you will not fund a chatbot of everything, and you can sit with someone who writes the code.

The split from day 8 still holds. You wrote the spec, the eval, the RAID, maybe a 40-line happy path. An engineer writes retries, the real connector, observability, the service. FDE sits in the overlap and may implement more. Delivery lead sits on spec and eval and go-live. Do not claim the platform in the pack. Claim the slice and the controls.

Week 3 turns this into the operating system: governance, evals as a program, vendors, the charter. This artifact is an input to that week, not a trophy you stop touching. When the charter asks for a thin slice, this is the slice. When vendor day asks for evals on our tasks, this is the table.

If you skipped practices on days 10–13, do not invent from a blank page for an hour and call it a pack. Go back, harvest the objects, then assemble. The lab is assembly. Invention without the catalog will recreate the demo you were taught to refuse.

Optional running code is extra credit this month, not the artifact. A repo without the design is a toy. A design without a repo is hireable for delivery and solutions. FDE loops will want both, later. Do not spend the last 40 minutes debugging an install. Spend them on ACL and the empty-gold row.

Your positioning line from day 1 can now mention this object: a wiki copilot design with retrieval evals, ACL, and a no-go. That is more useful in 15 seconds than 'I have been learning RAG.' Rewrite it after you tick, even if the rewrite is ugly.

Stop at 120 minutes. An 80% pack that exists beats a 100% pack you will start on Saturday. Tick the artifact. Tomorrow you are not a builder. You are back to operating.

Enough building this week means you can read a script, specify a client, design retrieval evals, draw RAG, catalog failures, and cap an agent — and then refuse the agent on this slice. You are not a staff engineer. You are dangerous in the useful way. Week 3 turns this object into the operating system. The split from day 8 still holds. Optional running code is extra credit; do not spend the last 40 minutes on an install. Rewrite the day 1 positioning line to mention this pack. Stop at 120 minutes. An 80 percent pack that exists beats a 100 percent pack you will start on Saturday. Tick. Tomorrow you are not a builder. You are back to operating. That is the job you are training for, pointed at a new object you can now draw, eval, and kill.