Agents do not fail because the model is dumb. They fail because the knowledge you fed them is a junk drawer.
Plenty of “SOP folders” are really a graveyard of Word docs, screenshots, and a PDF named FINAL_v7_USE_THIS. Somewhere in there is the truth. Somewhere else is last year’s truth. The agent will happily blend both and sound confident about it.
If you want an agent that behaves, you need a knowledge base built for retrieval, not for binders.
Why raw SOPs break agents
Human SOPs assume a human reader. They wander. They say “ask Sarah.” They skip the boring edge cases because everyone already knows them. They contradict each other across locations.
An agent cannot ask Sarah. It cannot read the room. It will take the nearest paragraph and run.
So stop pretending a dump of documents is a knowledge base. Rebuild.
A messy-to-usable pipeline
1. Inventory, do not worship. List every source people actually use: wiki pages, shared drives, Slack pins, training decks, the sticky note on the monitor. Rank by “still true” and “still used.” Archive the rest out loud so nobody gets sentimental.
2. Split by job, not by department. Agents need answers for moments: after-hours intake, insurance FAQ, scheduling rules, escalation. Cut content into those jobs. A 40-page handbook becomes a set of tight topic cards.
3. One claim, one place. If hours live in three docs, the agent will eventually pick the wrong one. Canonical pages only. Everything else links or gets removed.
4. Write for retrieval. Short headings. Plain language. Explicit do and do-not. Examples of phrasing the agent can reuse. No motivational paragraphs. No “our commitment to excellence.”
5. Add the missing edge cases. Humans remember that weekends are different, that Location B has no detox, that Spanish callers get a different path. Put it in writing or the agent will invent something cheerful.
A card format that works
For each topic, aim for roughly:
- Title a human would search
- When this applies
- Steps or facts in order
- Exceptions
- Escalation triggers
- Last reviewed date and owner
That last line matters more than people think. Knowledge without an owner becomes folklore by Thursday.
Chunking without getting weird about it
You do not need a PhD in embeddings on day one. You need chunks that answer one question cleanly.
Aim for sections that can stand alone in a retrieval hit. If a chunk needs three other chunks to make sense, split it.
Also: strip boilerplate. Headers, footers, “confidential” banners, and page numbers just pollute similarity search.
Contradictions are useful if you face them
When two SOPs disagree, that is useful signal. It means your ops already disagree. Fix the process, then update the KB. Do not ask the model to average two policies into a vibes-based third policy.
Keep a “conflicts” list during the rebuild. It often becomes the best meeting agenda of the quarter.
Connect tools, do not only paste text
Static FAQ text is fine for hours and addresses. Live facts should come from tools: calendars, capacity, CRM fields, ticket status.
If the agent answers “Do you have a bed?” from a PDF updated in March, you built a hallucination machine with citations.
How to test before you trust it
Do not ask, “Does the KB feel complete?”
Run the questions real callers ask. The weird ones. The angry ones. The ones that start mid-story. Check:
- Did retrieval pull the right card?
- Did the answer stay inside approved language?
- Did escalation fire when it should?
- Did the agent admit uncertainty instead of improvising?
If it fails, fix the card first, prompt second. Prompt gymnastics cannot rescue rotten source material.
Maintenance is the real product
A knowledge base is not a project with a ribbon cutting. It is a living system.
Schedule reviews. Owners get pinged when content ages out. New edge cases from call reviews become new cards the same week. If nobody owns updates, your agent slowly becomes last quarter’s intern.
A practical starting set
If you are overwhelmed, start with twenty cards max:
- Greeting and identity
- Hours and locations
- What you do and do not offer
- Intake fields
- Crisis escalate
- After-hours callback promise
- Insurance “we cannot verify on this call” script
- Scheduling rules
- Language access
- Common objections with approved replies
Ship that. Measure. Expand from pain, not from a desire to digitize the entire company binder.
Bottom line
Turning messy SOPs into an agent-ready KB is unglamorous. It is editing, arguing, deleting, and dating your facts.
Do that work and the model looks brilliant. Skip it and you will blame the model for repeating your own chaos back to customers in a calm voice. Clean the drawer first.
