Part 3 of 4 in The Brilliant Stranger, a field guide to LLMs, explained like you run a business.
Last time: the guitar, tuned before you ever met it, and why what changes the answer is the chart on the stand. This time, what’s on the chart.
“We connected it to our docs. Why is it still wrong?”
This is the most common disappointment in enterprise AI, and it arrives two weeks after the demo. The team points the assistant at the shared drive, the wiki, the policy folder. Now it has everything, and the answers get vaguer, not sharper. It cites a document superseded two years ago, or blends two contradicting versions into one confident paragraph.
The wrong picture: point it at the wiki and it reads everything
Under the filing-cabinet idea from the second post, this should work. More documents, more knowledge. But only a few pages fit under the door at once. The assistant has to choose, and whatever it chooses is all the expert gets.
The better picture: an open-book exam
Picture the smartest student you know sitting down for a timed open-book exam. The book is on the desk. In this picture, the student is the whole system from post one: the assistant flipping through the book, and the expert reading whatever gets handed through the door.
Now suppose the book is forty pages where two would do. Chapter three contradicts chapter nine because someone updated one and not the other. The pricing table predates the last price change. There’s no index, so finding anything means flipping.
The student fails. Not because the student is weak, but because the book was written for a different purpose, by different people, over years, and nobody expected it to be read under time pressure by someone who has to trust it.
That is what most companies hand the AI when they connect it to their docs.
Why your documents make bad context
Your documentation was written for humans. Humans skim. Humans know which version is current because they were in the meeting. Humans read the forty-page SOP once, remember what matters, and never open it again.
The expert behind the door has none of that. Every page it receives, it takes at face value. It doesn’t know the 2023 version is dead, or that the discount rule the sales team actually follows lives in an email thread and not the playbook, or that the one paragraph that matters is on page 31.
So four things that were fine for humans become failures for the model. Length, because pages are scarce and the important sentence is buried. Contradiction, because the model can’t know which version wins. Staleness, because nothing is marked current or retired. And missing structure, because it’s hard to pull the right piece out of prose under a page limit.
The guitar is still tuned. The expert is still brilliant. You handed them a chart that was a forty-page PDF.
What a good book looks like
The book that wins the open-book exam is not bigger. It’s built for the exam.
It’s current, and it says so. Every piece has a date and an owner, so the assistant can tell what’s live and what’s history.
It’s consistent. One answer per question. When pricing changes, the old pricing leaves instead of sitting next to the new one.
It’s compressed. The three things that matter from the forty-page SOP, written down as three things, with the full document available if the expert asks.
It’s structured. Facts, rules, definitions, and examples, shaped so one piece can be pulled without dragging the whole page through the door.
And it’s curated. Somebody owns it. Not the whole shared drive, but the layer on top that says: here is what we actually know, in the form the expert can use.
This is the part that surprises leaders. Connecting the AI to your documents takes an afternoon. The work is turning your documents into a book worth taking into the exam.
What this means for your business
Treat context as an asset, not a folder. It needs owners, review dates, and a retirement process, like anything else your business depends on.
Stop measuring by coverage. “It has access to everything” is not a feature. Ask instead: for the twenty questions people actually ask, like what we can discount and who has to approve it, are the right two pages in the book, current and unambiguous?
Write for the reader behind the door. When you document a process from now on, write the compressed version first. Humans will thank you too.
The book is fixed. Next time: who finds the page. A perfect book with a bad librarian still fails the exam.
Wondering what your book looks like from the expert’s side? Reach out.