Writing a document that comes back
A document on exactly the right subject, sitting in the knowledge base, that the agent could not find. The fix was the writing, and it is measurable.
When this knowledge base was first built it had one document about paperwork, sensibly called Documents you will need. A customer asked what documents they needed. The agent said the handbook did not list them.
Nothing was broken. The document was there, it was active, it had been indexed, and it was about exactly that. It simply did not come back — and no setting would have fixed it, because the problem was the writing.
The same failure, re-run today
A document in that original shape was put back into your knowledge base, asked the two questions below, and removed again. This is what it scored:
| Asked: “what documents do I need?” | Score |
|---|---|
| Personal and business loans — documents you will need | 0.9815 |
| Education loan — documents you will need | 0.9510 |
| Home loan — documents you will need | 0.8375 |
| Documents you will need | 0.0531 |
Personal and business loans — documents you will need
- Score
0.9815
Education loan — documents you will need
- Score
0.9510
Home loan — documents you will need
- Score
0.8375
Documents you will need
- Score
0.0531
It lost to three documents on the question it is named after. On the more specific question — the one in your battery, “what documents do I need for a home loan?” — it did not place at all; the top three were the same three, at 0.9993 and below.
Why: it never says what it is about
Anvaya Finserv has been arranging loans in Delhi NCR since 2016 and
works with a panel of twelve lenders. Our advisors will guide you
through the process from start to finish. Depending on the facility
you choose and your profile, a number of items may be requested
during the application journey…These are the documents we need for a home loan or a loan against
property.
IDENTITY AND ADDRESS — PAN card, …Read them out loud and the difference is obvious in one line. The second sentence of the first document is about the company. The first sentence of the second document is the answer. A search has nothing to go on but the words in the chunk, and the first document's chunk is mostly about Delhi NCR, a panel of twelve lenders, and being guided through a journey.
Three rules, and they are all about writing
- One subject per document. Not “documents” — “documents for a home loan”. This knowledge base has
3separate document lists for that reason, and each of them wins its own question. - Say the subject in the first sentence. The opening line is doing most of the work. Company background, reassurance and context all push the answer further down the chunk and dilute it.
- Use the words somebody would ask with. A customer types “what papers do you need”, not “indicative documentation requirements”. Write the document in the register of the question, not the register of a policy.
Notice what is not on that list. No setting, no model, no threshold, no chunk size. The fix for the original failure was rewriting one document into three, and the score on the same question went from 0.05 to 0.9993.
The hedging is the poison
Look again at “a number of items may be requested during the application journey”. That sentence is careful, professional, legally comfortable, and it contains no information at all. A knowledge base is the one place in a business where hedged writing has a measurable cost — it makes the document unfindable, so the customer is told nothing rather than something approximate.
Expect to receive documents written like that from clients. Their existing FAQ, their brochure copy, their compliance-approved wording. Loading it as-is is the commonest reason a client's assistant seems stupid, and the fix is a rewrite they have to agree to — which is a conversation to have in week one, not week six.
Change one thing — break it on purpose
This is the exercise that makes it stick, and it is safe because you will put it back.
- Pick one of the
3document lists in your customer knowledge base and search for its subject. Note where it ranks and what it scores. - Now rewrite its opening: put two sentences about the company first, and change the first line to something hedged — “a number of items may be required”.
- Save, and search for exactly the same thing again.
- Watch it fall. Then put the original opening back and watch it return.
- You have now seen the whole of knowledge-base tuning. It is writing.
Try it
- Take one document out of the internal playbooks and decide whether it would survive being asked for by a customer. Most internal writing does not.
- Write the one-sentence opening you would use for a document about a client's refund policy. Then write the question a customer would type, and check they share words.
Next: Giving an agent a knowledge base — the knowledge base is right; now something has to decide when to consult it and what to do when it comes back empty.
Related lessons
Base rates — what a piece of evidence is actually worth
A face-recognition system that is 99.9% accurate and almost entirely wrong, and a number that sent an innocent woman to prison. Both are the same arithmetic, and it is the arithmetic that decides what any piece of evidence is worth.
ReadConfirmation and survivorship — what you never looked for
Two questions about evidence you did not go looking for. One is a rule you have to discover, and one is a pattern in five famous people — and in both, the thing that would have told you the truth is the thing nobody checks.
ReadLoss aversion, sunk cost and regression — what it costs you
Four questions you answer about yourself rather than about a scenario, and your own answers are the finding. Then the pattern that makes praise look useless and criticism look like it works, whatever you actually do.
Read