Writing a document that comes back

Lesson 27 of 40 · 7 min

When this knowledge base was first built it had one document about paperwork, sensibly called Documents you will need. A customer asked what documents they needed. The agent said the handbook did not list them.

Nothing was broken. The document was there, it was active, it had been indexed, and it was about exactly that. It simply did not come back — and no setting would have fixed it, because the problem was the writing.

The same failure, re-run today

A document in that original shape was put back into your knowledge base, asked the two questions below, and removed again. This is what it scored:

Fourth — behind three documents that each cover one product.

Asked: “what documents do I need?”	Score
Personal and business loans — documents you will need	`0.9815`
Education loan — documents you will need	`0.9510`
Home loan — documents you will need	`0.8375`
Documents you will need	`0.0531`

Personal and business loans — documents you will need

Score: 0.9815

Education loan — documents you will need

Score: 0.9510

Home loan — documents you will need

Score: 0.8375

Documents you will need

Score: 0.0531

It lost to three documents on the question it is named after. On the more specific question — the one in your battery, “what documents do I need for a home loan?” — it did not place at all; the top three were the same three, at 0.9993 and below.

Why: it never says what it is about

Anvaya Finserv has been arranging loans in Delhi NCR since 2016 and
works with a panel of twelve lenders. Our advisors will guide you
through the process from start to finish. Depending on the facility
you choose and your profile, a number of items may be requested
during the application journey…

The opening of the document that scored 0.0531.

These are the documents we need for a home loan or a loan against
property.

IDENTITY AND ADDRESS — PAN card, …

The opening of Personal and business loans — documents you will need, which scored 0.9993 on the specific question.

Read them out loud and the difference is obvious in one line. The second sentence of the first document is about the company. The first sentence of the second document is the answer. A search has nothing to go on but the words in the chunk, and the first document's chunk is mostly about Delhi NCR, a panel of twelve lenders, and being guided through a journey.

Three rules, and they are all about writing

One subject per document. Not “documents” — “documents for a home loan”. This knowledge base has 3 separate document lists for that reason, and each of them wins its own question.
Say the subject in the first sentence. The opening line is doing most of the work. Company background, reassurance and context all push the answer further down the chunk and dilute it.
Use the words somebody would ask with. A customer types “what papers do you need”, not “indicative documentation requirements”. Write the document in the register of the question, not the register of a policy.

Notice what is not on that list. No setting, no model, no threshold, no chunk size. The fix for the original failure was rewriting one document into three, and the score on the same question went from 0.05 to 0.9993.

The hedging is the poison

Look again at “a number of items may be requested during the application journey”. That sentence is careful, professional, legally comfortable, and it contains no information at all. A knowledge base is the one place in a business where hedged writing has a measurable cost — it makes the document unfindable, so the customer is told nothing rather than something approximate.

Expect to receive documents written like that from clients. Their existing FAQ, their brochure copy, their compliance-approved wording. Loading it as-is is the commonest reason a client's assistant seems stupid, and the fix is a rewrite they have to agree to — which is a conversation to have in week one, not week six.

Change one thing — break it on purpose

This is the exercise that makes it stick, and it is safe because you will put it back.

Pick one of the 3 document lists in your customer knowledge base and search for its subject. Note where it ranks and what it scores.
Now rewrite its opening: put two sentences about the company first, and change the first line to something hedged — “a number of items may be required”.
Save, and search for exactly the same thing again.
Watch it fall. Then put the original opening back and watch it return.
You have now seen the whole of knowledge-base tuning. It is writing.

Try it

Take one document out of the internal playbooks and decide whether it would survive being asked for by a customer. Most internal writing does not.
Write the one-sentence opening you would use for a document about a client's refund policy. Then write the question a customer would type, and check they share words.

Next: Giving an agent a knowledge base — the knowledge base is right; now something has to decide when to consult it and what to do when it comes back empty.