An AI support assistant on a forum does not know your community, and it does not go and look. It answers from whatever you have given it. Install one on a site with nothing to read and you get an assistant that is confident, general, and wrong about the specifics — while billing you for every answer.
This surprises people, because the same models answer general questions well. The difference is that your members are not asking general questions. They are asking about your settings, your policies and your add-ons.
What "no knowledge base" actually looks like
- Plausible answers that are wrong for your site. Asked where a setting lives, it describes where that setting lives in a default install, or in a different version, or in software it has confused with yours.
- Hedging. A well-behaved assistant that has nothing to go on says "check your AdminCP settings", which is not an answer and reads as evasive.
- Credits spent on nothing. Every one of those non-answers costs the same as a good one.
What to write, and it is less than you think
You do not need a manual. You need the answers you are already typing by hand. A useful starting set is:
- Your ten most repeated support answers. Look at what you have actually replied to in the last month. That is the list.
- Anything with a non-obvious cause. The problems where the symptom points at the wrong thing are exactly the ones an assistant saves you the most time on — and exactly the ones it cannot guess.
- Your policies. Refunds, signatures, what gets a warning. These are pure site-specific fact and impossible to infer.
- Version and setup specifics. Which add-ons you run, what you have deliberately turned off.
Write each one as the answer to a question somebody actually asked, in plain prose. Do not write it as a specification — an assistant retrieves prose far better than it retrieves a table of settings.
Why it is worth the afternoon
An assistant grounded in real answers resolves the repeat questions without you, at any hour, in the member's own words rather than yours. The unglamorous part — writing down what you already know — is the entire difference between an assistant that deflects support and one that generates it.
The test is simple: take the last five questions you answered by hand, and ask the assistant those. If it cannot answer them, it is not going to answer them for a member either.
Related
The same expectation gap applies to other AI features: see Your classifier does not learn, and everyone will assume it does. In both cases the software works exactly as intended, and the disappointment comes from a reasonable but wrong mental model of what it is doing.
Related application: AI Assistant — AI Assistant is configured against a Pages database of articles alongside an index of your own forum posts, so its answers are drawn from documents you wrote.
Recommended Comments