A customer asked this about an auto-tagging application, a day after installing it:

But when it begins to learn? If a user published the next untagged thread?

It is a completely reasonable question, and the answer is that it never begins to learn, because it never stops being ready. There is no warm-up. It works correctly on the very first topic and it works exactly as well on the ten thousandth. Nothing a member does changes it.

If you are building anything that classifies content with embeddings — tagging, related items, routing, triage — your users will arrive with a mental model of a system that studies their community and gradually improves. Yours almost certainly does not do that. The gap between those two models is where support tickets, bad reviews and "it stopped working" reports come from, and it is worth closing deliberately rather than hoping nobody asks.

Two things that get called the same word

Building the index is embedding each label once and storing the vector. It reads the label list, and often a sample of how each label has been used, and produces one vector per label. It is a build step. It costs API calls proportional to the number of labels, not the amount of content, and its output is static until you run it again.

Learning — what the customer meant — is a feedback loop: the system observes outcomes and adjusts. Corrections change future behaviour. Accuracy improves with use.

Almost every embedding-based classifier in the wild does the first and none of the second. The model is frozen, the label vectors are frozen, and classifying a new item is a pure function of that item and the stored vectors. Two identical topics posted a year apart get identical labels. That is a feature — it is predictable and debuggable — but it is not what "learn" means to anybody who is not the person who wrote it.

The failure this actually causes

Nobody is harmed by a system that does not improve. They are harmed by the corollary, which is much easier to miss:

A label created after the last index build has no vector, so it can never be suggested.

This is silent in the worst way. The label exists. It appears in the label picker. The classifier is enabled and visibly working — it is tagging things every day. It simply never applies that one label, and there is no error, no warning, and no log line, because from the matcher's point of view the label was never a candidate. The administrator's reasonable conclusion is that the app is ignoring their new tag, or that the AI "doesn't understand" it.

On Invision Community this is sharper than it needs to be, because there is no event fired when a tag is created. You cannot hook tag creation and rebuild automatically even if you want to. Whatever you do about this, it cannot be "detect it and fix it silently".

What to do about it

Detect the gap and say so. Comparing the label list against the stored vectors is a set difference, costs nothing, and needs no API call. If labels exist with no vector, say so on the settings screen in the terms the administrator is thinking in:

3 tags have been added since the vocabulary was last built
(Onboarding, Billing, Mobile app) and cannot be suggested
until you run Learn tags.

That single check converts your most likely support ticket into a self-service fix.

Be careful what you call the button. "Learn tags" is a good label for what it does and an actively misleading one for what it implies. Whatever you name it, put the shape of the thing next to it in one sentence: what it reads, and when it needs running again. State plainly that it is not continuous. Users do not read documentation, but they do read the sentence under the button they are about to press.

Lead with the good news, because it is genuinely good. "No learning" sounds like a limitation and is mostly the opposite. There is no cold start, no minimum corpus, no period of bad output while it settles. A brand new community with twelve topics gets the same quality as an established one. Say that, because the customer asking when it starts learning is really asking "is it working yet?"

Where the automatic behaviour actually belongs

The classification is static; the application of it should not be. Three details matter more than they look:

Do the work off the request. Classifying costs an API call and a network round trip. Posting must never wait on either — a slow or down provider would turn into "the forum won't let me post". Queue it and let the item be created immediately.

Reclassify on edit and on move, not only on create. A retitled topic is frequently a topic that is now about something else, and a topic moved between containers has often changed scope. Both are cheap to handle if your "only untagged items" guard bails before it embeds anything.

Deduplicate the queued job. Somebody editing a title five times in a minute should produce one classification, not five. Key the job on class and id.

The related trap

Do not let the classifier invent labels outside the vocabulary. Taggable::setTags() validates against the tag store and silently discards anything that is not there, so a suggestion outside the list does not error — it just fails to appear, which looks identical to the classifier having chosen nothing. Constrain the candidate set to what exists, and treat the label list as the contract.

See also: Why a similarity threshold is never enough for automatic classification, There is no event when a tag is created, and what that means for your app, and Taggable::setTags() deletes every existing tag first.

Related: Why a similarity threshold is never enough for automatic classification, A chat model in an embeddings field fails silently, and looks like a broken app, There is no event when a tag is created, and what that means for your app

Related application: Auto Tagging — Auto Tagging is built around this limit: it compares each topic to a fixed vocabulary you control rather than pretending to learn, so what it does is inspectable and repeatable.