Auto Tagging — tag every topic

Auto Tagging — tags every topic by what it is about

Tagging only works if people do it, and people do not do it. Your tag list gets built with care, gets used properly for a fortnight, and then quietly stops being used at all. Everything posted after that is untagged, so tag pages go stale, filtering misses things, and the vocabulary you designed is doing nothing.

This reads each new topic and applies the tags that actually fit it.

It matches meaning, not words

A topic titled "my card keeps getting declined and I was charged twice" gets tagged billing — a word that appears nowhere in it. Invision's own tag box offers autocomplete over text the author has already typed, which cannot do this. Matching here is done on meaning, so a topic reaches the right tag even when it shares no vocabulary with it at all.

It uses your tags. Only your tags.

Nothing is invented and nothing new appears in your tag list. The app reads the tags you have already defined and chooses between them, so the vocabulary stays exactly as you designed it. Disable a tag and it stops being applied.

It also learns what each tag means in your community, not in general. A tag is understood partly from the topics that already carry it, so build on a woodworking forum and build on a gaming forum end up meaning quite different things — as they should.

It will not touch work someone did by hand

This is the part worth reading carefully. Adding a tag in Invision Community is not actually an operation the platform offers — saving tags replaces every tag on the item. A tagging tool that gets that wrong will silently delete tags your moderators set, and nobody will connect the loss to the app that caused it.

Auto Tagging always writes the complete set: existing tags, the new ones, and the item's prefix. By default it leaves anything already tagged completely alone.

Honest about confidence

Every match carries a score, and there is a threshold below which nothing is applied. The defaults are set to tag less rather than tag wrongly — a wrong tag is worse than a missing one, because someone has to find it and undo it.

There is a Preview screen that scores real topics from your own community and shows you the numbers, so you can pick a threshold from evidence rather than from a guess. It tags nothing while you do it.

Fits around the site rather than into it

• Posting never waits. Tagging happens in the background after the topic is saved. Nobody sits watching a spinner while an API is called.

• Backfill on demand. Tag the topics you already have, batched so a large community does not arrive as one enormous bill.

• Scope it. Restrict it to certain forums, and list any tags it must never apply — solved and staff pick mean something procedural that only a person should decide.

• Respects your limits. It will not push an item past your site-wide maximum tags per item.

• One key, one bill. If you run Related Topics or AI Assistant, it can borrow their embeddings key instead of asking for another.

• No core files are edited. Upgrading Invision Community will not break it.

What it costs to run

Auto Tagging needs an embeddings API key — Voyage AI or OpenAI. This is not a subscription to me; you hold the account and pay the provider directly.

Embeddings are among the cheapest things either provider sells. Your tag list is processed once and then only when it changes, and each new topic is a single small request. For an ordinary community this runs to a trivial amount per month, and there is no charge at all while nothing is being posted.

Requirements

• Invision Community 5.0 or newer (self-hosted)

• Forums

• Tags enabled, with at least a few tags defined

• An embeddings API key from Voyage AI or OpenAI — or Related Topics / AI Assistant already installed, in which case it can share theirs

File Information

Submitter Ernest Defoe

Submitted 07/26/26

Category SEO & Search apps for Invision Community 5

View File

@Claude_Bot I have a problem with this App Auto Tagging. I published a new thread and I edited threads, for example the thread title. But nothing is tagged 😕

Usage of API key in https://platform.openai.com/ is 0.

What could be the problem?

A few things to check, since usage showing zero at OpenAI suggests no embedding request ever reached them.

First, tagging happens in the background after a topic is saved, not instantly and not in the request cycle. So it relies on your scheduled tasks running. If cron/tasks aren't firing, nothing will get processed. Worth confirming your task setup is healthy.

Second, and this is likely the key point: by default Auto Tagging leaves anything already tagged completely alone, and it acts on new topics. Editing a title on an existing topic is not what triggers it — re-tagging edited or previously-seen topics isn't the same path as tagging a fresh post. For topics that already exist, that's what the Backfill option is for.

Third, check the confidence threshold. If nothing scored above it, nothing gets applied and you'd see no API usage only if it never got as far as embedding — so more likely the request isn't being made at all rather than being made and rejected.

To narrow it down, can you tell me:

1. Are your scheduled tasks running normally?
2. Is the key entered directly in Auto Tagging, or is it borrowing one from Related Topics / AI Assistant?
3. Did you run Preview? Does it score topics and show numbers, or does it also come back empty?

If Preview also produces nothing and usage stays at zero, that points at the key or connection rather than the tagging logic. Let me know what you find.

@Claude_Bot

1. yes. But I can't see any special task related to Auto Tagging. Which task should work for Auto Tagging?

2. API key is direct in Auto Tagging.

3. Preview doesn't score topics or numbers. There is a message "Learn the tag vocabulary first – there is nothing to match against yet". But I used "Learn tags" before that.

That Preview message is the important clue. It means the tag vocabulary has no stored vectors to match against — so even though you pressed "Learn tags", the build either didn't complete or didn't store anything. Until that vocabulary exists, nothing will be tagged and no embedding request goes out, which is why your OpenAI usage is still zero.

So the problem isn't editing versus new topics — it's that there's nothing to match against yet. A couple of things to check on that:

1. When you ran "Learn tags", did it report success, or did it error or appear to do nothing? If the key was rejected at that point, the vocabulary build would fail and you'd get exactly this message with zero usage.
2. Confirm you actually have tags defined and enabled, and that they're the tags you expect it to use. "Learn tags" reads your existing enabled tags — if there were none available to it at that moment, there'd be nothing to build.

Try running "Learn tags" once more and watch closely for any error or confirmation. If it comes back and Preview still says there's nothing to match against, the build isn't storing vectors, and I'd want the exact text of anything it shows when you run it — plus confirmation that the same OpenAI key works elsewhere (for example in Related Topics or AI Assistant if you have either installed).

On your task question: tagging runs via the scheduled task system in the background, but there isn't a healthy path to it until the vocabulary builds, so let's get "Learn tags" working first before chasing the task side.

@Claude_Bot

1. If I ran "Learn tags" it reports "Learning the tag vocabulary in the background".

2. Tags are defined and enabled. There are definitively tags that would use.

"Related Topics" or "AI Assistant" I haven't installed.

Model name is gpt-5.4-nano

That message — "Learning the tag vocabulary in the background" — is the key detail. Like tagging itself, the vocabulary build is handed off to the background task system rather than done on the spot. So "Learn tags" doesn't build anything itself; it queues the work. If your scheduled tasks aren't actually running, that job never executes, no vectors get stored, and Preview keeps telling you there's nothing to match against. That fits everything you're seeing, including zero OpenAI usage.

So the thing to verify is whether background tasks are genuinely running, not just configured. In Invision's AdminCP there's a system health/support area that will warn you if the task runner hasn't run recently. Check that first. If Invision is set to run tasks via traffic (the default) rather than a real cron, a quiet site can leave jobs sitting unprocessed.

On the model name: I can't confirm whether "gpt-5.4-nano" is a valid OpenAI model, and I don't want to guess. But note Auto Tagging needs an embeddings key, and embeddings use a different class of model from chat models like the gpt-* line. If a chat model name has been entered where an embeddings model is expected, the build would fail. Can you check whether Auto Tagging has a separate field for the embeddings model, and tell me exactly what's in it?

So, two things to report back:

1. Does the AdminCP system health screen show tasks running normally, or does it flag that the runner hasn't run recently?
2. What exactly is set as the embeddings model in Auto Tagging (separate from any chat model field, if there is one)?

If tasks are confirmed healthy and it still won't build, that starts to look like a genuine bug and I'll note it for review — but let's rule out the task runner first, since the "in the background" message points straight at it.

@Claude_Bot

1. In support tool at /admin/?app=core&module=support&controller=support&_new=1 are no problems shown.

2. As embedding model in Auto Tagging is set gpt-5.4-nano

Here's a screenshot of the part in settings:

There's your problem: gpt-5.4-nano is a chat model, not an embeddings model. Auto Tagging needs an embeddings model, and the vocabulary build will fail if a chat model name is in that field — which fits everything you're seeing, including zero usage and the "nothing to match against" message.

OpenAI's embeddings models have names in the text-embedding family rather than the gpt-* line. Put one of OpenAI's current embeddings model names in that field, save, and run "Learn tags" again, then check Preview.

Two things though. First, I can't confirm the exact valid embeddings model names off the top of my head without guessing at version details, so check OpenAI's own model list for their current embeddings models and use one of those — not a gpt- name. Second, if the field is a free-text box that accepted gpt-5.4-nano without complaint, and the build silently fails rather than telling you the model is wrong, that's worth flagging. A rejected model should surface an error, not leave you with an empty vocabulary and no explanation.

If a corrected embeddings model still won't build after that, let me know and I'll note it for review.

@Claude_Bot I tried "text-embedding-3-small" as model – and now Preview is working. Great!

@Claudia999 I use VoyageAI for my embedding and it's been great and with everything I have done it hasn't cost me a cent yet. Just something to maybe look into.

That was it. With a real embeddings model in the field, the vocabulary can build and Preview has something to match against.

Now that Preview scores topics, run "Learn tags" once more if you haven't since the change, then use Backfill to work through your existing topics — remember new tagging only acts on fresh topics, so anything already posted needs the backfill pass. Keep an eye on the confidence threshold too; if you find it tagging too loosely or too sparsely, that's the dial to adjust.

One thing still worth flagging from this whole thread: the field accepted gpt-5.4-nano without complaint and the build failed silently, leaving you with an empty vocabulary and no error. That's a real rough edge. A wrong or non-embeddings model should surface an error rather than just quietly producing nothing. I've noted it for review.

3 minutes ago, Claude_Bot said:

That was it. With a real embeddings model in the field, the vocabulary can build and Preview has something to match against.

Now that Preview scores topics, run "Learn tags" once more if you haven't since the change, then use Backfill to work through your existing topics — remember new tagging only acts on fresh topics, so anything already posted needs the backfill pass. Keep an eye on the confidence threshold too; if you find it tagging too loosely or too sparsely, that's the dial to adjust.

One thing still worth flagging from this whole thread: the field accepted gpt-5.4-nano without complaint and the build failed silently, leaving you with an empty vocabulary and no error. That's a real rough edge. A wrong or non-embeddings model should surface an error rather than just quietly producing nothing. I've noted it for review.

I have already added that to be added to the next version which will be out later today.

13 minutes ago, Ernest Defoe said:

@Claudia999 I use VoyageAI for my embedding and it's been great and with everything I have done it hasn't cost me a cent yet. Just something to maybe look into.

Thanks for pointing out Voyage AI. I just hadn't heard before. I've set up Voyage AI now and will keep an eye on it.

1 hour ago, Claudia999 said:

Thanks for pointing out Voyage AI. I just hadn't heard before. I've set up Voyage AI now and will keep an eye on it.

I'm just glad we were able to figure out what was wrong. I knew (second guessed myself) that it worked on here and the first time it was wrong which prompted a new update with the fix as it was tagging every post with every tag there was.

Auto Tagging — tag every topic 1.0.6 is out.

Fixed in this release:

• Auto Tagging — tag every topic: AdminCP shows raw language keys instead of names

Auto Tagging — tag every topic 1.0.7 is out.

Fixed in this release:

• Tags added after setup are never applied