Knowledge base

Things that cost me a day, so they cost you none

Working notes from building Invision Community and Flarum applications. Mostly the failures that give no error at all — the ones where everything installs cleanly and quietly does the wrong thing.

92 articles

Invision Community 5

86 articles

Extensions and contracts

What each extension point is for, what it must declare, and what happens when it is wrong — which is usually nothing visible.

Languages and text

The string table, translation, and the places where text does not appear where you expected it to.

Theming, templates and forms

Theme hooks, CSS that survives both colour schemes, and building forms that do not throw on render.

Background work and scheduled tasks

The queue system, work that has to happen after the response, and jobs that finish without doing anything.

Data, settings and storage

The database layer, settings, tags, file storage, and backing up a live site.

AI features and expectations

What these features do, what they cost, and what buyers reasonably but wrongly assume they do.

Application structure and releases

The JSON files an application is made of, versioning and upgrade steps, and testing from the command line.

Realtime, chat and calls

WebSocket gateways, relays and the server-side pieces live features depend on — where "it works when I test it" and "it works for your members" are different claims.

Your video calls work in testing and fail for a third of your membersNew

Flarum 2

6 articles

Nothing matches that.

Knowledge Base

Notes, gotchas and how-tos for Invision Community 5 and Flarum 2.

Show all categories

"You do not have permission for that" on your own AdminCP screen: the $csrfProtected property

You add an AdminCP controller. The list screen works. Then you click Add, or Edit, or a date-range link on your own report, and Invision Community answers:
Sorry, you do not have permission for that! You are logged in as a root administrator. Nothing is wrong with your ACP restrictions, and that is exactly where the message sends you to look.
The cause is CSRF, not permissions
IPS\Dispatcher\Admin runs this before dispatching:
if ( !isset( $this->classname::$csrfProtected ) and array_diff( array_keys( Request::i()->url()->queryString ), array( 'app', 'module', 'controller', 'id' ) ) ) { Session::i()->csrfCheck(); } Read it carefully. Any AdminCP URL carrying a query parameter other than app, module, controller or id is CSRF-checked automatically — unless the controller class declares $csrfProtected. And a failed CSRF check reports HTTP 403 with the language string admin_csrf_error, which renders as the permission message above.
So this is fine:
app=myapp&module=main&controller=things ← works app=myapp&module=main&controller=things&id=4 ← works, id is allowed and every one of these 403s:
...&controller=things&do=form ← "do" is not on the list ...&controller=things&do=settings ...&controller=report&days=90 ← your own filter links ...&controller=things&page=2 The fix
Declare the property. Its value is irrelevant — the dispatcher only calls isset() on it. It means "this controller takes responsibility for its own CSRF checks".
class things extends \IPS\Dispatcher\Controller { public static bool $csrfProtected = TRUE; // ... } Then call Session::i()->csrfCheck() yourself on the methods that actually change something — delete, toggle, reorder — and link to those with ->csrf() on the URL. Read-only screens such as a form or a report do not need a token.
Invision Community's own code declares this on essentially every AdminCP controller it ships. It is easy to miss precisely because it is boilerplate that nothing reminds you about.
Why nothing catches this before release
The failure is invisible to every check short of clicking the link:
The app installs cleanly. The menu entry appears. The default screen — no extra query parameters — renders perfectly. Rendering the form's fields in a test harness works, because the harness never goes through the dispatcher. Nothing is written to any log. Only a real HTTP request to a real do= URL as a signed-in administrator reveals it. If you have a screen-rendering test, make it fetch the actual AdminCP URLs rather than constructing the form objects.
A trap while you are testing this
Do not "fix" it by appending csrfKey to the URL. In developer mode Output::sendOutput() deliberately fatals on any 2xx response whose URL still contains a CSRF key:
An 200 response is being sent however the CSRF key is present in the requested URL. CSRF keys should be sent via POST or the request should be redirected to a URL not containing a CSRF key once finished. So without the property you get a 403, and with a token in the URL you get a 500. Both symptoms point away from the one-line cause.
Related: A stray "menutab_system" in your AdminCP: acpmenu.json tabs are not validated, Raw language keys in the AdminCP menu: acpmenu.json must be an object, What you can and cannot test from the command line

0 comments
43 views

Ernest Defoe

July 26Jul 26

"This theme may be out of date" usually means your form field, not the theme

You add a field to an AdminCP form, load the page, and get this:
[[Template core/global/forms/checkboxset is throwing an error. This theme may be out of date. Run the support tool in the AdminCP to restore the default theme.]] The theme is fine. Restoring it will not help. The message is what Invision Community prints when a template throws, and the template threw because of the options you passed to the field — but the message names the template, so it sends you to the support tool and the theme system instead of to the six lines you just wrote.
Why it points at the wrong place
Field construction does almost no validation. new CheckboxSet( ... ) stores your options and returns happily; the options are not read until the template renders them. So the stack you would want — "bad option in your controller" — does not exist by the time anything goes wrong. What is left is a template that could not cope with what it was handed.
Two consequences worth internalising: the error appears on page load rather than at the line you wrote, and it is worded as an infrastructure problem rather than a code one.
The usual cause: `unlimited` not matching the value
For a multi-select field with an "everything" option, three things have to agree. This is the shape core uses (see applications/core/modules/admin/applications/applications.php, the app_disabled_groups field):
$form->add( new CheckboxSet( 'my_groups', $allowed === '*' ? '*' : explode( ',', $allowed ), FALSE, array( 'options' => array_combine( array_keys( \IPS\Member\Group::groups() ), array_map( function ( $group ) { return (string) $group; }, \IPS\Member\Group::groups() ) ), 'multiple' => TRUE, 'unlimited' => '*', 'unlimitedLang' => 'my_groups_all', 'impliedUnlimited' => TRUE, ) ) ); 🚨 unlimited must be a scalar sentinel, and the field's value must be that same sentinel when everything is selected. Passing 'unlimited' => array() and a value of array() looks reasonable and produces the template error above.
🚨 options must be id => label. array_map() over Group::groups() looks like it builds that, but you lose the ids the moment the callback returns a string, so the checkboxes have nothing to submit. Hence the array_combine( array_keys( ... ), array_map( ... ) ) dance in core's own code — it is not superstition.
Then handle the sentinel on the way back out
With impliedUnlimited, the field returns the literal sentinel, not an array:
if ( $values['my_groups'] === '*' ) { $save = '*'; } else { $ids = array_filter( array_map( 'intval', (array) $values['my_groups'] ) ); $save = $ids ? implode( ',', $ids ) : '*'; } 🚨 Casting first is a real bug: (array) '*' is array('*'), which intval()s to 0 and gets filtered away, turning "everyone is allowed" into "nobody is allowed" — silently, and in the direction that hides things rather than exposing them.
Test it by rendering, not by constructing
Because construction does not validate, a test that builds the field proves nothing. Call html() and look for the marker:
$html = (string) $field->html(); $broken = str_contains( $html, 'is throwing an error' ) or str_contains( $html, 'This theme may be out of date' ); Two things make this awkward from the command line, both avoidable:
🚨 Do not boot Dispatcher\Admin::i() to "set up the ACP context". From CLI it tries to redirect to the login screen and dies on a missing REQUEST_METHOD. You do not need it: Theme::getTemplate() only consults the dispatcher when the location argument is omitted, and the form templates pass one. Some fields — Node, and anything reaching Request->url() — read $_SERVER directly. Stub the handful they want rather than building a request: $_SERVER['REQUEST_METHOD'] ??= 'GET'; $_SERVER['SERVER_NAME'] ??= 'localhost'; $_SERVER['SERVER_PORT'] ??= '80'; $_SERVER['HTTP_HOST'] ??= 'localhost'; $_SERVER['REQUEST_URI'] ??= '/admin/'; $_SERVER['SCRIPT_NAME'] ??= '/index.php'; $_SERVER['QUERY_STRING'] ??= ''; $_SERVER['HTTPS'] ??= ''; And assert that the broken version really is broken. A render check that passes whatever you feed it is worse than none, because it reads as coverage. Construct the field the wrong way on purpose, render it, and require that one to fail.
Verified against
Invision Community 5.0.19, against system/Helpers/Form/CheckboxSet.php, system/Theme/Dev/Theme.php and core's own app_disabled_groups and menu_manager_access fields.
Related: Form\Editor needs an EditorLocations extension that actually exists, Theme hook types are an enum of four, there is no "replace", and how to change the page title anyway, Custom CSS that works in both colour schemes, and the var() name trap

0 comments
47 views

Ernest Defoe

July 26Jul 26

Custom CSS that works in both colour schemes, and the var() name trap

Invision Community 5 renders in a light or a dark scheme. Custom CSS written while looking at one of them usually breaks in the other, and the way it breaks is quiet: nothing errors, nothing logs, and the developer's own screen looks correct.
How the two schemes work
Core declares its palette twice: light values at :root, then overrides under [data-ips-scheme="dark"]. The attribute is written onto <html> from front/global/htmlDataAttributes.phtml and can be light, dark, or system. When it is system, script resolves the OS preference and rewrites the attribute; data-ips-scheme-active keeps the unresolved choice.
So light is the default and dark is the override — not the other way round.
🚨 The trap: a wrong variable name is not an error
This is the one that will cost you a day.
/* Looks scheme-aware. Is not. */ background: var(--i-box--background, #171b21); --i-box--background does not exist. The correct name is --i-box--ba-co. When a custom property is undefined, var() quietly uses the fallback — so this rule paints #171b21 in both schemes. On dark it looks perfect. On light you get black cards on a white page, and because you wrote a fallback you will be certain the rule is scheme-aware and go looking somewhere else.
The abbreviations are not guessable, which is exactly why this happens. Verified names:
--i-box--ba-cobox background --i-box--bo-cobox border colour --i-box--bo-rabox radius --i-boxHeader--ba-cobox header background --i-background_1 … --i-background_6surface ramp, 1 = page, 6 = strongest --i-color_hard / --i-color_softprimary / secondary text --i-color_primary, --i-primaryaccent --i-design-radiustheme radius setting Names that look right and are wrong: --i-box--background, --i-box--border, --i-border, --i-background-color_1, --i-background-color_2.
Check before you rely on one. In the browser console:
getComputedStyle(document.documentElement).getPropertyValue('--i-box--ba-co') // empty string means the property does not exist and your fallback is what renders Worth a build-time sweep too: grep your CSS for --i-[a-z0-9_-]+ and compare against a known-good list. Finding this by eye across a catalogue of applications does not happen.
Declare your own tokens, twice
Rather than duplicating every rule per scheme, mirror what core does — but with your own prefix. Never redefine core's --i-* variables at :root; every component that depends on them changes underneath you.
:root{ /* On light a "raised surface" highlight has to be a DARK tint — a white highlight on a white card is invisible. */ --ed-lift: rgba(15, 23, 42, .035); --ed-hairline: rgba(15, 23, 42, .12); /* Light needs far less shadow opacity or it reads as soot. */ --ed-shadow: rgba(15, 23, 42, .16); --ed-art-bg: #e9eef4; --ed-code-bg: #f6f8fa; } [data-ips-scheme="dark"]{ --ed-lift: rgba(255, 255, 255, .055); --ed-hairline: rgba(255, 255, 255, .10); --ed-shadow: rgba(0, 0, 0, .85); --ed-art-bg: #0f141b; --ed-code-bg: #0d1117; } Two habits worth keeping: rgba(255,255,255,x) for hairlines and highlights is a dark-only assumption, and heavy rgba(0,0,0,.85) shadows that read as depth on dark read as dirt on light.
Not everything should follow the scheme
Some surfaces carry their own ground and must stay fixed. Tokenising these is a regression, not a fix:
A header band that is always a brand gradient — white text on it is correct in both schemes. A chip or badge sitting on top of a photograph or cover art, with its own dark backing. A darkening tint over a header photograph. Lighten it in light mode and the texture washes out and the logo stops standing off the background. Comment those, or the next person to "fix" the scheme handling will flatten them.
Testing it: the attribute is set once
🚨 data-ips-scheme is written at page load. Changing your OS scheme in another window does not update it — the stale value stays and your CSS appears broken when it is fine. Reload after every scheme change.
Also: computed values are not verification. Confirming that --ed-art-bg resolved to the light value proves the token works, not that the page looks right — a single un-tokenised rule elsewhere still ruins it. Look at the rendered page in both schemes.
Writing custom CSS from code
🚨 $theme->custom_css = $css; $theme->save(); is a silent no-op. The setter writes to _data while save() only persists $this->changed. Write the column and clear the compiled copy:
\IPS\Db::i()->update( 'core_themes', [ 'set_custom_css' => $css ], [ 'set_id=?', $id ] ); $key = $id . '_theme_editor_custom_css'; unset( \IPS\Data\Store::i()->$key ); Bonus: the 1px white seam on hover
A card with overflow: hidden, a border-radius, and a transform on hover, containing an image that also transforms, flashes a hairline of the page through its rounded corners. The rounded clip is re-rasterised on the card's compositing layer and leaks an antialiased edge.
🚨 The instinctive cures — transform: translateZ(0), will-change: transform — make it worse, because promoting the element to its own layer is what causes the clip to be rasterised separately in the first place.
Fix it geometrically instead: never let the image's edge sit at the clipping boundary.
.card{ overflow: clip; isolation: isolate; } /* clip on the image's OWN container, which never transforms, with the radius matched to the card's (16px card minus its 1px border) */ .card__image{ overflow: clip; border-radius: 15px 15px 0 0; } /* resting overscan: invisible on a photo, but the edge already sits outside the clip before the animation starts */ .card__image img{ transform: scale(1.02); } .card:hover .card__image img{ transform: scale(1.08); } overflow: clip is preferable to hidden here: it does not create a scroll container and rasterises more predictably. Note also that a hover box-shadow containing a hard 0 0 0 1px ring is itself indistinguishable from a flash — check for one before blaming the clip.
Verified against
Invision Community 5.0.19, variable names read from the live computed styles of a running community and from applications/core/dev/css/global/framework/1-3-variables.css and 1-4-colors.css.
Related: The Invision Community 4 CSS class that silently does nothing in 5, A container query cannot style its own container, Setting a theme custom_css and calling save() silently does nothing

0 comments
39 views

Ernest Defoe

July 26Jul 26

There is no event when a tag is created, and what that means for your app

Invision Community 5 fires 58 event-listener hooks. None of them concern tags.
Tags do not go through a model with a save() you can observe. They arrive through Content\Taggable::setTags(), which deletes the item's existing rows in core_tags and inserts the new ones directly. No event, no queue job, no notification. So there is no supported way to be told that a tag has come into existence.
The trap this sets
Any application whose state is derived from the tag list has to refresh that state somehow, and the easy place to do it is wherever the administrator already is:
if ( $values = $form->values() ) { $form->saveAsSettings( $values ); Palette::refresh(); // rebuild derived state ... } That is correct as far as it goes, and it is also the entire failure. If the only callers of your refresh are ACP controllers, then your app is up to date exactly when an administrator is looking at it and at no other time. A tag created by a member, arriving in an import, or added by another application is never picked up.
Worse, this fails invisibly and in the most misleading way possible: the ACP screen is always right, because visiting it is what makes it right. The administrator sees correct state on the settings page and wrong state on the front end, and concludes the app is broken rather than stale.
A tag-colouring application shipped exactly this. Colours were only ever assigned, and its stylesheet only ever rebuilt, from the admin screen. Every tag created after setup kept the theme's default colour indefinitely.
Do not solve it in the request cycle
core/Loader::onFinish() runs on every front-end request and is tempting. Resist it for this: detecting a new tag means a grouped query over core_tags, and running that on every page view of the community is not a reasonable price for a cosmetic feature. (For work that genuinely must happen per request, see Running code on every request, after the page has been sent.)
Solve it with a task
A scheduled sweep is what core itself uses for comparable housekeeping.
// data/tasks.json — the key must match the class and the filename {"myappscan": "P0Y0M0DT0H15M0S"} namespace IPS\myapp\tasks; class myappscan extends \IPS\Task { public function execute(): ?string { try { $done = Thing::refreshDerivedState(); return $done ? "Updated {$done} things" : NULL; } catch ( \Throwable $e ) { \IPS\Log::log( $e, 'myapp_scan' ); /* Swallowed deliberately: core disables a task that keeps throwing, and losing the sweep permanently is worse than missing one run. */ return NULL; } } public function cleanup(): void {} } Make the refresh free when nothing changed
🚨 A refresh written for a button press is usually written to assume something changed. Called every fifteen minutes, that becomes a settings write, and therefore a settings-cache invalidation across every node, four times an hour forever. Compare before writing:
public static function refresh(): string { $css = static::build(); $hash = $css === '' ? '' : substr( md5( $css ), 0, 10 ); if ( (string) Settings::i()->myapp_hash === $hash and (string) Settings::i()->myapp_css === $css ) { return $hash; // nothing to do, and now genuinely free } Settings::i()->changeValues( array( 'myapp_css' => $css, 'myapp_hash' => $hash ) ); return $hash; } 🚨 And the one that is easier to get wrong than any of the above
There are two tag tables and they mean different things:
core_tags_datathe tags defined for the community — what the ACP tag manager edits, and what Tag::getStore() returns (enabled only, recommended first, cached) core_tagswhich items carry which tag — usage, not existence Listing "the community's tags" by grouping core_tags looks right and is a real, plausible list. It is also wrong: a tag created in the tag manager and not yet applied to any content has no rows there, so it is invisible. The same application above shipped a screen for choosing tag colours that could not show a newly created tag at all — which is backwards, because creating a tag is exactly when you want to pick its colour, before anybody sees it in the default one.
Depending on what you are building you usually want the union, not either one:
/* defined — authoritative for spelling, since core_tags stores whatever case the item happened to be tagged with */ foreach ( \IPS\Content\Tag::getStore() as $tag ) { $key = mb_strtolower( (string) $tag ); $text[ $key ] = (string) $tag; $uses[ $key ] = $uses[ $key ] ?? 0; } Keep the usage side too: if the community allows free-form tags, members create tags that exist in core_tags and nowhere else.
The general shape is worth internalising. Any query of the form "distinct values in the usage table" silently means "things that have already happened", which is not the same as "things that exist". It fails quietly, because what it returns is genuinely a list of real tags — just not the list you asked for.
Two things that bite while wiring this up
A task file that is not declared in data/tasks.json is never registered and never runs, with nothing anywhere saying so — the same silent-manifest family as a malformed listeners.json. Worth a static check in both directions: every declared task has a file, and every file is declared.
🚨 A settings field takes its language string from its own key. If a key of that exact name already exists for something else, adding the setting silently relabels the other thing. A setting named myapp_auto where myapp_auto is already a button label renames the button, with no error and no warning — you find it by noticing the button says something odd. Grep dev/lang.php for a key before naming a setting after it.
Tell the administrator what is going to happen
Once the sweep is periodic, state is briefly stale by design. Say so where they will look: "3 tags have no colour yet — these will be coloured automatically within about fifteen minutes, or press Colour them for me now." Showing pending state without saying what will resolve it reads exactly like the bug you just fixed.
Verified against
Invision Community 5.0.19, by reading system/Content/Taggable.php, system/Task/Task.php, and the complete set of hooks in system/Events/Event.php.
Related: Taggable::setTags() deletes every existing tag first, Your classifier does not learn, and everyone will assume it does
Related application: Tag Colors — Tag Colours uses a scheduled scan for this reason, so a tag created by a member, by an import or by another application is picked up without an administrator opening the screen.

0 comments
40 views

Ernest Defoe

July 26Jul 26

Why a similarity threshold is never enough for automatic classification

If you build anything that decides "is this content about that thing" using embeddings — auto-tagging, related content, routing to a forum, spam scoring — the obvious design is a threshold. Score the item against each candidate, keep whatever clears a number, take the top few. It is one setting, it is easy to explain to an administrator, and it is wrong.
The failure
An auto-tagging application shipped with exactly that. On the community that installed it, it gave sixty consecutive topics the same three tags. Its threshold was 40%, its limit was three tags, and it hit the limit every single time.
The tag list was the reason:
Invision Application Flarum Extension Premium Flarum Extension Flarum Theme Premium Flarum Theme Reasonable tags for a site that sells those things. Measuring the stored tag vectors against each other:
0.827 Premium Flarum Extension | Premium Flarum Theme 0.823 Flarum Extension | Premium Flarum Extension 0.815 Flarum Theme | Premium Flarum Theme 0.747 Flarum Extension | Premium Flarum Theme 0.732 Invision Application | Premium Flarum Extension 0.701 Flarum Extension | Flarum Theme 0.684 Invision Application | Premium Flarum Theme 0.682 Flarum Extension | Invision Application 0.669 Flarum Theme | Premium Flarum Extension 0.590 Flarum Theme | Invision Application The least similar pair of tags on the whole site is 0.590. The vocabulary is one tight cluster, so any topic relevant to one tag is relevant-looking to all five, and no threshold separates them. Below ~0.59 everything is applied; above the best score nothing is. There is no value in between that behaves.
Why raising the threshold does not help
Cosine similarity has no absolute meaning. Its scale depends on the embedding model, on whether you embedded symmetrically or as query/document, on how long each side is, and on how tightly clustered your candidate set happens to be. A number tuned on one community is meaningless on the next.
Specifically, comparing a short label against a whole topic is asymmetric: those scores sit far lower than the 0.85+ you would expect from comparing two topics. So the developer picks a low threshold to make anything match at all — and a low threshold on a clustered vocabulary admits everything.
What to do instead
Keep the threshold, but demote it to a backstop, and add two relative tests. Sort candidates by score, then:
1. Floor. If the best candidate does not clear the threshold, the item is about something your candidate set does not cover. Return nothing. This is the only job an absolute number can do honestly: distinguishing "no match" from "some match".
2. Dominance. Accept the best candidate. Accept a second only if it scores close to the first — a proportion of the best score, not a fixed distance:
if ( $accepted and $row['score'] < $best * $dominance ) // 0.95 works well { continue; // sorted descending, so everything after this fails too } This is what stops the "and whatever else cleared the bar" behaviour. Note it is scale-free: it does not care what range the model's scores occupy.
3. Distinctness. This is the one that actually fixes a clustered vocabulary, and it is the one people leave out. Compare the candidate against the candidates already accepted, not against the content:
foreach ( $accepted as $already ) { if ( Vector::cosine( $row['vector'], $already['vector'] ) >= $distinct ) // 0.70 { continue 2; // a competing label, not an additional one } } "Premium Flarum Extension" and "Flarum Extension" are 0.823 alike. They are two answers to one question, and applying both is always wrong regardless of how the content scored. Because this measures candidate-to-candidate, it is independent of the item, of the model's scale, and of any per-community tuning — which is what makes it the reliable gate.
On the vocabulary above, these three take three tags per topic down to one, and it is the right one. On a vocabulary of genuinely separate subjects — billing, mobile app, server setup — pairwise similarity runs 0.1–0.3, the distinctness gate never fires, and genuinely multi-topic items still get multiple labels.
Two things worth building alongside it
Show the rejects. A wrong label and a right one look identical once applied, so a screen that lists only the winners cannot be used to diagnose anything. List every candidate with the reason it was kept or dropped — "too similar to Flarum Extension (82%)". In the case above the administrator had no way to tell whether the threshold, the model or the vocabulary was at fault, because the preview screen only showed successes.
Surface the overlap. The real cause was not in the code, it was in the tag list, and the person who created those tags had no reason to think of them as a problem. Comparing your candidate set against itself is cheap and needs no API call, so show which pairs overlap and by how much. It explains the behaviour before somebody concludes the app is broken.
And make bulk classification reversible. Anything that labels content in bulk will get it wrong in bulk at least once; if the only remedy is editing items one at a time, that is the real defect.
Testing it without an API key
You can build vectors with any pairwise similarities you like — Gram-Schmidt over the target Gram matrix — which means a real production failure can be reproduced exactly, with no network and no key, from nothing but the similarity numbers you measured:
/* each vector is placed so its dot product with every earlier one is the number asked for; unit length, so dot product IS cosine */ for ( $j = 0; $j < $i; $j++ ) { $dot = 0.0; for ( $d = 0; $d < $j; $d++ ) { $dot += $v[ $d ] * $out[ $j ][ $d ]; } $v[ $j ] = ( $target[ $j ] - $dot ) / $out[ $j ][ $j ]; } $v[ $i ] = sqrt( 1.0 - $used ); // $used = sum of squares placed so far 🚨 Do not use a candidate's own vector as the content vector. That scores 1.00, the dominance gate then rejects everything else on its own, and the distinctness gate never runs — so the test passes without testing the thing it exists to test. The first version of this test did exactly that and reported success. Build the content vector as a weighted mixture of the candidates plus an orthogonal component standing in for everything else in the text, and solve for how much orthogonal signal reproduces the score profile you actually observed. Real content lands in the middle of the cluster; that is the case that breaks.
Verified against
Invision Community 5.0.19, with Voyage voyage-3.5-lite embeddings. The similarity figures are measured from a live community's stored tag vectors.
Related: Your classifier does not learn, and everyone will assume it does, A chat model in an embeddings field fails silently, and looks like a broken app, Your AI assistant is only as good as what you gave it to read
Related application: Auto Tagging — Auto Tagging scores candidates relative to the best match rather than on a fixed threshold alone, and lists the tags in a vocabulary that overlap too closely to be told apart.

0 comments
39 views

Ernest Defoe

July 26Jul 26

Running code on every request, after the page has been sent

Some work has to happen on every page view — recording an analytic, touching a cache, noting that something was seen — and none of it should make the visitor wait. Invision Community supports this, but the pieces are not obvious and the method that looks right is the wrong one.
core/Loader is the only per-request extension point
Dispatcher\Front invokes exactly one extension point on every front-end request, and it invokes it three times:
// early, while building the page foreach ( Application::allExtensions( 'core', 'Loader' ) as $loader ) { foreach ( $loader->js() as $js ) { ... } foreach ( $loader->css() as $css ) { ... } } // before dispatch, first one wins foreach ( Application::allExtensions( 'core', 'Loader' ) as $loader ) { if ( $redirect = $loader->checkForRedirect() ) { Output::i()->redirect( $redirect ); } } // at the end foreach ( Application::allExtensions( 'core', 'Loader' ) as $loader ) { $loader->onFinish(); } Despite the name, core/Loader is not only for assets. onFinish() is a general end-of-request hook, and it is the supported place for per-request work.
onFinish() does NOT run after the response
This is the trap. Front::finish() calls onFinish() and then calls parent::finish(), which is what actually sends the page:
foreach ( Application::allExtensions( 'core', 'Loader' ) as $loader ) { $loader->onFinish(); } parent::finish(); // <- Output::sendOutput() happens in here So anything done directly inside onFinish() delays the response. A database write there is time the visitor spends looking at a blank tab.
The part that makes it work
Output::sendOutput() ends like this:
/* Flush and exit */ @ob_end_flush(); @flush(); /* If using PHP-FPM, close the request so that __destruct tasks are run after data is flushed to the browser */ if( function_exists( 'fastcgi_finish_request' ) ) { fastcgi_finish_request(); } exit; Core deliberately releases the connection and keeps executing — and exit still runs shutdown functions and destructors. So the pattern is:
Gather what you need in onFinish() from data already in memory, and register a shutdown function to do the writing.
public function onFinish(): void { try { $row = array( /* ... built from Request/Member/$_SERVER, no queries ... */ ); register_shutdown_function( static function () use ( $row ) { try { \IPS\Db::i()->insert( 'myapp_events', $row ); } catch ( \Throwable $e ) { /* the visitor already has their page */ } } ); } catch ( \Throwable $e ) { /* never break a page */ } } Under PHP-FPM the write happens after the browser has the page. Without FPM the shutdown function still runs after the output has been flushed, so it is never worse than doing the work inline.
Rules for anything on this hook
Do no reads. Use what is already in memory — Request::i(), Member::loggedIn(), $_SERVER. A query here runs on every page on the site. Swallow everything. This code runs during somebody reading a topic. An uncaught exception is their broken page. Guard against running twice with a static flag, in case the hook fires more than once in one request. Skip AJAX if you are counting page views — Request::i()->isAjax() — or every figure you produce will be inflated. One statement. Even after the response, the PHP-FPM worker is still occupied and cannot serve anyone else until the script ends. 🚨 And the one that takes the whole site down
If you register a core/Loader extension you must implement js() and css() correctly even when you have no assets, because the dispatcher merges each returned element:
foreach ( $loader->js() as $js ) { $jsFiles = array_merge( $jsFiles, $js ); } Each element must itself be an array. Returning array( $url ) throws inside the dispatcher and every front page on the site returns a 500 — not just yours, and with nothing in the trace naming the application responsible.
public function js(): array { return array(); } // safe: no assets public function css(): array { return array( array( $url ) ); } // an array OF arrays An empty array is safe, because the loop simply does not run.
Verified against
Invision Community 5.0.19, by reading system/Dispatcher/Front.php, system/Dispatcher/Dispatcher.php, system/Output/Output.php and system/Extensions/LoaderAbstract.php.
Related: Your background task is running inside a visitor's page load, The core/Queue extension in Invision Community 5, The report that shows nothing: only_full_group_by, and why catching the exception is the real bug, A realtime gateway will let any member join any channel unless you sign the channel into the token
Related application: Pulse — Pulse records page views with this pattern, gathering the row in onFinish() and writing it from a shutdown function once the response has gone.

0 comments
36 views

Ernest Defoe

July 26Jul 26

A malformed manifest silently disables your whole application

An application whose listeners.json or extensions.json does not parse installs perfectly, shows its AdminCP screens, saves its settings — and never runs a single line of its own automation. There is no error, no log entry, and nothing in the interface that looks wrong.
The cause
Core reads both manifests the same way, inside an and chain, with the warning suppressed:
if ( file_exists( $jsonFile ) and $json = @json_decode( file_get_contents( $jsonFile ), TRUE ) ) { foreach( $json as $filename => $data ) { // ... register } } A malformed file makes json_decode() return NULL. The and short-circuits, the loop never runs, and the method returns an empty array. The @ hides the only clue there would otherwise be.
The application is still enabled. Its tables exist. Its ACP module works. It simply contributes no listeners and no extensions.
The usual cause: a single backslash
These manifests are full of namespaced class names, and JSON has no \e, \M or \n-that-you-meant escape.
// INVALID - \e and \l and \M are not JSON escapes "classname": "IPS\myapp\listeners\MemberEvents" // CORRECT "classname": "IPS\\myapp\\listeners\\MemberEvents" Both look reasonable in an editor. Only one parses.
Worse, a single backslash before certain letters produces a valid escape that silently corrupts the value rather than failing outright. "IPS\nexus\\Invoice\\Item" parses cleanly and yields a string containing a literal newline followed by exus\Invoice\Item. The manifest loads, the class name is nonsense, and class_exists() quietly returns false.
The second silent failure: a class that is not there
Even with valid JSON, core checks before registering:
if( file_exists( $directory . '/' . $filename . '.php' ) ) { if( class_exists( $data['classname'] ) ) { $listeners[] = $data['classname']; } } Declare something you never wrote — easy to do when a manifest is created during scaffolding and the class comes later, or never — and it is skipped without a word.
How to detect it
Neither failure is visible at runtime, so check the files themselves. Two passes: does every manifest parse, and does every declared class have a file?
foreach ( glob( $root . '/*/data/*.json' ) as $file ) { $raw = file_get_contents( $file ); if ( trim( $raw ) === '' ) { continue; } json_decode( $raw, TRUE ); if ( json_last_error() !== JSON_ERROR_NONE ) { echo 'BROKEN ' . $file . ' - ' . json_last_error_msg() . PHP_EOL; } } Then, for each entry in listeners.json, confirm listeners/<key>.php exists; for each entry in extensions.json, confirm extensions/<app>/<type>/<key>.php exists. The key in the manifest is the filename.
Confirming it at runtime
If you suspect an application is contributing nothing, ask core directly rather than reading the file:
foreach ( \IPS\Application::enabledApplications() as $dir => $app ) { foreach ( (array) $app->listeners() as $class ) { echo $dir . ' :: ' . $class . PHP_EOL; } } foreach ( \IPS\Application::allExtensions( 'core', 'Loader' ) as $key => $ext ) { echo $key . PHP_EOL; } An application missing from that output has a manifest problem, whatever the file looks like. Clear the datastore first, since extension lists are cached.
Why this is worth a standing check
Every other mistake in an application announces itself somewhere — a fatal, a missing language string, a blank block. This one produces an application that is perfectly plausible and does nothing, and the natural response is to go and debug the code that was never invoked. Validating the manifests takes a second and rules it out for good.
Verified against
Invision Community 5.0.19, by reading Application::listeners() and Application::extensions() in system/Application/Application.php, and by shipping an application that had this exact fault.
Related: Raw language keys in the AdminCP menu: acpmenu.json must be an object, Application versions: why re-uploading the same version does nothing, Extension method signatures are fixed, and a mismatch is a fatal on install

0 comments
34 views

Ernest Defoe

July 26Jul 26

The core/ContentRouter extension in Invision Community 5

The core/ContentRouter extension is how an application tells the rest of Invision Community that a class is a content item. It is not a feature in itself — it is the list that roughly thirty other subsystems read to discover what content exists on the site: search, activity streams, the leaderboard, the member profile "Content" tab, moderator permissions, the Report Center, sitemaps, IP address lookup, achievement rules, follow records, warnings, advertisements, embeds and the REST/GraphQL content endpoints. If a class is not returned by a ContentRouter, none of those subsystems can see it, and no error is raised anywhere.
Core calls it constantly and from very different contexts, sometimes with a member, sometimes with a group, sometimes with nothing at all. That variety is where nearly every trap in this article comes from.
The contract
IPS\Extensions\ContentRouterAbstract is unusually small — it declares two properties and one abstract method, and the abstract method is the constructor.
abstract class ContentRouterAbstract { /* Content item classes this app exposes. Each entry must be a fully-qualified class name that is a subclass of IPS\Content\Item. */ public array $classes = array(); /* Whether these classes may appear in the "Similar Content" widget. Elasticsearch only. Defaults to FALSE, which actively EXCLUDES them. */ public bool $similarContent = FALSE; /* You MUST implement this. $member is NULL when the caller does not want an access check, a Member when it does, or a Group when the call came from the ACP group form. */ abstract public function __construct( Member|Group $member = NULL ); } Two further properties are read by core but are not declared on the abstract. Core only ever touches them behind isset(), in IPS\Text\Parser::_findItemFromUrl():
/* Node classes this app owns, for URL->embed resolution. e.g. IPS\blog\Blog, IPS\gallery\Album */ public array $ownedNodes = array(); /* Things that are embeddable but are NOT IPS\Content\Item subclasses. e.g. IPS\Member\Club, IPS\gallery\Album\Item */ public array $embeddableContent = array(); There is one optional method, discovered by method_exists() rather than declared anywhere:
public function customTableHelper( string $className, Url $url, array $where = array() ): ?Content Every routed class in $classes must, at minimum, declare the statics $application, $module, $title, $databaseTable, $databasePrefix, $databaseColumnId and $databaseColumnMap. $commentClass, $reviewClass, $containerNodeClass and $archiveClass are optional and are always tested with isset().
A minimal example
For an app in applications/acme with a content item IPS\acme\Ticket that lives in the front module tickets. The file is applications/acme/extensions/core/ContentRouter/Tickets.php:
<?php namespace IPS\acme\extensions\core\ContentRouter; use IPS\Application\Module; use IPS\Extensions\ContentRouterAbstract; use IPS\Member; use IPS\Member\Group; use function defined; if ( !defined( '\IPS\SUITE_UNIQUE_KEY' ) ) { header( ( $_SERVER['SERVER_PROTOCOL'] ?? 'HTTP/1.0' ) . ' 403 Forbidden' ); exit; } class Tickets extends ContentRouterAbstract { /** * @brief Allow these items in the Similar Content widget */ public bool $similarContent = TRUE; /** * @param Member|Group|null $member */ public function __construct( Member|Group|null $member = NULL ) { if ( $member === NULL or $member->canAccessModule( Module::get( 'acme', 'tickets', 'front' ) ) ) { $this->classes[] = 'IPS\acme\Ticket'; } } } The matching entry in applications/acme/data/extensions.json:
{ "core": { "ContentRouter": { "Tickets": "IPS\\acme\\extensions\\core\\ContentRouter\\Tickets" } } } Fatal error: Class ... contains 1 abstract method and must therefore be declared abstract
The exact text is:
Fatal error: Class IPS\acme\extensions\core\ContentRouter\Tickets contains 1 abstract method and must therefore be declared abstract or implement the remaining methods (IPS\Extensions\ContentRouterAbstract::__construct) Most IPS extension abstracts declare no abstract members, so it is natural to write a ContentRouter as nothing but a property:
class Tickets extends ContentRouterAbstract { public array $classes = array( 'IPS\acme\Ticket' ); /* WRONG - fatal */ } ContentRouterAbstract::__construct() is abstract. You must declare a constructor even if it does nothing conditional. The fatal is a compile-time error triggered by the autoloader during class_exists() inside IPS\Application::extensions(), which means it fires on essentially every page of the site, front and ACP, not just yours.
The extension file exists but is never called
Nothing happens. No error, no entry in the system log, your class simply does not exist as far as the suite is concerned.
IPS\Application::extensions() does not scan the extensions/ directory. It reads applications/<app>/data/extensions.json and nothing else:
foreach ( $json[ $app ][ $extension ] as $name => $classname ) { if( !is_string( $classname ) or !class_exists( $classname ) ) { /* Switching between branches confuses extensions */ continue; } ... } Two ways to hit this. First, you created the PHP file by hand in your editor and never touched extensions.json. That file is only ever rewritten by Application::buildExtensionsJson(), which is called from exactly two places, both in applications/core/modules/admin/developer/extensions.php: the Developer Center's "Create New Extension" and "Delete" actions. Nothing else regenerates it — not the build process, not an upgrade. Either add the extension through the Developer Center or edit the JSON yourself.
Second, the class name inside the file does not match the key/path in the JSON. The JSON value is derived from the file path, so Tickets.php must declare class Tickets. Note that a pure case difference survives, because PHP class names are case-insensitive — core relies on this in applications/gallery/extensions/core/ContentRouter/gallery.php, which declares class Gallery and registers ...\ContentRouter\gallery. A genuinely different name does not survive: class_exists() returns FALSE and the continue above skips it in total silence.
I added the extension but the site behaves as if it is not there
Same symptom as above, different cause, and this one bites even when extensions.json is correct.
IPS\Application::allExtensions() caches the resolved class-name map in the datastore under the key extensions:
try { $allExtensions = Store::i()->extensions; } catch( OutOfRangeException ) { $allExtensions = []; } if( !array_key_exists( $extension, $allExtensions ) ) { /* ... walk every application, build the list ... */ Store::i()->extensions = $allExtensions; } The cache is keyed by extension type only. Once ContentRouter is in there, the directory walk never runs again. unset( Store::i()->extensions ) happens when you add or remove an extension through the Developer Center, and the extensions key is listed in Application::$caches so it is cleared when an application record is saved (install, enable, disable, uninstall). Dropping a file in over FTP and editing the JSON by hand clears nothing. Use ACP → Support → Clear Caches, or save the application.
My content vanished from search and sitemaps site-wide, and I cannot reproduce it
This is the same cache, but the failure mode is much worse and it is worth understanding precisely because the symptom does not point at the cause.
The datastore-cached list is built after application-level access filtering, and the filtered result is what gets persisted for everyone:
foreach ( $apps as $application ) { /* Skip third party apps if recovery mode is enabled */ if( RECOVERY_MODE and !in_array( $application->directory, IPS::$ipsApps ) ) { continue; } if ( !static::appIsEnabled( $application->directory ) ) { continue; } if( $checkAccess !== FALSE ) { if( !$application->canAccess( $checkAccess === TRUE ? NULL : $checkAccess ) ) { continue; } } /* ... collect this app's extension class names ... */ } Store::i()->extensions = $allExtensions; Whoever triggers the rebuild decides the contents of the cache for every subsequent visitor. Two concrete consequences:
Recovery mode. If the ContentRouter list is first built while RECOVERY_MODE is on, every third-party app is omitted, and that list is written to the datastore. Turning recovery mode back off does not clear it. Your app's content stays missing from search, sitemaps, moderator permissions and the profile Content tab until something else clears the cache. The apparent cause (recovery mode) and the symptom (content missing days later) are far apart. Group-restricted apps. If your application has disabled_groups set and the first request after a cache clear is a guest, canAccess() returns FALSE, your app is skipped, and the resulting cache is served to admins too. You cannot fix this from inside your extension. It is a reason to treat the constructor's $member check as the real gate (see below) and to clear caches after any recovery-mode session.
The extension silently returns nothing and there is no error at all
Your ContentRouter class is registered, cached, and instantiated — and still no classes reach any consumer. This is the most common ContentRouter bug and it is deliberately invisible.
Application::constructExtensionClass() swallows two exception types from your constructor:
try { $obj = new $classToUse( ... ); ... return $obj; } catch( RuntimeException | OutOfRangeException $e ){} return null; Returning null means the extension is dropped from the returned array entirely. And IPS\Application\Module::get() — which the core template for this extension tells you to call — throws a bare OutOfRangeException when the module cannot be found:
public static function get( string $app, string $key, string $area=NULL ): Module { $modules = static::modules(); if ( isset( $modules[ $app ] ) ) { $area = $area ?: Dispatcher::i()->controllerLocation; if ( isset( $modules[ $app ][ $area ] ) ) { if ( isset( $modules[ $app ][ $area ][ $key ] ) ) { return $modules[ $app ][ $area ][ $key ]; } } } throw new OutOfRangeException; } So a typo in the module key, a module that exists in admin but not front, or an app whose modules were never installed, all produce the same result: your whole extension disappears with no log entry. Check the module key against core_modules (sys_module_application, sys_module_key, sys_module_area = 'front').
Always pass the third argument. Module::get( 'acme', 'tickets' ) falls back to Dispatcher::i()->controllerLocation, which is 'admin' inside the ACP and undefined under CLI tasks — so a router that works on the front end will quietly evaporate in the ACP moderator-permission form and in cron.
TypeError when editing a member group in the ACP
You see something like:
TypeError: IPS\acme\extensions\core\ContentRouter\Tickets::__construct(): Argument #1 ($member) must be of type ?IPS\Member, IPS\Member\Group given on ACP → Members → Groups → Edit — a screen that has nothing to do with your app.
IPS\core\extensions\core\GroupForm\Content passes a Group, not a Member:
foreach ( Application::allExtensions( 'core', 'ContentRouter', $group ) as $ext ) and that value is handed straight to your constructor. A TypeError extends Error, not Exception, so the catch( RuntimeException | OutOfRangeException ) above does not catch it. The group edit form dies outright, and the stack trace names your extension on a page owned by core. Type the parameter as Member|Group|null. Both classes implement canAccessModule( Module $module ): bool, so the body needs no branching.
Error: Access to undeclared static property on the member profile Content tab
Typical text: Error: Access to undeclared static property IPS\acme\Ticket::$application, seen on /profile/1-someone/?do=content, on the Leaderboard, or on every front-end page of your own app.
Consumers read the statics of everything you put in $classes with no guard. In applications/core/modules/front/members/profile.php:
$types[ $class::$application . '_' . $class::$module ][ ... ] = $class; In system/Dispatcher/Front.php, on every page load of your app:
if ( SearchContent::isSearchable( $class ) and $this->module->key == $class::$module ) None of $application, $module, $title is declared with a default on IPS\Content or IPS\Content\Item, so omitting them in your item class is an Error, not a warning. ($commentClass is declared — public static ?string $commentClass = NULL; — which is why core can safely write isset( $class::$commentClass ) everywhere.)
Two related crashes from the same omission, both outside your app:
IPS\core\extensions\core\ModeratorPermissions\ContentGenerator::getPermissions() calls Application::load( $class::$application ) with no try/catch. A $application value that is not an installed app directory throws OutOfRangeException and takes out ACP → Members → Staff → edit moderator. IPS\core\extensions\core\Sitemap\Content::getFilenames() calls Module::get( $class::$application, $class::$module, 'front' ) inside a try { } catch ( OutOfRangeException ) { return array(); }. Here the wrong module key does not crash — it silently returns no sitemap files, so your content is simply absent from sitemap.php forever. Class IPS\acme\Ticket not found, thrown from a core file
Core validates the extension class with class_exists(). It never validates the strings inside $this->classes. A renamed or removed content class, or a string with a typo, produces an uncatchable "Class not found" from wherever the list is consumed — system/Content/Search/Mysql/Query.php, system/Member/Member.php::topMembersOptions(), the profile controller — never from your file. Since $classes is a plain array of strings, the only defence is your own:
if ( class_exists( 'IPS\acme\Ticket' ) ) { $this->classes[] = 'IPS\acme\Ticket'; } This matters most for apps that build the list dynamically. IPS\cms\extensions\core\ContentRouter\Records does exactly this, wrapping its whole loop:
try { foreach ( Databases::databases() as $id => $database ) { ... $this->classes[] = 'IPS\cms\Records' . $id; } } catch ( Exception $e ) {} // If you have not upgraded pages but it is installed, this throws an error Raw language keys appear on the Leaderboard and the moderator permission form
You see literal text such as acme_ticket_pl or can_edit_acme_ticket instead of readable names.
$class::$title is a language key prefix, not a title. Consumers append suffixes to it and pass the result to addToStack(), which returns the key itself when the key is missing. From the source, you need at least:
KeyUsed by <title>Achievement rule filters (ContentAchievementActionAbstract::filters()), the mostContributions widget <title>_plLeaderboard areas (discover/popular.php), sitemap settings, IP address lookup <title>_pl_lcTop Members filters (Member::topMembersOptions()) can_edit_<title>, can_delete_<title>, can_hide_<title>, etc.Generated moderator permissions If your item has a $containerNodeClass you additionally need <nodeTitle>_sg and <nodeTitle>_sg_lc. Core's own defensive code confirms these are expected to be missing sometimes — ContentAchievementActionAbstract wraps the lookup:
try { $nodeTitle = Member::loggedIn()->language()->get( ($class::$containerNodeClass)::$nodeTitle . '_sg_lc' ); } catch( UnderflowException $e ) { $nodeTitle = ($class::$containerNodeClass)::$nodeTitle . '_sg'; } Only one of my app's two ContentRouter extensions appears in the Report Center filters
If your app registers more than one ContentRouter — as cms does, with Pages and Records — the Report Center's per-app filter only reflects one of them, the last one alphabetically by extension key.
applications/core/extensions/core/ModCp/Reports.php:
$apps = []; foreach ( Application::allExtensions( 'core', 'ContentRouter') as $app => $classes ) { $appKey = explode('_', $app)[0]; $apps[ $appKey ] = $classes; } allExtensions() keys its return value as <appDirectory>_<extensionKey>, so cms_Pages and cms_Records both reduce to cms and the second assignment overwrites the first. The variable is also misnamed — $classes holds the router object, not an array — and is immediately re-read as $data->classes in the next loop. The practical rule: put everything in one ContentRouter per app unless you have a reason not to.
The same line has a second edge: explode('_', $app)[0] truncates at the first underscore, and the result is then used as Application::applications()[ $appKey ] with no isset(). An application directory containing an underscore will throw an undefined-index error on the shared ModCP Reports screen, for every moderator, caused by an app that is otherwise working fine.
My items never show up in the Similar Content widget
$similarContent defaults to FALSE, and FALSE is not neutral — it is an active exclusion. IPS\Content\Search\Elastic\Query::filterByMoreLikeThis():
foreach ( Application::allExtensions( 'core', 'ContentRouter', FALSE ) as $object ) { $classes = array_merge( $object->classes, $classes ); if ( empty( $object->similarContent ) ) { foreach( $object->classes as $class ) { $noSimilarContentClasses[] = $class; if ( is_subclass_of( $class, 'IPS\Content\Item' ) and isset( $class::$commentClass ) ) { $noSimilarContentClasses[] = $class::$commentClass; } } } } Those classes end up in a must_not clause. Set public bool $similarContent = TRUE; to opt in.
Note the scope carefully: this property is read in one place in the entire suite, and that place is the Elasticsearch query builder. IPS\Content\Controller::getSimilarContent() only calls filterByMoreLikeThis() when Settings::i()->search_method == 'elastic'. On a MySQL-search site the similar-content widget uses tag matching via core_tags and $item->similarContentFilter() instead, and $similarContent has no effect whatsoever. Testing this on a MySQL site will tell you nothing.
Putting a node or a non-item class in $classes breaks unrelated screens
Every consumer of $classes assumes IPS\Content\Item. Adding a IPS\Node\Model subclass, or any other embeddable object, produces failures on the profile Content tab, the leaderboard and the ACP statistics charts, all of which call $class::supportsComments(), $class::$databaseColumnMap or $class::reactionType().
Core carves these out into the two undeclared properties. IPS\core\extensions\core\ContentRouter\Clubs registers IPS\Member\Club — which is not a content item — and leaves $classes empty:
class Clubs extends ContentRouterAbstract { public array $embeddableContent = array(); public function __construct( Member|Group|null $memberOrGroup = NULL ) { if ( $memberOrGroup === NULL or $memberOrGroup->canAccessModule( Module::get( 'core', 'clubs', 'front' ) ) ) { $this->embeddableContent[] = 'IPS\Member\Club'; } } } Blog uses $ownedNodes = array( 'IPS\blog\Blog' ); Gallery uses both. Both properties are read only by IPS\Text\Parser when resolving a pasted URL into an embed, and only for classes that implement IPS\Content\Embeddable:
$classes = $extension->classes; if ( isset( $extension->ownedNodes ) ) { $classes = array_merge( $classes, $extension->ownedNodes ); } if ( isset( $extension->embeddableContent ) ) { $classes = array_merge( $classes, $extension->embeddableContent ); } $classes = array_filter( $classes, function( $class ) { return in_array( 'IPS\Content\Embeddable', class_implements( $class ) ); } ); The isset() guards exist precisely because these properties are not on the abstract. Declaring them is optional; declaring them with the wrong contents is not recoverable.
My access check in the constructor is being ignored
You gate $this->classes on canAccessModule(), then find your content classes exposed to members who should not see them — in embed previews, attachment lookups, follow records or GraphQL.
There are two entry points with different defaults, and they are easy to confuse:
CallWhat your constructor receives Application::allExtensions( 'core', 'ContentRouter' )Member::loggedIn() ($checkAccess defaults to TRUE) Application::allExtensions( 'core', 'ContentRouter', FALSE )NULL $application->extensions( 'core', 'ContentRouter' )NULL — $checkAccess defaults to FALSE here The per-application form defaults the opposite way to the global one. From Application::extensions():
public function extensions( Application|string $app, string $extension, bool $construct=TRUE, bool|Group|Member|null $checkAccess = FALSE ): array Core uses that form, with the default, in IPS\Text\Parser (URL embeds), applications/core/modules/front/system/ajax.php (attachment info), IPS\core\Warning, IPS\core\Followed\Follow, IPS\core\Advertisement, system/Dispatcher/Front.php and several GraphQL resolvers. In all of those your constructor is handed NULL and every class you can return is returned.
The consequence: the member check in a ContentRouter constructor is advisory, not a permission boundary. It controls which screens offer your content type; it does not control who can load an object. Real access control belongs in canView() / loadAndCheckPerms() on the content class itself, which is what those call sites go on to invoke.
customTableHelper is called with my comment and review classes too
If you implement customTableHelper(), the member profile controller registers your router as the handler for the item class and its comment and review classes. From applications/core/modules/front/members/profile.php:
if( method_exists( $router, 'customTableHelper' ) ) { $hasCallback[ ... $class ... ] = $router; if ( $supportsComments ) { $hasCallback[ ... $class::$commentClass ... ] = $router; } if ( $supportsReviews ) { $hasCallback[ ... $class::$reviewClass ... ] = $router; } } There is no interface and no abstract declaration — presence of the method is the whole contract, so you get no signature checking either. Your implementation must therefore expect a $className it never registered. Gallery's guard is the pattern to copy:
public function customTableHelper( string $className, Url $url, array $where=array() ): ?Content { if( !in_array( $className, $this->classes ) or $className == 'IPS\gallery\Album\Item' ) { return new Content( $className, $url, $where, null, Filter::FILTER_AUTOMATIC, 'read' ); } /* ... custom table for the classes we actually own ... */ } Note that $this->classes may be empty at this point if the current member failed your constructor's access check, in which case the guard sends everything down the generic path — which is the safe outcome.
SQL error on the ACP Community Activity chart, but only on Elasticsearch sites
IPS\core\extensions\core\Statistics\CommunityActivity::getResults() queries core_search_index when the search method is MySQL, but falls back to querying each routed class's own table when it is not:
foreach ( Application::allExtensions( 'core', 'ContentRouter' ) as $contentRouter ) { foreach ( $contentRouter->classes as $class ) { if ( isset( $class::$databaseColumnMap['author'] ) ) { $results = array_replace_recursive( $results, $this->getSqlResults( $class::$databaseTable, $class::$databasePrefix . ( $class::$databaseColumnMap['updated'] ?? $class::$databaseColumnMap['date'] ), $class::$databasePrefix . $class::$databaseColumnMap['author'], $chart ) ); } } } The ?? only protects the 'updated' lookup. If your $databaseColumnMap has an author entry but neither updated nor date, the second lookup emits an undefined-key notice, resolves to an empty string, and the chart runs a query against a column name that is just your table prefix. If either entry maps to an array — which is legal and common — you get "Array to string conversion" and the same broken column.
Compare IPS\core\extensions\core\MemberACPProfileBlocks\Header, which walks the same class list and handles both cases explicitly:
if ( isset( $class::$databaseColumnMap['date'] ) ) { $dateColumn = $class::$databaseColumnMap['date']; } elseif ( isset( $class::$databaseColumnMap['updated'] ) ) { $dateColumn = $class::$databaseColumnMap['updated']; } else { continue; } if ( is_array( $dateColumn ) ) { $dateColumn = array_pop( $dateColumn ); } The asymmetry is the tell. Give every routed class a scalar date entry in $databaseColumnMap whenever it has an author entry — the failure surfaces on an ACP statistics page nobody associates with your app, and only on sites using Elasticsearch.
What ContentRouter does not do
Two things developers routinely expect from this extension and do not get. Registering a class here does not make it searchable — search indexing is driven by a separate core/SearchContent extension, and SearchContent::searchableClasses() intersects that list with Content::routedClasses( $member ), so you need both. And it does not create a Sitemap, IpAddresses or ModeratorPermissions extension for your app; rather, core's own generated extensions (subclasses of IPS\Content\ExtensionGenerator) iterate Content::routedClasses() and produce one instance per routed class. That is why a broken ContentRouter causes moderator permissions and sitemap entries to disappear rather than to error.
Verified against
Read from the source of Invision Community 5.0.19 (latest upgrade step applications/core/setup/upg_5001908), PHP 8.1–8.3 per applications/core/data/requirements.json. Key files: system/Extensions/ContentRouterAbstract.php, system/Application/Application.php (allExtensions(), extensions(), constructExtensionClass(), buildExtensionsJson()), system/Content/Content.php (routedClasses()), system/Content/ExtensionGenerator.php, system/Text/Parser.php, system/Content/Search/Elastic/Query.php, system/Content/Search/Mysql/Query.php, applications/core/modules/front/members/profile.php, applications/core/extensions/core/ModCp/Reports.php, applications/core/extensions/core/Sitemap/Content.php, applications/core/extensions/core/ModeratorPermissions/ContentGenerator.php, applications/core/extensions/core/GroupForm/Content.php, applications/core/extensions/core/Statistics/CommunityActivity.php, and all eight core implementations under applications/*/extensions/core/ContentRouter/.
Related: The core/Sitemap extension in Invision Community 5, Your friendly URLs have the app name in them twice, Extension method signatures are fixed, and a mismatch is a fatal on install

0 comments
36 views

Ernest Defoe

July 26Jul 26

The core/Dashboard extension: ACP dashboard blocks in Invision Community 5

The core/Dashboard extension point adds a block (widget) to the AdminCP dashboard at app=core&module=overview&controller=dashboard. Core calls it in two completely separate places: once on the full page render in IPS\core\modules\admin\overview\dashboard::manage(), and again — through a different code path with different arguments — on the AJAX endpoint do=getBlock that the dashboard's JavaScript hits every time a block is added, dragged or refreshed.
The contract
system/Extensions/DashboardAbstract.php declares four members, but only two of them are ever called by core.
abstract class DashboardAbstract { /* REQUIRED. Called on every dashboard page load, for every registered Dashboard extension in every enabled app, whether or not the admin has your block on their dashboard. Return FALSE to hide the block from the "Add" dropdown entirely. */ abstract public function canView(): bool; /* REQUIRED. Must return a string of HTML. Called only for blocks the admin actually has placed, and again on the do=getBlock AJAX route. */ abstract public function getBlock(): string; /* DEAD CODE in 5.0.19. Nothing calls it. See below. */ public function getInfo(): array { return array(); } /* DEAD CODE in 5.0.19. Nothing calls it. See below. */ public function saveBlock() {} } Registration is by file plus a data/extensions.json entry. The block's key is {appDirectory}_{ExtensionName} — built in Application::allExtensions() as $application->directory . '_' . $key — and its title comes from the language key block_{appDirectory}_{ExtensionName}, not from anything your class returns.
A minimal example
Four things are needed. First, applications/myapp/extensions/core/Dashboard/ServerLoad.php:
<?php namespace IPS\myapp\extensions\core\Dashboard; use IPS\Extensions\DashboardAbstract; use IPS\Member; use IPS\Theme; use function defined; if ( !defined( '\IPS\SUITE_UNIQUE_KEY' ) ) { header( ( $_SERVER['SERVER_PROTOCOL'] ?? 'HTTP/1.0' ) . ' 403 Forbidden' ); exit; } class ServerLoad extends DashboardAbstract { public function canView(): bool { return Member::loggedIn()->hasAcpRestriction( 'myapp', 'settings', 'myapp_settings_manage' ); } public function getBlock(): string { /* Re-check here too: the AJAX route does NOT call canView() */ if ( !$this->canView() ) { return ''; } $load = function_exists( 'sys_getloadavg' ) ? sys_getloadavg() : array( 0, 0, 0 ); /* Pass the app key AND the location explicitly */ return (string) Theme::i()->getTemplate( 'dashboard', 'myapp', 'admin' )->serverLoad( $load ); } } Second, applications/myapp/data/extensions.json:
{ "core": { "Dashboard": { "ServerLoad": "IPS\\myapp\\extensions\\core\\Dashboard\\ServerLoad" } } } Third, the language string in applications/myapp/dev/lang.php. The key is not optional:
'block_myapp_ServerLoad' => "Server Load", Fourth, applications/myapp/dev/html/admin/dashboard/serverLoad.phtml:
<ips:template parameters="$load" /> <div class='ipsPad'> <p>{$load[0]} / {$load[1]} / {$load[2]}</p> </div> The block title renders as the literal text "block_myapp_ServerLoad"
You see the raw key in both the widget header and the "Add" dropdown. Your class never gets a chance to supply a title. In manage() core builds the block name itself:
$info[ $key ] = array( 'name' => Member::loggedIn()->language()->addToStack('block_' . $key ), 'key' => $key, 'app' => substr( $key, 0, strpos( $key, '_' ) ) ); and widgetWrapper.phtml re-resolves the same key with {lang="block_{$info['key']}"}. When a key has no row in core_sys_lang_words, Lang::replaceWords() falls back to the key itself:
else { $values['options']['escape'] = TRUE; $replacement = $values[ 'key' ]; } Add block_{appDirectory}_{ExtensionName} to your app's lang.php and rebuild the language pack. There is no way to override this from the extension class.
getInfo() and saveBlock() do nothing — the docblock describes an API that no longer exists
The abstract's docblock for getInfo() promises array( 'name' => 'Block title', 'key' => 'unique_key', 'size' => [1,2,3], 'by' => 'Author name' ). Grepping the whole 5.0.19 file set turns up exactly one occurrence of getInfo in the extension namespace: its own definition in DashboardAbstract.php. Nothing calls it. There is no size handling anywhere; the dashboard has a fixed 8/4 two-column grid in dashboard.phtml. The only surviving trace of the old API is a dangling {{by}} placeholder in ips.templates.dashboard.js, which addWidget() never populates.
saveBlock() is the same story — declared, documented as throwing LogicException, and called by nothing. The one core block that saves submitted data, core_AdminNotes, does it inside getBlock() by posting a form back to the do=getBlock route and handling $form->values() there. The stub that the Developer Center generates from applications/core/data/defaults/extensions/Dashboard.txt omits both methods, which is the honest version of the contract.
If you override either method expecting it to be honoured, you get no error and no effect — just a block with no title and no save behaviour.
The whole ACP dashboard 500s and the error blames core, not your app
This is the most important trap in this extension point. manage() calls your code with no protection at all:
foreach ( Application::allExtensions( 'core', 'Dashboard', TRUE, 'core' ) as $key => $extension ) { if ( $extension->canView() ) { ... $blocks[ $key ] = $extension->getBlock(); } } No try, no catch, no per-block isolation. Any uncaught throwable from any installed app's canView() or getBlock() takes down the entire dashboard for every administrator. The error page and the system log will name IPS\core\modules\admin\overview\dashboard, so the first thing an admin does is blame core or the last core upgrade. Your bug surfaces on a screen owned by core, in an app you did not write, with a stack trace that only mentions your class deep inside.
The exposure is wider than it looks, because canView() runs for every registered Dashboard extension on every dashboard load, whether or not any admin has ever added your block. A canView() that queries a table your migration has not created yet will break the dashboard on the very first page an admin sees after installing your app.
Keep canView() to permission checks and settings reads, and wrap everything in getBlock():
public function getBlock(): string { try { return (string) Theme::i()->getTemplate( 'dashboard', 'myapp', 'admin' )->serverLoad( $this->data() ); } catch ( \Throwable $e ) { \IPS\Log::log( $e, 'myapp_dashboard' ); return ''; } } "Return value must be of type string, null returned"
getBlock(): string carries a declared return type on the abstract, so returning NULL, an array, or a template object that has no __toString() is a TypeError, not a silent empty block. Because of the previous trap, that TypeError is an ACP-wide dashboard failure rather than a broken widget. Cast template output explicitly — return (string) Theme::i()->getTemplate(...)->myBlock(); — and return '', never NULL, for the "nothing to show" case.
The block renders on page load, then comes back blank when it is dragged or re-added
Dragging a widget, adding one from the dropdown, or firing refreshWidget does not re-render the page. ips.dashboard.main.js issues:
ips.getAjax()( '?app=core&module=overview&controller=dashboard&do=getBlock', { data: { appKey: key.substr( 0, key.indexOf( '_' ) ), blockKey: key } }) and the server side is a second, hand-rolled lookup that does not use allExtensions() at all:
$output = ''; foreach( Application::load( Request::i()->appKey )->extensions( 'core', 'Dashboard', 'core' ) as $key => $_extension ) { if( Request::i()->appKey . '_' . $key == Request::i()->blockKey ) { if( method_exists( $_extension, 'getBlock' ) ) { $output = $_extension->getBlock(); } break; } } Output::i()->output = $output; If the key comparison never matches, $output stays '' and you get an empty widget with no error, no log entry, and a 200 response. The comparison fails when the app key derived by JavaScript is wrong, and JavaScript splits on the first underscore. An app directory containing an underscore (say my_app) produces appKey = "my", Application::load('my') throws or returns the wrong app, and the block silently never reloads even though it rendered perfectly on the initial page load. Extension class names may contain underscores safely; app directories may not.
Note also the method_exists( $_extension, 'getBlock' ) guard. Core is re-checking the existence of a method its own abstract declares abstract. That guard exists because extensions.json is only validated with class_exists() in Application::extensions() — nothing verifies that your class actually extends DashboardAbstract. The two call sites disagree about this: the AJAX route guards getBlock and degrades to an empty string, while manage() calls canView() with no guard at all. A Dashboard extension that forgets extends DashboardAbstract therefore fatals the whole dashboard on page load but returns a quiet empty widget over AJAX.
"Argument #1 ($member) must be of type IPS\Member, null given" — but only when the block reloads
The two call paths construct your class with different arguments. Application::constructExtensionClass() does:
$obj = new $classToUse( $checkAccess === TRUE ? Member::loggedIn() : ( $checkAccess === FALSE ? NULL : $checkAccess ) ); manage() uses allExtensions( 'core', 'Dashboard', TRUE, 'core' ), so $checkAccess is TRUE and your constructor receives Member::loggedIn(). The AJAX route uses Application::extensions( 'core', 'Dashboard', 'core' ), whose signature is extensions( $app, $extension, bool $construct=TRUE, bool|Group|Member|null $checkAccess = FALSE ) — the string 'core' lands in $construct (coerced to TRUE) and $checkAccess stays at its default FALSE, so your constructor receives NULL.
Declare no constructor, or make the parameter nullable:
public function __construct( ?Member $member = NULL ) { $this->member = $member ?: Member::loggedIn(); } Getting this wrong is invisible on the dashboard page itself and only bites the AJAX reload. Worse, constructExtensionClass() catches only RuntimeException | OutOfRangeException, so a TypeError or ArgumentCountError from your constructor escapes and takes out whichever screen triggered it.
The block's CSS and JS vanish after a drag or an add
IPS\nexus\extensions\core\Dashboard\PendingActions::getBlock() does this:
Output::i()->cssFiles = array_merge( Output::i()->cssFiles, Theme::i()->css( 'widgets.css', 'nexus', 'front' ) ); That works on the full page render only. On an AJAX request the ACP dispatcher wraps all of its CSS/JS assembly in if ( !Request::i()->isAjax() ), and do=getBlock assigns your HTML directly to Output::i()->output with nothing else attached. The JS then does .html( response ) into the widget body. So anything you push onto Output::i()->cssFiles or jsFiles inside getBlock() is discarded whenever the block is added, dragged, or refreshed.
The symptom is confusing because it is order-dependent: the block looks right until someone moves it, then loses its styling until the page is reloaded. Either inline the styles you need in the block markup, or register your CSS from your app's Application.php / an ACP output hook so it is present on every dashboard page load regardless of which path rendered the block. The JS controller does fire $( document ).trigger( 'contentChange', [ widget ] ) after injection, so already-loaded controllers will re-bind — but only if their file was loaded by the page, not by the block.
canView() is not an access control check on the AJAX route
Read the do=getBlock method again: it never calls canView(). It also never calls Dispatcher::i()->checkAcpPermission( 'view_dashboard' ) — that call lives only in manage(). The only gate on the AJAX route is the dispatcher's generic module restriction check (hasAcpRestriction( $this->application, $this->module ) for core/overview).
The practical consequence: an administrator who can reach the overview module but fails your canView() — say they lack transactions_manage, which is what nexus_Income::canView() requires — can still retrieve your block's rendered HTML by requesting index.php?app=core&module=overview&controller=dashboard&do=getBlock&appKey=myapp&blockKey=myapp_ServerLoad. Treat canView() as a visibility hint for the "Add" dropdown, not as authorization. Repeat the permission check at the top of getBlock() and return '' if it fails.
State-changing work inside getBlock() has no CSRF protection unless you add it
The ACP dispatcher auto-CSRF-checks any request carrying query-string keys beyond app/module/controller/id, but only when the controller has not opted out:
if ( !isset( $this->classname::$csrfProtected ) and array_diff( array_keys( Request::i()->url()->queryString ), array( 'app', 'module', 'controller', 'id' ) ) ) { Session::i()->csrfCheck(); } The dashboard controller declares public static bool $csrfProtected = TRUE;, which disables that automatic check for every method on it. Only update() calls Session::i()->csrfCheck() explicitly; getBlock() does not. This is why AdminNotes builds its form URL as Url::internal( "...&do=getBlock&appKey=core&blockKey=core_AdminNotes" )->csrf() — it is supplying its own token because the controller opted out of the framework's.
If your block posts back to do=getBlock and writes anything, append ->csrf() to the form action exactly as AdminNotes does.
Theme::i()->getTemplate('dashboard') silently resolves to core's templates, not yours
You get Call to undefined method IPS\Theme\class_core_admin_dashboard::serverLoad(), or — worse — core's dashboard group happens to have a method with the same name and you render the wrong markup. Theme::getTemplate() infers the app from the dispatcher:
if( $app === NULL ) { $app = Dispatcher::i()->application->directory; } if( $location === NULL ) { $location = Dispatcher::i()->controllerLocation; } On the dashboard the dispatched application is always core, no matter whose extension is running. Every first-party block outside core passes its app key for this reason (Theme::i()->getTemplate( 'dashboard', 'blog' ), ...( 'dashboard', 'nexus' )). Pass the location too — Theme::i()->getTemplate( 'dashboard', 'myapp', 'admin' ) — so the call still resolves correctly if you ever reuse the method outside an ACP request.
The new block does not appear in the "Add" dropdown at all
Three separate caches and one file have to line up.
Application::extensions() reads only applications/{app}/data/extensions.json. A file dropped into extensions/core/Dashboard/ without a matching JSON entry does not exist as far as core is concerned. The Developer Center writes this file via buildExtensionsJson(). Application::allExtensions() caches the entire resolved list in the datastore under Store::i()->extensions, keyed by extension type, and rebuilds only when the 'Dashboard' key is absent. Application declares protected array $caches = array( 'updatecount_applications', 'applications', 'extensions' ), so saving an Application record (install, upgrade, enable/disable) clears it — but hand-editing files on a live site does not. The Developer Center's create/remove extension actions unset( Store::i()->extensions ) explicitly. Application::extensions() also keeps a per-request static, static::$_loadedExtensions, so nothing you change mid-request is picked up. There is a fourth, nastier variant. The cache is written after apps have been filtered, and the filter includes recovery mode:
foreach ( $apps as $application ) { /* Skip third party apps if recovery mode is enabled */ if( RECOVERY_MODE and !in_array( $application->directory, IPS::$ipsApps ) ) { continue; } if ( !static::appIsEnabled( $application->directory ) ) { continue; } ... } /* Store for next time */ Store::i()->extensions = $allExtensions; Loading the dashboard once while RECOVERY_MODE is on bakes a first-party-only Dashboard list into the datastore, and that list survives after recovery mode is turned off. The symptom appears long after the cause: every third-party dashboard block quietly disappears, and nothing in the log connects it to a recovery-mode page view days earlier. Clearing the system cache (or any Application save) fixes it.
The block is installed and viewable but nobody ever sees it
Registering a Dashboard extension does not place it. The per-admin layout lives in the single acp_dashboard_blocks setting, a JSON blob keyed by member_id, and the defaults are hard-coded in dashboard::current():
$toShow = array( 'main' => array( 'core_Registrations', 'core_BackgroundQueue' ), 'side' => array( 'core_AdminNotes', 'core_OnlineUsers' ), 'collapsed' => array( 'core_BackgroundQueue' ), ); There is no extension hook into that list. Every administrator who already has a saved layout must add your block manually from the "Add" dropdown. Conversely, when your app is uninstalled the stale key stays in acp_dashboard_blocks forever; dashboard.phtml hides it with {{if array_key_exists( $cellKey, $blocks ) and isset($info[ $cellKey ])}}, so it fails silently rather than erroring — which also means an admin who removed your app and reinstalled it will find the block already positioned where they left it.
Verified against
Read from Invision Community 5.0.19 source (applications/core/data/versions.json top entry 5001908 => "5.0.19"). Files inspected: system/Extensions/DashboardAbstract.php; the consumer applications/core/modules/admin/overview/dashboard.php; Application::allExtensions(), Application::extensions() and Application::constructExtensionClass() in system/Application/Application.php; system/Dispatcher/Admin.php; system/Lang/Lang.php; system/Theme/Theme.php; the templates dashboard.phtml, dashboardHeader.phtml and widgetWrapper.phtml; the controller ips.dashboard.main.js; and all thirteen first-party implementations under applications/{core,blog,downloads,gallery,nexus}/extensions/core/Dashboard/. forums, cms and calendar ship no Dashboard extension.
Related: The core/OverviewStatistics extension in Invision Community 5, Widget conventions: two language keys, ipsWidget markup, and cached blanks, Extension method signatures are fixed, and a mismatch is a fatal on install

0 comments
41 views

Ernest Defoe

July 26Jul 26

The core/FrontNavigation extension in Invision Community 5

A core/FrontNavigation extension defines one type of item that an administrator can place in the front-end navigation bar via AdminCP → System → Menu Manager. The extension class is not the menu item itself: each row in the core_menu table names an app and an extension key, and core instantiates your class once per row, passing that row's stored configuration.
Core builds the menu in IPS\core\FrontNavigation::roots() and ::subBars() (applications/core/sources/FrontNavigation/FrontNavigation.php), and renders it from the global front templates navBar, navBarItems, navBarChildren, navColumn*, mobileFooterBar and mobileNavigationChildren. That means your methods run on every front-end page request. The only place Application::allExtensions( 'core', 'FrontNavigation', ... ) is called is applications/core/modules/admin/applications/menu.php line 328 — the AdminCP Menu Manager — where it is used purely to build the "Type" dropdown on the add/edit form.
The contract
Your class extends IPS\core\FrontNavigation\FrontNavigationAbstract (applications/core/sources/FrontNavigation/FrontNavigationAbstract.php). There is no system/Extensions/FrontNavigationAbstract.php — unlike most IC5 extension points, this one lives inside the core application.
/* Required — declared abstract */ abstract public function title() : string; // shown as the tab label abstract public function link() : Url|string|null; // NULL => rendered as href="#" abstract public static function typeTitle() : string; // label in the ACP "Type" dropdown abstract public function active() : bool; // is the current page this item? /* Optional — the abstract supplies these defaults */ public string $defaultIcon; // FA codepoint escape, e.g. '\f019' public static function allowMultiple() : bool // default FALSE public static function permissionsCanInherit() : bool // default TRUE public static function isEnabled() : bool // default TRUE public function canAccessContent() : bool // default TRUE public function canView() : bool // default: isEnabled() + permissions public function children( bool $noStore=FALSE ) : ?array // default NULL (no dropdown) public function attributes() : string // default '' — echoed RAW into <li> public function isAvailableFor( string $type ) : bool // header|sidebar|smallscreen public static function configuration( array $existingConfiguration, ?int $id=NULL ) : array public static function parseConfiguration( array $configuration, int $id ) : array /* Duck-typed, NOT declared on the abstract */ public function target() : string // templates use method_exists() /* Constructor signature core calls with */ public function __construct( array $configuration, // json_decode( core_menu.config, TRUE ) int $id, // core_menu.id string|null $permissions, // '*', 'group,ids', or NULL to inherit string $menuTypes, // '*' or a JSON array of header/sidebar/smallscreen array|null $icon, // json_decode( core_menu.icon, TRUE ) int|null $parent = 0 // core_menu.parent ) The core_menu table (applications/core/data/schema.json) is the storage: id, app, extension, config (TEXT, JSON), position, parent (nullable), permissions (nullable TEXT), is_menu_child, menu_types (NOT NULL, default *), icon (MEDIUMTEXT, nullable).
A minimal example
Three files. First, the extension class at applications/edmyapp/extensions/core/FrontNavigation/Directory.php:
<?php namespace IPS\edmyapp\extensions\core\FrontNavigation; use IPS\Application\Module; use IPS\core\FrontNavigation\FrontNavigationAbstract; use IPS\Dispatcher; use IPS\Http\Url; use IPS\Member; use OutOfRangeException; use function defined; if ( !defined( '\IPS\SUITE_UNIQUE_KEY' ) ) { header( ( $_SERVER['SERVER_PROTOCOL'] ?? 'HTTP/1.0' ) . ' 403 Forbidden' ); exit; } class Directory extends FrontNavigationAbstract { /* A Font Awesome *codepoint*, not a class name. See the icon trap below. */ public string $defaultIcon = '\f02d'; public static function typeTitle(): string { return Member::loggedIn()->language()->addToStack( 'frontnavigation_edmyapp_directory' ); } public function canAccessContent(): bool { try { /* Pass 'front' explicitly — Module::get() otherwise resolves the area from Dispatcher::i()->controllerLocation, which is 'admin' in the ACP. */ return Member::loggedIn()->canAccessModule( Module::get( 'edmyapp', 'directory', 'front' ) ); } catch ( OutOfRangeException ) { return FALSE; } } public function title(): string { return Member::loggedIn()->language()->addToStack( 'frontnavigation_edmyapp_directory' ); } public function link(): Url|string|null { return Url::internal( 'app=edmyapp&module=directory&controller=browse', 'front', 'edmyapp_directory' ); } public function active(): bool { return Dispatcher::i()->application?->directory === 'edmyapp' and Dispatcher::i()->module?->key === 'directory'; } } Second, register it in applications/edmyapp/data/extensions.json. The outer key is core (the app that owns the extension point), not your app:
{ "core": { "FrontNavigation": { "Directory": "IPS\\edmyapp\\extensions\\core\\FrontNavigation\\Directory" } } } Third, to have the item placed automatically on install, override defaultFrontNavigation() in applications/edmyapp/Application.php:
public function defaultFrontNavigation(): array { return array( 'rootTabs' => array(), 'browseTabs' => array( array( 'key' => 'Directory' ) ), 'browseTabsEnd' => array(), 'activityTabs' => array() ); } And add 'frontnavigation_edmyapp_directory' => "Directory" to applications/edmyapp/dev/lang.php.
My new navigation type does not appear in the Menu Manager's "Type" dropdown
Two separate causes, and they look identical.
First, the class file on disk is not enough. Application::getExtensionClass() (system/Application/Application.php line 548) does nothing but read applications/{app}/data/extensions.json and look up $content['core']['FrontNavigation'][$key]; if the key is absent it throws OutOfRangeException. Autoloading is irrelevant — the JSON file is the registry.
Second, Application::allExtensions() caches the entire map in the datastore under a single key: Store::i()->extensions, and the rebuild condition is if( !array_key_exists( $extension, $allExtensions ) ). Once the FrontNavigation key exists in that array it is never refreshed, so adding an entry to extensions.json on a site that has already rendered a menu changes nothing. Clear it — AdminCP → Support → "Clear caches", or in code unset( \IPS\Data\Store::i()->extensions );.
Note the asymmetry: getExtensionClass() reads the JSON file every call with no cache, while the ACP dropdown reads the datastore. So it is entirely possible for an existing menu item of your type to render correctly on the front end while your type is still missing from the ACP dropdown.
I added the item in the Menu Manager but the front end still shows the old menu
This is by design and catches everyone once. FrontNavigation::frontNavigation() caches the whole core_menu table in Store::i()->frontNavigation and only rebuilds when that key is unset. The Menu Manager's add, edit, delete and reorder actions all write to core_menu without clearing it. Only the publish() action does unset( Store::i()->frontNavigation ). That is what the yellow banner at the top of the Menu Manager is for — menu_manager_publish_desc, "Publish your menu once you've finished making changes".
Note that "Restore default menu" has the same behaviour: restore() calls buildDefaultFrontNavigation(), which deletes and rebuilds every row but never unsets the store, then calls frontNavigation(), which happily returns the stale cached copy.
If your own code inserts or changes a core_menu row, you must clear it yourself:
\IPS\Db::i()->insert( 'core_menu', $row ); unset( \IPS\Data\Store::i()->frontNavigation ); Every page on the site is a 500 error, including pages that have nothing to do with my extension
This is the single most important thing to know about this extension point. There is no try/catch anywhere in the navigation render path.
roots(), subBars(), FrontNavigationAbstract::subItems() and Menu::children() each wrap only the new $class( ... ) call, and each catches only OutOfRangeException. Once the object exists, the templates call canView(), isAvailableFor(), activeOrChildActive(), children(), getIconData(), getDefaultIcon(), title(), link() and attributes() completely unguarded — see applications/core/dev/html/front/global/navBarItems.phtml.
The navigation bar is part of the global wrapper, so an exception thrown from any of those methods takes down every page of the community, not the page your item links to. The visitor sees the generic error page; the real message goes only to AdminCP → System Logs under the uncaught_exception type (IPS\IPS::exceptionHandler, init.php line 817). Because the log entry names your class, developers usually chase the wrong page for an hour first.
The two calls that most often throw are:
Module::get( $app, $key ) throws OutOfRangeException when the module does not exist (system/Application/Module.php line 184). Worse, its third parameter defaults via $area = $area ?: Dispatcher::i()->controllerLocation, so Module::get( 'edmyapp', 'directory' ) looks for an admin module of that key whenever it is evaluated inside the AdminCP. Url::internal( $qs, 'front', $seoTemplate ) throws IPS\Http\Url\Exception with the message INVALID_SEO_TEMPLATE when the FURL key is not in a data/furl.json. Note system/Http/Url/Friendly.php line 225: outside IN_DEV this is swallowed and degraded to a plain index.php?... URL, but in IN_DEV it rethrows. A typo'd FURL key therefore produces a working production site and a completely dead development site. Wrap anything that can fail:
public function canAccessContent(): bool { try { return Member::loggedIn()->canAccessModule( Module::get( 'edmyapp', 'directory', 'front' ) ); } catch ( OutOfRangeException ) { return FALSE; } } Fatal "Call to a member function url() on null" on every page after an admin deletes some content
A related but distinct hazard: active() is not always guarded by canView().
FrontNavigationAbstract::activeOrChildActive() does check — if ( $this->isAvailableFor( $menuType ) and $this->active() ), and isAvailableFor() starts with if( !$this->canView() ) return false;. But IPS\core\extensions\core\FrontNavigation\Menu::active() does not:
public function active(): bool { foreach ( $this->children() as $child ) { if ( $child->active() ) // no canView() / canAccessContent() check { return TRUE; } } return FALSE; } So any item placed inside a dropdown has active() called on it regardless of whether it can be viewed. Core's own Tags extension demonstrates the consequence. Its title() and link() use the null-safe operator ($this->tag?->text, $this->tag?->url()) because the tag may have been deleted, but active() does not:
public function active(): bool { return stristr( (string) Request::i()->url(), (string) $this->tag->url() ); } Put a Tags item inside a dropdown menu, delete the tag, and every page fatals. Menu::permissionsCanInherit() returns FALSE, so dropdowns are saved with an explicit * permission and Menu::canView() returns TRUE, which is what makes the path reachable. Write active() so it is safe when the thing it points at no longer exists, exactly as you would title() and link().
My item vanished from the front end, and its row in the Menu Manager is blank
When Application::getExtensionClass() throws OutOfRangeException, roots(), subBars() and subItems() all catch it and simply skip the item. There is no log entry and no admin warning. It throws when the app is disabled, when RECOVERY_MODE is on and the app is not one of IPS::$ipsApps (so every third-party navigation item disappears in recovery mode), or when the extension key is missing from extensions.json.
In the AdminCP the same failure surfaces differently: menu.php::_getClassFromRow() returns null and _getMenuRow() then returns an empty string, so the tree renders an empty row with no title and no edit or delete buttons.
There is a live example of this in the shipped source. applications/nexus/Application.php's defaultFrontNavigation() includes array( 'key' => 'Referrals' ), but there is no Referrals entry in applications/nexus/data/extensions.json and no Referrals.php under applications/nexus/extensions/core/FrontNavigation/. The row is inserted into core_menu on install and is then permanently invisible and unremovable through the UI.
The tab is labelled "menu_item_57" instead of its name
The 'title' element of a defaultFrontNavigation() entry must be a language key, not a translated string. FrontNavigation::insertMenuItem() does:
Lang::copyCustom( $config['real_app'], $config['title'], "menu_item_{$insertedId}" ); and Lang::copyCustom() (system/Lang/Lang.php line 437) selects from core_sys_lang_words with word_app=? AND word_key=?. If no row matches, it writes nothing at all — no error, no fallback — and the item ends up with no menu_item_X string, so any title() that returns addToStack( "menu_item_{$this->id}" ) renders the raw key.
Core gets this wrong itself. applications/nexus/Application.php passes 'title' => Member::loggedIn()->language()->get( 'module__nexus_store' ) for its root Store dropdown — the resolved English text, so copyCustom looks for a word key literally named Store and finds nothing. The nested entry in the same array gets it right: 'title' => 'default_menu_item_my_details'.
Also note the key must already exist in core_sys_lang_words at the moment the menu item is inserted, i.e. your dev/lang.php strings must have been installed first.
Adding a completely different menu item type throws an error from my extension
The Menu Manager builds one form containing the configuration fields of every registered FrontNavigation extension, hidden and shown by JavaScript toggles keyed on the "Type" dropdown. menu.php line 350:
foreach ( $class::configuration( $existing ? json_decode( $existing['config'], TRUE ) : array(), $existing ? $existing['id'] : NULL ) as $field ) This runs inside a loop over all extensions, with no try/catch. Three consequences:
If your configuration() throws, the entire add/edit dialog dies — for every item type, not just yours. An admin trying to add a Custom Item sees an error caused by an unrelated third-party app. It runs on every render of that dialog, so keep it cheap. Core's own are not: cms\Pages::configuration() selects every row of cms_pages, and core\Node::configuration() walks every routed content class. $existingConfiguration is the configuration of whatever item is being edited, which is almost never one of yours. Edit a Tags item and your configuration() is handed array( 'id' => 7 ). Likewise $id is that other item's ID, so a Translatable field built with 'key' => "menu_item_{$id}" points at someone else's language string. Every core implementation therefore uses null-coalescing access and never assumes a key is present:
new Select( 'edmyapp_menu_mode', $existingConfiguration['edmyapp_menu_mode'] ?? NULL, NULL, ... ) Required fields are safe: Form::values() (system/Helpers/Form/Form.php line 691 onwards) computes $htmlIdsToIgnore from the toggle map and substitutes the default value for hidden fields rather than validating them. menu.php guarantees every field is in that map by assigning $field->htmlId = md5( mt_rand() ) to any field that does not already have one. Do not set an htmlId that collides with the four literals the form itself uses: menu_manager_access_type, menu_manager_access, menu_manager_menutype, menu_manager_icon.
Some of my configuration values are missing after saving
Only values whose keys were returned by your configuration() are kept. menu.php line 423:
if ( isset( $fieldNames[ $values['menu_manager_extension'] ] ) ) { foreach ( $values as $k => $v ) { if ( in_array( $k, $fieldNames[ $values['menu_manager_extension'] ] ) ) { $config[ $k ] = $v; } } } You cannot smuggle a value in through $form->hiddenValues or a field added elsewhere. Everything you want persisted must come from a field object returned by configuration(), or be added by parseConfiguration().
Because all extensions share one form, field names must be unique across the entire suite. Two extensions using the same field name collide on the same form element. This is why every core field name is prefixed by its owner: menu_custom_item_url, menu_custom_menu_title, menu_content_page, menu_stream_id, menu_node_content_classes.
Everything breaks if my app directory or extension key contains an underscore
The extension identity is the string "{appDirectory}_{ExtensionKey}", built in Application::allExtensions() as $appExtensions[ $application->directory . '_' . $key ]. The Menu Manager splits it back apart with a bare explode(), twice (menu.php lines 346 and 418):
$exploded = explode( '_', $key ); // $exploded[0] is treated as the app, $exploded[1] as the extension key There is no limit argument and no rejoin. An app directory of my_app yields app my; an extension file named Live_Topics.php yields extension key Live. The row is written to core_menu with the truncated values, then getExtensionClass() throws OutOfRangeException for it forever after — which, per the trap above, means the item silently never appears and leaves a blank row in the Menu Manager. Keep both the app directory and the extension class name free of underscores.
My default menu items come out in a random order
On a fresh install and on "Restore default menu", FrontNavigation::buildDefaultFrontNavigation() sets $position = 1, increments it twice for the built-in Browse and Activity tabs, and then calls:
static::insertMenuItem( $parent ?? null, $config, $position ); $position is passed by value and insertMenuItem() never returns or increments it, so every item contributed by every application is inserted with position = 4. frontNavigation() orders by position, so the relative order of default items is whatever MySQL returns for a tie.
The other code path behaves differently. When an app is installed after the community, Application::installOther() (line ~2299) re-queries MAX(position) for each item, so positions there do increment properly. Do not rely on ordering from defaultFrontNavigation(); the admin is expected to drag items into place.
My browseTabs items ended up as top-level tabs
Two different mechanisms consume defaultFrontNavigation() and they do not agree.
buildDefaultFrontNavigation() maps rootTabs to parent = NULL, browseTabs to parent = 1 and activityTabs to parent = 2, and defers browseTabsEnd to a second pass under parent 1. But Application::installOther() — the path taken when your app is installed onto a running community — iterates the same array and calls FrontNavigation::insertMenuItem( NULL, $config, ... ) unconditionally. The parent is always NULL there, because the Browse and Activity tabs are just ordinary rows with IDs 1 and 2 that the admin may already have deleted or renumbered. The docblock on Application::defaultFrontNavigation() states this outcome ("or in the top row if installing the app later").
A second, undocumented detail: the switch ( $type ) in buildDefaultFrontNavigation() has cases only for rootTabs, browseTabs and activityTabs, and $parent is declared outside the loop. Any other key in your returned array falls through the switch and silently inherits $parent from the previous iteration — or becomes a root tab via $parent ?? null if it is first. Return exactly the four documented keys.
The config I set in defaultFrontNavigation() is not the shape my extension reads
parseConfiguration() is called only from the AdminCP form handler. insertMenuItem() does 'config' => json_encode( $config['config'] ?? array() ) and stores it verbatim. So the array you supply under 'config' must already be in post-parse shape — the shape your constructor and link() expect, not the shape your form fields produce.
Compare core\Node: its form field is menu_node_content_classes, but parseConfiguration() rewrites that into array( 'nodeClass' => ..., 'selected' => ..., 'id' => ... ), and that is what the constructor reads. A defaultFrontNavigation() entry supplying menu_node_content_classes would produce a permanently broken item.
The 'icon' element has the same problem in reverse: insertMenuItem() writes 'icon' => $config['icon'] ?? null straight into the MEDIUMTEXT column with no encoding, so it must be a JSON string you encode yourself. applications/nexus/Application.php shows the required shape:
'icon' => json_encode( [ [ 'key' => 'cart-shopping:fas', 'type' => 'fa', 'raw' => '<i class="fa-solid fa-cart-shopping"></i>', 'title' => 'cart-shopping', 'html' => '...' ] ] ) The key must contain a colon. menu.php::_getMenuRow() does $bits = explode( ":", $iconData[0]['key'] ) and then switches on $bits[1] with no isset() guard, so a key without a colon produces an undefined-array-key notice in the Menu Manager tree.
My default icon renders as a blank square
$defaultIcon is not a Font Awesome class name. Look at how the templates use it:
<i class="fa-solid" style="--icon:'{$item->getDefaultIcon()|raw}'" aria-hidden="true"></i> It is injected as a CSS content value, so it must be a unicode escape, and the family class is hardcoded to fa-solid. Every core value follows this: '\f019' (downloads), '\f075' (forums), '\f87c' (gallery), '\f133' (calendar), '\e494' (followed content). The abstract's fallback is '\f1c5'. Write it in single-quoted PHP so the backslash survives. A regular or brands glyph will not render, because the class is not yours to set.
The admin-chosen icon is separate: getIconData() returns $this->icon[0]['raw'], the stored <i> markup from the icon picker, and the template prefers it when non-empty. Note that getIconDataForAttribute() exists on the abstract but is not called from anywhere in the shipped source.
My dropdown does not drop down
There are two child mechanisms and they are not the same thing.
children() is what the templates render as a dropdown. It returns NULL on the abstract, so by default your item is a plain link. Only core\Menu overrides it to read the core_menu rows beneath itself. subItems() is concrete on the abstract, always reads the core_menu rows beneath this item, and is used only by activeOrChildActive(). The templates test {{if $children = $item->children()}}, so returning an empty array is indistinguishable from returning NULL — the item renders as a leaf link. core\Menu exploits this deliberately: for a root-level menu it returns [] and lets the separate subBars() array drive the second-level bar instead.
If you return objects from children(), each one must implement at minimum canView(), isAvailableFor( string ), children() and title() — and active() too, since Menu::active() will call it. MenuHeader, MenuSeparator and MenuButton in IPS\core\extensions\core\FrontNavigation are helper objects for exactly this purpose. They do not extend FrontNavigationAbstract, they have different constructor signatures, and the templates identify them with instanceof. They appear in core's extensions.json but are excluded from the ACP dropdown by if ( method_exists( $class, 'typeTitle' ) ) at menu.php line 344 — that guard exists solely because those three classes do not implement it.
An admin can only add my item once
allowMultiple() defaults to FALSE, and the ACP counts existing rows across the whole menu:
if ( $class::allowMultiple() or !isset( $current[ $exploded[0] ][ $exploded[1] ] ) or ( $existing and $existing['app'] == $exploded[0] and $existing['extension'] == $exploded[1] ) ) So if defaultFrontNavigation() already placed one on install, your type is absent from the "Add" dropdown from day one and only reappears after the admin deletes it. Return TRUE from allowMultiple() for any item that carries configuration (a page, a node, a tag, a custom URL); the core types that do are CustomItem, Menu, Node, Tags, Pages and YourActivityStreamsItem.
Fatal "Attempt to read property 'key' on null" from active()
system/Dispatcher/Standard.php declares both properties as nullable:
public ?Application $application = NULL; public ?Module $module = NULL; public ?string $controller = NULL; The nexus extensions defend against this and the core ones do not. nexus\Store::active():
return Dispatcher::i()->application->directory === 'nexus' and Dispatcher::i()->module and Dispatcher::i()->module->key === 'store' and ... whereas core\Search::active() dereferences Dispatcher::i()->module->key directly. Since active() runs on every page including error and offline pages, use ?-> throughout:
return Dispatcher::i()->application?->directory === 'edmyapp' and Dispatcher::i()->module?->key === 'directory'; Related: if you override the constructor, take all six parameters. Core's Node, Tags and YourActivityStreamsItem declare only five and omit $parent. PHP silently discards extra arguments to userland functions, so this does not error — it just leaves $this->parent at 0 forever, which makes isSideBarItemCollapsed() return FALSE and changes the default expanded state in the sidebar layout.
Deleted menu items leave language strings behind
menu.php::_remove() deletes the row, recurses into children, and then calls exactly one cleanup:
Lang::deleteCustom( 'core', "menu_item_{$id}" ); Any other key your parseConfiguration() saved is orphaned in core_sys_lang_words. Core leaks two of its own this way: cms\Pages writes cms_menu_title_{$id} under app cms, and YourActivityStreamsItem writes menu_stream_title_{$id}. Neither is ever deleted.
There is a matching app mismatch on the install path: Lang::copyCustom( $config['real_app'], $config['title'], "menu_item_{$insertedId}" ) writes the row with word_app set to your application, but _remove() only ever deletes with word_app = 'core'. Lookups by addToStack() are by key alone so the item displays correctly; the row simply never goes away until the app is uninstalled. Prefer the standard menu_item_{$id} key under core if you want your strings cleaned up.
Two things worth knowing that are not quite traps
title() runs in the AdminCP too. menu.php::_getMenuRow() calls $menuItem->title() for every row in the Menu Manager tree, with no try/catch and with Dispatcher::i()->controllerLocation === 'admin'. A title() that loads a record which may have been deleted — cms\Pages::title() calls Page::load() unguarded — will 500 the Menu Manager rather than the front end.
canView() is evaluated twice per item per render. The templates test $item->canView() and $item->isAvailableFor( ... ), and isAvailableFor() itself begins with if( !$this->canView() ) return false;. If canAccessContent() issues a query, that query runs twice for every item on every page. Cache the result in a property.
Verified against
Read from the source of Invision Community 5.0.19 (highest entry in applications/core/data/versions.json is 5001908 => 5.0.19). Specifically: applications/core/sources/FrontNavigation/FrontNavigationAbstract.php, applications/core/sources/FrontNavigation/FrontNavigation.php, applications/core/modules/admin/applications/menu.php, all 35 core implementations under applications/*/extensions/core/FrontNavigation/, the front templates in applications/core/dev/html/front/global/, and the supporting code in system/Application/Application.php, system/Application/Module.php, system/Helpers/Form/Form.php, system/Http/Url/Friendly.php and system/Lang/Lang.php.
Related: The core/ContentRouter extension in Invision Community 5, Extension method signatures are fixed, and a mismatch is a fatal on install

0 comments
38 views

Ernest Defoe

July 26Jul 26

The core/GroupForm extension: adding fields to the member group form

A core/GroupForm extension adds a tab of fields to the member group form in the ACP (Members → Groups → edit), writes those values back onto the group when it is saved, and gets a say in what happens when a group is copied or deleted. It is the supported way for an application to attach its own per-group settings.
Core calls it from exactly four places: IPS\core\modules\admin\members\groups::form() (build and save), and three methods on IPS\Member\Group — __clone(), delete() and canDelete(). Every one of them fetches the extensions with the same call:
Application::allExtensions( 'core', 'GroupForm', FALSE, 'core', 'GroupSettings', TRUE ); The FALSE is $checkAccess — your extension runs regardless of whether the acting admin can access your application. The 'core', 'GroupSettings' arguments force core's own tab to sort first, so it is always the default active tab; everything else follows in application order. The array is keyed {appDirectory}_{extensionKey}, e.g. blog_Blog, and that key is what core turns into your tab's language string.
The contract
namespace IPS\Extensions; abstract class GroupFormAbstract { /* REQUIRED. Add your fields to $form. Core has already called $form->addTab( 'group__{app}_{key}' ) before this runs, so everything you add lands on your own tab automatically. $group may be an unsaved `new Group` with NO data at all. */ abstract public function process( Form $form, Group $group ) : void; /* REQUIRED. $values is the FLAT, WHOLE-FORM value array from $form->values() - every field from every app, keyed by field name. Mutate $group; do NOT call $group->save() (the caller does it for you immediately afterwards). */ abstract public function save( array $values, Group $group ) : void; /* OPTIONAL, default no-op. Called from Group::__clone() AFTER the new row has already been INSERTed. If you change $newGroup you must call $newGroup->save() yourself. */ public function cloneGroup( Group $oldGroup, Group $newGroup ) : void {} /* OPTIONAL, default TRUE. Group::canDelete() returns FALSE if ANY extension returns FALSE. Advisory only - see below. */ public function canDelete( Group $group ) : bool { return TRUE; } /* OPTIONAL, default no-op. Called from Group::delete() BEFORE the core_groups row is removed, so $group->g_id is still valid. */ public function delete( Group $group ) : void {} } No method declares a thrown exception, and nothing catches one on your behalf. The only try/catch anywhere near these calls is in the ACP delete controller, and it misattributes what it catches — see below.
A minimal example
For an app in applications/myapp. First, register it in applications/myapp/data/extensions.json:
{ "core": { "GroupForm": { "MySettings": "IPS\\myapp\\extensions\\core\\GroupForm\\MySettings" } } } Then applications/myapp/extensions/core/GroupForm/MySettings.php:
<?php namespace IPS\myapp\extensions\core\GroupForm; use IPS\Extensions\GroupFormAbstract; use IPS\Helpers\Form; use IPS\Helpers\Form\Number; use IPS\Helpers\Form\YesNo; use IPS\Member\Group; use IPS\Settings; use function defined; if ( !defined( '\IPS\SUITE_UNIQUE_KEY' ) ) { header( ( $_SERVER['SERVER_PROTOCOL'] ?? 'HTTP/1.0' ) . ' 403 Forbidden' ); exit; } class MySettings extends GroupFormAbstract { public function process( Form $form, Group $group ) : void { $form->addHeader( 'myapp_group_widgets' ); /* $group->g_id is NULL for a new group and every column reads back as NULL, so seed defaults explicitly. */ $form->add( new YesNo( 'myapp_can_widget', $group->g_id ? (bool) $group->myapp_can_widget : TRUE, FALSE, array( 'togglesOn' => array( 'myapp_widget_limit' ) ) ) ); /* Guests can never own widgets, so this field is conditional - save() must repeat the same condition. */ if ( $group->g_id != Settings::i()->guest_group ) { $form->add( new Number( 'myapp_widget_limit', $group->g_id ? $group->myapp_widget_limit : 10, FALSE, array( 'unlimited' => -1 ), NULL, NULL, NULL, 'myapp_widget_limit' /* html id, matches togglesOn */ ) ); } } public function save( array $values, Group $group ) : void { $group->myapp_can_widget = (int) $values['myapp_can_widget']; if ( array_key_exists( 'myapp_widget_limit', $values ) ) { $group->myapp_widget_limit = $values['myapp_can_widget'] ? (int) $values['myapp_widget_limit'] : 0; } } } Three language strings are required in applications/myapp/dev/lang.php — the tab, and one per field (the field name is the label key):
'group__myapp_MySettings' => "My App", 'myapp_group_widgets' => "Widgets", 'myapp_can_widget' => "Can create widgets?", 'myapp_widget_limit' => "Maximum widgets", And the columns have to exist. Add them in applications/myapp/setup/install/queries.json, which is also what core reads back at uninstall time to drop them again:
[ { "method": "addColumn", "params": [ "core_groups", { "name": "myapp_can_widget", "type": "TINYINT", "length": 1, "allow_null": true, "default": "0", "comment": "", "unsigned": true, "auto_increment": false } ] } ] Saving a group dies with "Unknown column 'myapp_can_widget' in 'field list'"
IPS\Member\Group is a plain ActiveRecord. Its __set() puts any property you assign into $this->changed, and save() hands changed straight to Db::i()->update( 'core_groups', ... ). There is no whitelist, so the first sign that your column does not exist is a MySQL error at save time, after the form has already validated.
This is worth calling out because the extension stub that the Developer Center generates (applications/core/data/defaults/extensions/GroupForm.txt) does exactly this and nothing else:
public function save( array $values, SystemGroup $group ) : void { $group->example = $values['example']; } while the ACP's own description of the extension (language key ext__GroupForm) says the opposite: "Recommended to store your group settings as an application setting, rather than add to the core_groups table." Pick one and be consistent. If you store settings outside core_groups, do not assign to $group at all; write to your own table in save(). If you add columns, add them via queries.json as above — note that every first-party group column (g_blog_*, idm_*, g_create_albums …) lives in core's data/schema.json, not in the owning app's, so you cannot copy that pattern.
Reading is safe in the opposite direction: ActiveRecord::__get() returns NULL for an unknown key rather than throwing, so a typo in a column name reads as NULL forever and silently.
"Undefined array key" in your own save(), and the setting quietly becomes 0
$values only contains fields that were actually added to the form. Any if in process() that skips a field must be repeated in save(), or you get a PHP warning and store NULL.
Core gets this wrong itself. IPS\core\extensions\core\GroupForm\Social::process() only adds gbw_view_helpful when Settings::i()->reputation_enabled, but save() pushes it into $bwKeys for every non-guest group and then does an unguarded read:
foreach ( $bwKeys as $k ) { $group->g_bitoptions[ $k ] = $values[ $k ]; /* no isset() */ } Everywhere else core guards. Follow the guarded pattern:
foreach ( array( 'myapp_widget_limit', 'myapp_widget_quota' ) as $k ) { if ( array_key_exists( $k, $values ) ) { $group->$k = $values[ $k ]; } } Use array_key_exists() rather than isset() — a legitimately NULL value is indistinguishable from an absent one under isset().
A field the admin edited comes back with its old value
If a field is hidden by a togglesOn/togglesOff/toggles relationship whose controlling field is off, Form::values() does not omit it and does not return the POSTed value. It returns the element's defaultValue — the second constructor argument, i.e. whatever you seeded the field with when you built the form:
if ( $element->htmlId and in_array( $element->htmlId, $htmlIdsToIgnore ) ) { $values[ $_name ] = $element->defaultValue; continue; } So a toggled-off field silently reverts to the current database value on an existing group, and to your hard-coded default on a new one. That is why core normalises by hand before writing, e.g. Content::save():
if ( !$values['g_attach'] ) { $values['gbw_delete_attachments'] = false; } Do the same: if your master switch is off, force the dependent values yourself instead of trusting what came back.
The tab is labelled "group__myapp_MySettings" in the ACP
The controller adds your tab before it calls you, using the extension key, not anything you control:
foreach ( $extensions as $k => $class ) { $form->addTab( 'group__' . $k ); $class->process( $form, $group ); } With $k = "myapp_MySettings" the required key is group__myapp_MySettings. When a language key is missing, Lang::replaceWords() falls through to $replacement = $values['key'] — it prints the key itself. So the symptom is a tab button reading group__myapp_MySettings, and nothing in the logs.
The key is derived from the file name of the extension. Renaming MySettings.php to Groups.php silently changes the required language key, and your old string becomes dead.
The good news is that addTab() alone does not create a tab. It only records a title and sets currentTab; Form::__toString() builds the rendered tab list from $this->elements, which stays untouched until something is actually added. So a process() that adds nothing produces no tab, and the missing string is never printed — which is why Commerce's PreventDeletion, a canDelete()-only extension with an empty process(), ships with no group__nexus_PreventDeletion string at all. (Be careful with the third argument to addTab(): passing a $blurbLang inserts an element, and that alone is enough to make the tab appear.)
The Developer Center does not know that, though. Its missing-strings scan (admin/developer/details.php) builds group__{app}_{key} from extensions.json for every GroupForm extension, whether or not that extension renders anything, so a canDelete()-only extension will always be reported as a missing string.
The tab never appears at all, and there is no error anywhere
Three separate silent-skip paths, all of which produce exactly the same nothing:
Application::extensions() skips any entry in extensions.json where !is_string( $classname ) or !class_exists( $classname ) — a namespace typo, a wrong file name, or a parse error in your class removes the tab with no message. Application::constructExtensionClass() catches RuntimeException and OutOfRangeException thrown from your constructor and returns null. Any other exception type propagates and takes the page down instead. The whole list is cached in the extensions datastore (Store::i()->extensions), keyed only by extension type. It is rebuilt when an Application record is saved or deleted (Application::$caches includes 'extensions') or from the Developer Center's Extensions screen. Hand-editing extensions.json on a live site changes nothing until that cache is cleared. Also note allExtensions() skips disabled applications entirely and, when RECOVERY_MODE is on, skips every non-IPS application. In recovery mode your delete() and cloneGroup() hooks do not run, so a group deleted in recovery mode leaves your rows orphaned.
Copying a group copies the wrong group's data (or nothing at all)
This is the worst one, because the signature lies. Group::__clone():
public function __clone() { $oldId = $this->g_id; $oldGroup = $this; /* NOT a copy - an alias */ parent::__clone(); /* sets g_id = NULL, then INSERTs, giving $this (and therefore $oldGroup) the NEW id */ ... foreach ( $extensions as $class ) { $class->cloneGroup( $oldGroup, $this ); } } PHP's clone invokes __clone() on the new object, so $this is already the copy. $oldGroup = $this takes an object handle, not a value copy. By the time your cloneGroup() runs, $oldGroup and $newGroup are the same object and both report the new group's g_id. The real source id is held in the local $oldId, which is never passed to you.
The practical consequence: you cannot write SELECT ... WHERE group_id = $oldGroup->g_id to copy rows from your own table — that reads the brand-new group, finds nothing, and the copy silently comes out empty. Core's own GroupSettings::cloneGroup() is unaffected only because it reads $oldGroup->g_icon, a core_groups column value that parent::__clone() already copied into the new row.
That is the argument for storing per-group data as core_groups columns: the row copy is automatic and cloneGroup() becomes unnecessary. If you store data in your own table keyed by group id, there is no reliable way to identify the source group from inside cloneGroup() in 5.0.19 — do not build on the first argument.
Changes made in cloneGroup() are not persisted
Group::__clone() never calls save() after the extension loop; the only save is the INSERT inside parent::__clone(), which happened before you were called. If you assign to $newGroup you must persist it yourself, exactly as core does:
public function cloneGroup( Group $oldGroup, Group $newGroup ) : void { $newGroup->myapp_licence_key = NULL; $newGroup->save(); /* required - nothing else will */ } Without the save() the property is set on an in-memory object that is then discarded, and the ACP redirects to the new group's edit form showing the un-changed value.
Deleting a group reports a 404 "We could not locate the item you are trying to view" (2C108/2)
This is the trap whose error surfaces nowhere near its cause. The ACP delete action wraps everything — including $group->canDelete() and $group->delete() — in one try block:
try { $group = Group::load( Request::i()->id ); if( !$group->canDelete() ) { ... } ... $group->delete(); } catch ( OutOfRangeException $e ) { Output::i()->error( 'node_error', '2C108/2', 404, '' ); } catch( InvalidArgumentException $e ) { Output::i()->error( 'cannot_delete_protected_group', '1C108/4', 403, '' ); } An OutOfRangeException escaping your delete() or canDelete() — and that is the exception every ActiveRecord::load() and Node::load() in IPS throws when a record is missing — is reported to the admin as "the group you are trying to view does not exist", error code 2C108/2, HTTP 404. An InvalidArgumentException is reported as "You cannot remove the preset groups". Neither message mentions your application, and neither is logged as a fault in your app.
Worse, Group::delete() runs the extension loop before parent::delete(), with no transaction:
foreach ( $extensions as $class ) { $class->delete( $this ); } parent::delete(); Core sorts itself first, so by the time your delete() throws, GroupSettings::delete() has already deleted the group's icon file — but the core_groups row survives. The admin sees a 404, the group is still listed, and its icon is gone. Catch everything inside your own delete():
public function delete( Group $group ) : void { try { Db::i()->delete( 'myapp_group_settings', array( 'group_id=?', $group->g_id ) ); } catch ( Exception $e ) { } } One bad app blanks the group form, or 500s the whole group list
Nothing in any of the four call sites is defensive. process() and save() are called in a bare foreach in groups::form(), so an exception in your process() propagates straight out of the controller and takes down the entire group add/edit screen for every application, not just your tab. There is no per-extension isolation to fall back on.
canDelete() is the more dangerous one, because of where it is called from. The group listing calls it once per row inside the rowButtons closure:
if ( Member::loggedIn()->hasAcpRestriction( 'core', 'members', 'groups_delete' ) AND Group::load( $row['g_id'] )->canDelete() ) An exception there breaks Members → Groups itself, so the admin cannot reach any group form to disable whatever is broken. It also means every query you run in canDelete() runs once per group per page load — Commerce's PreventDeletion::canDelete() issues two.
$group->my_column is NULL in process() when adding a new group
The controller builds the form from new Group when there is no id in the request. That object has an empty _data array, so every column reads back as NULL — not the schema default. This is why core writes $group->g_id ? $group->g_attach_max : 500000 throughout. Blog's save() spells out the other half of the problem:
/* We intval here because (some of) the columns do not accept null values, but these are null when creating a new group */ $group->g_blog_maxblogs = (int) $values['g_blog_maxblogs']; Note also the save order for a new group: the controller calls $group->save() once before the extension loop, purely so the group has an id for translatable fields, then calls each save(), then saves again. So $group->g_id is always populated inside save(), even on creation — but it was NULL in process() moments earlier.
Members in secondary groups get the wrong value for your setting
This is the "reading it back" half, and it is where most third-party group settings go wrong. $member->group is not a Group object. Member::get_group() returns a flat array built by merging the primary group's _data with g_bitoptions->asArray(), and then merging every secondary group over the top. The default merge rule for a key nobody has registered is "bigger number wins":
if ( !isset( $this->_group[ $k ] ) OR $v > $this->_group[ $k ] ) { $this->_group[ $k ] = $v; } For a boolean permission that is usually what you want (any group granting it wins). For anything else it is wrong:
Your semanticsDefault merge doesRegister 0 means unlimitedpicks the largest finite limit; 0 always loseszeroIsBest -1 means unlimited-1 always loses to any real numberneg1IsBest lower is more permissive (a wait period)picks the longest waitlessIsMore a CSV / JSON blobstring comparison, then an arbitrary winnercallback meaningless to merge (an id, an icon)merges anywayexclude The registration point is a second extension, core/GroupLimits, whose single getLimits() method returns those five keys. Downloads' is the model to copy:
public function getLimits(): array { return array( 'exclude' => array(), 'lessIsMore' => array( 'idm_wait_period' ), 'neg1IsBest' => array( 'idm_max_size' ), 'zeroIsBest' => array( 'idm_throttling' ), 'callback' => array( 'idm_restrictions' => function( $a, $b, $k ) { ... } ) ); } A callback receives ( $mergedSoFar, $thisSecondaryGroupData, $key, $memberData ); returning an array merges it wholesale into the group data, returning anything non-NULL sets that one key, and returning NULL leaves the value untouched.
Two further points about reading. First, use $member->group['myapp_x'] ?? $default — if your app is disabled or its column has been dropped, the key is simply absent and you get an undefined-key warning in front-end code. Second, group data is served from the groups datastore (Group::getStore()). Saving through the Group object clears it, because Group::$caches lists 'groups'; a raw Db::i()->update( 'core_groups', ... ) does not, and readers will keep seeing the old value until something else clears the cache.
Another app's setting overwrites yours (or yours overwrites theirs)
$values is one flat array for the entire form. Form::_insert() keys elements by $input->name, and Form::values() writes $values[ $element->name ], so two extensions that both add a field called enabled or max_items produce two <input> elements with the same name attribute on the same page. The browser posts one value, both extensions read it, and the field label (which is the same language key) will show whichever string won.
There is no namespacing. Prefix every field name with your app directory — myapp_can_widget, not can_widget — exactly as Downloads uses idm_. This applies to addHeader() keys and to the html ids you pass as the eighth constructor argument too.
Your tab is not hidden when the group cannot view the site
Core's GroupSettings::process() tries to hide every other extension's tab behind the g_view_board toggle:
foreach ( Application::allExtensions( 'core', 'GroupForm', FALSE ) as $key => $class ) { if ( $key != 'core_GroupSettings' ) { $tabs[] = $form->id . '_tab_group__' . $key; } } $form->add( new YesNo( 'g_view_board', ..., array( 'togglesOn' => array_merge( $tabs, ... ) ) ) ); Those ids do not exist in IC5's markup. The admin form template renders tab buttons as id='ipsTabs_{$formId}_{$tabKey}' and panels as ipsTabs_{$formId}_{$tabKey}_panel; the toggle JS resolves targets with [id='…'], so 4_tab_group__myapp_MySettings matches nothing. Server-side it matches nothing either, because Form::values() compares the toggle list against element htmlIds, and a tab is not an element.
The consequence is not cosmetic: your fields are always rendered, always submitted, and your save() always runs, including for a group with g_view_board = 0. If your setting is meaningless for a group that cannot see the site, enforce it in save() the way core does for its own fields:
if ( !$values['g_view_board'] ) { $values['myapp_can_widget'] = false; } canDelete() returning FALSE does not actually prevent deletion
Group::canDelete() is consulted in exactly two places: the ACP list, to decide whether to render a delete button, and the top of the ACP delete controller. Group::delete() itself never calls it — its only guard is a hard-coded check on the guest, member and admin group ids:
public function delete(): void { if ( in_array( $this->g_id, array( Settings::i()->guest_group, Settings::i()->member_group, Settings::i()->admin_group ) ) ) { throw new InvalidArgumentException; } ... } So canDelete() is a UI affordance, not a constraint. Any other code path that calls $group->delete() directly — another app, a task, a support-tool script — deletes the group regardless of what your extension said. If the data really must not be orphaned, handle it in delete() as well.
One cosmetic bug to be aware of while testing: when canDelete() returns FALSE and the admin reaches the delete URL directly, the controller calls Output::i()->error( 'cannot_delete_group', '2C108/5', 403, '' ), and cannot_delete_group is not defined anywhere in core's language files. The admin sees the literal string cannot_delete_group on the error page. That is core's missing string, not yours.
Verified against
Read from Invision Community 5.0.19 source: system/Extensions/GroupFormAbstract.php, system/Member/Group.php, system/Member/Member.php, system/Application/Application.php, system/Patterns/ActiveRecord.php, system/Helpers/Form/Form.php, applications/core/modules/admin/members/groups.php, applications/core/modules/admin/developer/details.php, applications/core/dev/html/admin/forms/template.phtml, dev/js/framework/common/ui/ips.ui.form.js, and all seven core implementations (core/Content, core/GroupSettings, core/Social, blog/Blog, downloads/Downloads, gallery/Gallery, nexus/PreventDeletion). Nothing here is inferred from Invision Community 4.
Related: The core/GroupLimits extension: merging secondary group settings in Invision Community 5, The core/Permissions extension and node permission integration in Invision Community 5, Extension method signatures are fixed, and a mismatch is a fatal on install

0 comments
38 views

Ernest Defoe

July 26Jul 26

The core/Permissions extension and node permission integration in Invision Community 5

The core/Permissions extension is how an application tells the suite two different things: which of its node classes own rows in core_permission_index and can be edited from the group permission matrix, and — separately — how the application wants to override normal permission checks at runtime. Core calls the first half exactly once, when the AdminCP renders Members → Groups → (a group) → Permissions; it calls the second half from IPS\Content\Permissions, which sits in front of nearly every can*() method on nodes, items, comments and reviews.
The two halves have almost nothing to do with each other, and each one has a different set of ways to fail.
The contract
Every method on IPS\Extensions\PermissionsAbstract is optional — the abstract implements all four with harmless defaults. Nothing is abstract, so an empty class that extends it is legal (that is exactly what the Developer Center skeleton in applications/core/data/defaults/extensions/Permissions.txt generates).
namespace IPS\myapp\extensions\core\Permissions; class Permissions extends \IPS\Extensions\PermissionsAbstract { /** * Node classes this app owns, mapped to a matrix-row callback (or NULL). * Callback signature: function( array $current, \IPS\Member\Group $group ) : array * $current = every core_permission_index row for this app + perm_type, * keyed by perm_type_id * returns = rows keyed by node id: * [ '_level' => int, 'label' => string, * "{$class::$permissionLangPrefix}perm__{$key}" => bool|null ] * (null renders a *disabled* checkbox) */ public function getNodeClasses() : array; /** * Runtime override for one permission check on one object. * MUST return exactly one of: * \IPS\Content\Permissions::PERM_DEFAULT (0) - do nothing, fall through * \IPS\Content\Permissions::PERM_ALLOW (1) - force allow * \IPS\Content\Permissions::PERM_DENY (2) - force deny * $object may be a node, item, comment or review. $member may be NULL. */ public function checkPermission( string $permission, object $object, ?\IPS\Member $member = null ) : int; /** Node IDs to force-allow, merged into the SQL permission clause. */ public function nodeIdsToAllow( string $permission, string $class, ?\IPS\Member $member = null ) : array; /** Node IDs to force-deny, merged into the SQL permission clause. */ public function nodeIdsToDeny( string $permission, string $class, ?\IPS\Member $member = null ) : array; } Register it in applications/myapp/data/extensions.json:
{ "core": { "Permissions": { "Permissions": "IPS\\myapp\\extensions\\core\\Permissions\\Permissions" } } } The consumers, in full, are:
CallerUsesNotes applications/core/modules/admin/members/groups.php::permissions()getNodeClasses()Called with allExtensions( 'core', 'Permissions', FALSE ) — every enabled app, no access check. IPS\Content\Permissions::can()checkPermission()First extension returning a non-zero value wins; the rest are never asked. IPS\Content\Permissions::loadMappings()nodeIdsToAllow(), nodeIdsToDeny()Results from all apps are array_merged into one include list and one exclude list. applications/core/modules/admin/developer/details.phpgetNodeClasses()Developer Center app scan; warns devscan__missing_permissions if a permission-bearing node class is not listed. applications/convert/sources/Library/Core.phpgetNodeClasses()Maps an imported perm_type string back to a node class. The include/exclude lists are combined asymmetrically in nodePermissionClause(). Includes are OR-ed into the group clause, so they grant access a member would not otherwise have. Excludes are appended as a separate AND ... NOT IN(...). If the same node id appears in both, the deny wins.
A minimal example
A third-party app myapp with a node class IPS\myapp\Category. Note that the node class itself needs $canBeExtended, $permApp, $permType, $permissionMap and $permissionLangPrefix — the extension is useless without them.
<?php namespace IPS\myapp\extensions\core\Permissions; use BadMethodCallException; use IPS\Content\Permissions as PermissionCheck; use IPS\Extensions\PermissionsAbstract; use IPS\Member; use IPS\myapp\Category; use function defined; if ( !defined( '\IPS\SUITE_UNIQUE_KEY' ) ) { header( ( $_SERVER['SERVER_PROTOCOL'] ?? 'HTTP/1.0' ) . ' 403 Forbidden' ); exit; } class Permissions extends PermissionsAbstract { /** * @brief Per-request memo. checkPermission() is NOT cached by core. */ protected static array $memo = []; public function getNodeClasses(): array { return array( 'IPS\myapp\Category' => function( $current, $group ) { $rows = array(); foreach( Category::roots( NULL ) as $root ) { try { Category::populatePermissionMatrix( $rows, $root, $group, $current ); } catch( BadMethodCallException $e ) {} } return $rows; } ); } public function checkPermission( string $permission, object $object, ?Member $member = null ): int { /* Bail immediately on anything that is not ours. */ if ( !( $object instanceof Category ) or $permission !== 'view' ) { return PermissionCheck::PERM_DEFAULT; } $member = $member ?: Member::loggedIn(); $key = $object->_id . '-' . (int) $member->member_id; if ( isset( static::$memo[ $key ] ) ) { return static::$memo[ $key ]; } return static::$memo[ $key ] = $object->members_only && !$member->member_id ? PermissionCheck::PERM_DENY : PermissionCheck::PERM_DEFAULT; } public function nodeIdsToDeny( string $permission, string $class, ?Member $member = null ): array { if ( $class !== Category::class or $permission !== 'view' ) { return array(); } $member = $member ?: Member::loggedIn(); if ( $member->member_id ) { return array(); } /* Cast: these ids go straight into SQL. */ return array_map( 'intval', array_values( Category::membersOnlyIds() ) ); } } And on the node class:
class Category extends \IPS\Node\Model implements \IPS\Node\Permissions { public static ?string $permApp = 'myapp'; public static ?string $permType = 'category'; public static string $permissionLangPrefix = 'perm_myapp_category_'; public static array $permissionMap = array( 'view' => 'view', 'read' => 2, 'add' => 3, ); /* Without this, checkPermission() is never consulted for this class. */ public static bool $canBeExtended = true; } You also need language strings perm_myapp_category_perm__view, ..._perm__read, ..._perm__add, perm_myapp_category_perm__label and __app_myapp (the group-permission tab title is '__app_' . $class::$permApp).
The node disappears from the front end after I call setPermissions()
IPS\Node\Model::setPermissions( array $insert ) does not populate app, perm_type or perm_type_id. It deletes the existing row using the class statics, then inserts your array verbatim:
Db::i()->delete( 'core_permission_index', array( 'app=? AND perm_type=? AND perm_type_id=?', static::$permApp, static::$permType, $this->_id ) ); $permId = Db::i()->insert( 'core_permission_index', $insert ); Pass only perm_view/perm_2/… and you get a row with app = '', perm_type = '', perm_type_id = 0 (the schema defaults). The insert succeeds. Nothing errors. But every front-end query joins on core_permission_index.app = ? AND perm_type = ? AND perm_type_id = <node id>, so the node now matches nothing and silently vanishes from roots(), children(), loadIntoMemory() and every content listing. The failure surfaces on the front end, on a completely different request, with no error at all — not at the point where you wrote the bad row.
Do it a second time for a second node and you finally get an error, from the perm_type unique index on (app, perm_type, perm_type_id): an IPS\Db\Exception with MySQL code 1062, Duplicate entry … for key 'perm_type'. The reported node is the second one; the node that is actually broken is the first one.
Copy what IPS\Node\Controller::permissions() and IPS\Node\Api\NodeController::_createOrUpdate() both do — build the three identity keys yourself, and carry perm_id over if a row already existed so the primary key is stable:
$insert = array( 'app' => $node::$permApp, 'perm_type' => $node::$permType, 'perm_type_id' => $node->_id, ); if ( isset( $current['perm_id'] ) ) { $insert['perm_id'] = $current['perm_id']; } foreach( $node::$permissionMap as $key => $column ) { $insert[ 'perm_' . $column ] = $groupsCsvOrStar; /* '*' or '1,2,4' */ } $node->setPermissions( $insert ); Two more things about that array. Every column in $permissionMap must be present: if $permissionMap['read'] exists and $insert omits it, setPermissions() throws an Undefined array key warning and writes NULL into core_tags_perms.tag_perm_text. And once a bad row has been written, Model::permissions() will "repair" it on the next request by inserting a fresh row with perm_view => '' — so the node reappears, visible to nobody, and looks like an admin mistake.
checkPermission() fires for my topics but never for my categories
IPS\Content\Item::can(), Comment::canView() and friends call Permissions::can() unconditionally. IPS\Node\Model::can() does not:
if( !in_array( $permission, [ 'edit', 'delete' ] ) ) { if( static::$canBeExtended and !( $member instanceof Group ) and $permCheck = PermissionsExtension::can( $permission, $this, $member ) ) { return ( $permCheck === PermissionsExtension::PERM_ALLOW ); } } IPS\Node\Model::$canBeExtended defaults to false, and its docblock says only "Determines if this class can be extended via UI Extension" — it does not mention permissions at all. Every core node class that participates (IPS\forums\Forum, IPS\downloads\Category, IPS\gallery\Category, IPS\calendar\Calendar, …) redeclares it as true. If yours does not, your override runs for content inside the node and never for the node itself. Add public static bool $canBeExtended = true;.
Note the second half of that guard too: node can( 'edit' ) and can( 'delete' ) deliberately skip extensions. Model::canDelete(), however, does call PermissionsExtension::can( 'delete', $this ) — with no member argument, so always against Member::loggedIn(). So 'delete' reaches your extension through one path and not the other.
Returning PERM_DENY works, but returning anything else denies everything too
Every call site follows the same shape:
if( $permCheck = Permissions::can( 'view', $this, $member ) ) { return ( $permCheck === Permissions::PERM_ALLOW ); } That is a truthiness test followed by a strict identity test. PERM_DEFAULT is 0, so falling through works. But any non-zero return that is not exactly 1 short-circuits the entire check and returns false. Return 2 and you deny (intended). Return 3, or a node id, or a count, and you also deny — permanently, with no error.
The abstract's own note is the rule: "You MUST return PERM_DEFAULT for any object that you are not checking." A common way to break this is a missing early return, so a helper that returns a bool leaks out. Because PermissionsAbstract does not declare strict_types, and neither will your extension file unless you add it, return true; is weak-coerced to 1 — a silent PERM_ALLOW — and return false; becomes PERM_DEFAULT. Neither raises anything.
The override works on the forum but not in the AdminCP, the installer, or a CLI task
Both entry points in IPS\Content\Permissions are gated on the dispatcher:
if( !Dispatcher::hasInstance() or ( Dispatcher::i()->controllerLocation != 'front' and Dispatcher::i()->controllerLocation != 'api' ) ) { return static::PERM_DEFAULT; /* can() */ /* getMapping() returns null, so no include/exclude clause is added at all */ } Only IPS\Dispatcher\Front ('front'), Api ('api'), and the two dispatchers that borrow 'front' (External, Build) qualify. Admin is 'admin', Setup is 'setup', and anything that runs with no dispatcher at all — CLI scripts, cron entry points, some background-queue contexts — fails hasInstance().
This is the one to remember, because the consequence lands somewhere else entirely. Anything computed in a non-front context and then persisted is computed as if your extension did not exist: search index permissions, deletion-log permissions, digest and notification recipient lists, and moderator tooling in the AdminCP. Your extension is not a security boundary for those; it only filters what the front end renders.
My deny disappears for guests, or for one particular group
Content\Permissions calls Application::allExtensions( 'core', 'Permissions' ) with the default third argument, $checkAccess = TRUE. Inside allExtensions() that becomes:
if( $checkAccess !== FALSE ) { if( !$application->canAccess( $checkAccess === TRUE ? NULL : $checkAccess ) ) { continue; } } canAccess( NULL ) checks Member::loggedIn() against the application's disabled_groups. So whether your extension is loaded at all depends on the logged-in member, while checkPermission() is handed a $member argument that may be somebody else. Two consequences:
If an admin restricts your app to certain groups in AdminCP → System → Applications, every member outside those groups gets no override — including guests, if guests are excluded. Your deny becomes an allow for exactly the audience you were probably trying to block. When core checks permissions for a member other than the visitor (permission previews, API calls authenticated as one member checking another, notification eligibility), the app filter still uses the visitor. By contrast groups.php passes FALSE explicitly, so getNodeClasses() is always collected from every enabled app. The two halves of the extension are loaded under different rules.
Another app's extension wins and mine is never asked
can() returns on the first non-default answer:
foreach( Application::allExtensions( 'core', 'Permissions' ) as $ext ) { $result = $ext->checkPermission( $permission, $object, $member ); if( $result !== static::PERM_DEFAULT ) { return $result; } } There is no priority system and no way to signal "deny, and mean it". Iteration order comes from Application::applications(), which reads Store::i()->applications, built as Db::i()->select( '*', 'core_applications', NULL, 'app_position' ). app_position is the drag-to-reorder order in AdminCP → System → Applications. An administrator reordering applications for cosmetic reasons can flip which app's permission override wins, on a live site, with no code change and no log entry.
Nothing wraps that loop in try/catch either. An exception thrown from checkPermission() escapes into the middle of whatever was rendering — a topic list, a search result row — and takes the page with it.
SQL syntax error from Node::roots() after adding nodeIdsToAllow()
The two ID lists are not handled the same way. Excludes go through Db::i()->in(), which drops non-scalars and escapes strings. Includes are interpolated raw:
/* nodeIdsToAllow - no escaping, no binds */ $clause[] = 'core_permission_index.perm_type_id in (' . implode( ",", $includes ) . ')'; /* nodeIdsToDeny - escaped */ $return[] = [ Db::i()->in( 'core_permission_index.perm_type_id', $excludes, true ) ]; Return anything that is not a bare integer from nodeIdsToAllow() and you get a broken query. Return an associative array such as [ 4 => 'Announcements' ] and implode() uses the values, producing in (Announcements). Return arrays and you get Array to string conversion plus in (Array). Return user-supplied text and you have an injection point.
The exception is raised by IPS\Node\Model::nodesWithPermission() or Content\Item::getItemsWithPermission(), naming a table and a query your app does not own. Nothing in the error mentions your extension. Always finish with array_map( 'intval', array_values( $ids ) ).
Both lists are also merged across apps with array_merge, so an empty array is the correct "no opinion" answer — getMapping() checks count() and adds no clause when the merged list is empty.
The wrong member's allow/deny list is applied halfway through a request
loadMappings() memoises per request — but the member is not part of the cache key:
protected static function loadMappings( string $permission, string $nodeClass, Member $member ) : void { if( !isset( static::$nodeIncludesMap[ $nodeClass ][ $permission ] ) ) { ... } } $member is passed to your methods but never consulted when deciding whether to rebuild. The first member checked in a request wins for every subsequent check of the same node class and permission key. This shows up wherever one request evaluates node visibility for more than one member: notification and follower fan-out, "who can see this" previews, API requests that resolve one member's content while authenticated as another, and any place that calls Node::roots( 'view', $someOtherMember ) after the page has already rendered a listing for the visitor.
You cannot fix this from the extension — the cache lives in core and is keyed above your code. What you can do is make nodeIdsToAllow()/nodeIdsToDeny() member-independent (rules that depend on the node, not the viewer) and put viewer-specific logic in checkPermission(), which is not cached.
checkPermission() runs hundreds of times on a single page
Node\Model::can() maintains $this->cachedPermissionChecks, and Content\Item has its own caching, but in both cases the extension call happens before the cache lookup — the comment in the source is literally "Extensions go first". A cached permission check still invokes every Permissions extension in the suite. Content\Permissions::can() adds no memoisation of its own.
On a forum index that is one call per forum per permission key per render pass; on a topic list it is one per topic, plus one per comment on the topic view. A single Db::i()->select() inside checkPermission() becomes hundreds of queries. Return PERM_DEFAULT from an instanceof check as the very first statement, and memoise anything expensive in a static property on the extension (extension objects are constructed fresh by constructExtensionClass(), so instance properties do not survive).
Search results and activity streams still show content my extension denies
Nothing under system/Content/Search/ references IPS\Content\Permissions or nodePermissionClause(). The MySQL search query filters on one column:
$where[] = array( "( index_permissions = '*' OR " . Db::i()->findInSet( 'index_permissions', $this->permissionArray() ) . ' )' ); index_permissions is written by Model::updateSearchIndexPermissions() from searchIndexPermissions(), which reads the raw core_permission_index columns and intersects them up the parent chain. Your checkPermission() and nodeIdsToDeny() are never involved. The same applies to DeletionLog::updateNodePermissions() and deleteLogPermissions().
If a node must be invisible in search as well as in listings, the denial has to be expressed as real group permissions in core_permission_index, not as an extension override.
My node type is missing from the group permissions screen
The group permission matrix is built only from getNodeClasses(). Implementing IPS\Node\Permissions and setting $permApp/$permType gets you a per-node permissions tab in the node's own ACP screen, but nothing on the group screen. This is not hypothetical: blog, cms and nexus all ship node classes implementing IPS\Node\Permissions and none of them registers a core/Permissions extension, so their categories are simply absent from Members → Groups → Permissions in a stock install.
Two further ways to be skipped even after registering:
groups.php does if( !is_callable( $callback ) ) { continue; }. A NULL callback registers the class for the Developer Center scan and the converter's perm_type lookup, but adds no tab. Core uses this deliberately for 'IPS\Application\Module' => NULL. The extension list is cached in Store::i()->extensions and only rebuilt when that key is missing. Hand-editing extensions.json without going through the Developer Center leaves the stale list in place; applications/core/modules/admin/developer/extensions.php is the only core code that calls unset( Store::i()->extensions ) on extension changes. The permissions tab for my app appears but has no rows
Your callback is only invoked if permission rows for your app already exist:
if ( isset( $current[ $class::$permApp ][ $class::$permType ] ) ) { $matrix->rows = $callback( $current[ $class::$permApp ][ $class::$permType ], $group ); } /* Add the matrix - runs unconditionally */ $form->addMatrix( $class, $matrix ); $current is built by selecting the entire core_permission_index table and indexing it [app][perm_type][perm_type_id]. On a fresh install where categories exist but no permission row has ever been written, that key is absent, the callback never runs, and the tab renders with headers and no rows. The tab is still added, because addTab() and addMatrix() are outside the isset().
Model::permissions() self-heals a missing row (it catches UnderflowException and inserts perm_view => ''), so merely loading the node once on the front end can make the tab populate later — which makes this look intermittent. Write real rows at install/upgrade time instead of relying on that.
Related: two node classes sharing a $permApp do not get two tabs. Form::addTab() keys on the language string, so '__app_' . $permApp collides and both matrices stack inside one tab.
A white screen on Members → Groups → Permissions after installing my app
This screen loops every enabled application's extension with no error handling around getNodeClasses(), and immediately dereferences the array keys as class names:
foreach ( Application::allExtensions( 'core', 'Permissions', FALSE ) as $ext ) { foreach ( $ext->getNodeClasses() as $class => $callback ) { ... $form->addTab( '__app_' . $class::$permApp ); $matrix = new Matrix( $class ); foreach ( $class::$permissionMap as $k => $v ) { ... } } } A typo in a key, or a class that is not loadable because a file is missing after a partial upload, is a fatal Error: Class "IPS\myapp\Categry" not found raised inside applications/core/modules/admin/members/groups.php. The AdminCP error names a core file and a core controller; nothing in it identifies your application. The same is true of the Developer Center app scan and the converter, both of which call getNodeClasses() and use the keys as class names with no class_exists() guard.
Exceptions from the row callback are barely better. Only one type is handled:
try { $matrix->rows = $callback( $current[ $class::$permApp ][ $class::$permType ], $group ); } catch( UnderflowException $e ) { if( $e->getCode() != 199 ) { throw $e; } Output::i()->error( 'generic_error', '4T382/1', 500, $e->getMessage() ); } Code 199 is a specific signal — IPS\cms\Categories::disabledPermissions() throws new UnderflowException( 'invalid_guestgroup_admin', 199 ) when the configured guest group no longer exists — and it produces error 4T382/1 (the equivalent on the per-node screen is 4F383/1). An UnderflowException with any other code is rethrown; every other exception type is not caught at all.
Note in particular that Model::populatePermissionMatrix() opens with throw new BadMethodCallException if the node does not implement IPS\Node\Permissions, and it recurses into $node->children( NULL ). Of the four core extensions that use it, only downloads wraps the call:
try { Category::populatePermissionMatrix( $rows, $root, $group, $current ); } catch( BadMethodCallException $e ) {} forums, calendar and gallery do not. Copy the downloads version — if a subclass or child node in your tree ever stops implementing the interface, the unguarded version takes down a shared core screen for every application at once.
disabledPermissions() returns the right keys but the checkboxes are still enabled
The docblock on IPS\Node\Model::disabledPermissions() says:
@return array array( {group_id} => array( 'read', 'view', 'perm_7' ); That example is wrong in two ways. populatePermissionMatrix() compares against the values of $permissionMap, not its keys, and not a perm_-prefixed string:
foreach( $node->permissionTypes() AS $k => $v ) { ... if ( array_key_exists( $group, $disabledPermissions ) and is_array( $disabledPermissions[ $group ] ) ) { $disabled = in_array( $v, array_values( $disabledPermissions[ $group ] ) ); } } $v is the column suffix — 'view', 2, 3, … So 'read' never matches (the key is 'read', the value is 2), and 'perm_7' never matches (the value is 7). Only 'view' happens to work, because for that one entry key and value are the same string. The real core implementation, IPS\forums\Forum::disabledPermissions(), returns array( Settings::i()->guest_group => array( 2, 3, 4, 5 ) ) — raw column suffixes. Note also that in_array() here is loose, so the integer 2 and the string '2' both match, but 'read' does not.
There is no error when you get this wrong; the checkbox is simply left enabled, and an admin can grant a permission your node class expects to be impossible. The same array is also consulted by Node\Controller::permissions() for the per-node screen, so the two screens disagree in exactly the same way.
Unknown column 'perm_8'
core_permission_index has exactly seven permission columns: perm_view (TEXT NOT NULL DEFAULT '') and perm_2 through perm_7 (TEXT NULL). Every consumer builds its column name by string concatenation — 'perm_' . static::$permissionMap[ $permission ] — with no validation, in can(), canOnAny(), nodePermissionClause() and setPermissions().
An eighth entry in $permissionMap therefore produces Unknown column 'core_permission_index.perm_8' in 'where clause' from whichever listing query ran first. Seven permission keys per node class is a hard ceiling; if you need more, model the extra state as columns on your own node table and enforce it in checkPermission().
The reverse mistake is quieter: a $permissionMap key that no row ever populates leaves perm_N as NULL, and Model::can() treats a falsy permission string as "denied to everyone" without complaint.
Verified against
Read from Invision Community 5.0.19 source: system/Extensions/PermissionsAbstract.php, system/Content/Permissions.php, system/Node/Model.php, system/Node/Permissions.php, system/Node/Controller.php, system/Node/Api/NodeController.php, system/Application/Application.php, system/Db/Db.php, system/Content/Search/Mysql/Query.php, applications/core/modules/admin/members/groups.php, applications/core/modules/admin/developer/details.php, applications/convert/sources/Library/Core.php, applications/core/data/schema.json, and all five core implementations under applications/{core,forums,calendar,downloads,gallery}/extensions/core/Permissions/. Behaviour described here is what those files do, not what the docblocks say; where the two disagree, the disagreement is called out above.
Related: The core/ModeratorPermissions extension in Invision Community 5, The core/GroupForm extension: adding fields to the member group form, Extension method signatures are fixed, and a mismatch is a fatal on install

0 comments
31 views

Ernest Defoe

July 26Jul 26

The core/Sitemap extension in Invision Community 5

A core/Sitemap extension contributes URLs to the XML sitemap that sitemap.php serves, and adds the per-app fields to ACP → Promotion → Search Engine Optimisation → Sitemap. Core calls it from three places: the hourly core/sitemapgenerator task (via IPS\Sitemap::buildNextSitemap()), the core/RebuildSitemap background queue task fired by the ACP "Rebuild Sitemap" button, and the ACP SEO controller itself when it builds and saves the settings form.
The contract
All four methods in IPS\Extensions\SitemapAbstract are abstract. There are no optional methods and the abstract declares no @throws at all — so there is no exception core is documented to expect from you, and (as the traps below show) almost none that it catches.
namespace IPS\Extensions; abstract class SitemapAbstract { /* Merged into the ACP "recommended settings" pool and compared field-by-field */ public array $recommendedSettings = array(); /* Form fields for the ACP Sitemap tab. Keys MUST match keys in $recommendedSettings. */ abstract public function settings(): array; /* Called once per extension with the WHOLE form's values, including every other app's fields and the always-present 'sitemap_configuration_info' YesNo. */ abstract public function saveSettings( array $values ) : void; /* Every sitemap file this extension owns, as a flat array of strings. These become the primary key values in core_sitemap.sitemap (VARCHAR 255). */ abstract public function getFilenames(): array; /* Build one file. $filename is one of the strings getFilenames() returned. The return value is DISCARDED by both callers - see the trap below. */ abstract public function generateSitemap( string $filename, Sitemap $sitemap ) : ?int; } The only way to emit anything is IPS\Sitemap::buildSitemapFile(), which writes the XML into the core_sitemap table:
public function buildSitemapFile( string $filename, array $entries, int $lastId = 0, array $namespaces = array() /* e.g. array( 'image' => 'https://www.google.com/schemas/sitemap-image/1.1' ) */ ) : void Each entry in $entries is an array. $entry['url'] is read with no isset() guard — it is mandatory. lastmod, priority and changefreq are optional and are each only written when the value is truthy (isset( $entry[...] ) AND $entry[...]). lastmod may be an IPS\DateTime or a raw unix timestamp; anything else is fed to DateTime::ts() and will produce a nonsense date rather than an error. core_sitemap has four columns — sitemap (VARCHAR 255, PRIMARY KEY on the first 191 characters), data (MEDIUMTEXT, nullable), updated (INT), last_id (BIGINT UNSIGNED).
Batching is entirely your responsibility. Core has one constant, \IPS\SITEMAP_MAX_PER_FILE (default 500, defined in init.php), and one storage slot, core_sitemap.last_id. Every core implementation follows the same unwritten convention: name your files <prefix>_1, <prefix>_2, … , explode('_')/array_pop() the block number back off $filename in generateSitemap(), and read the previous block's last_id out of core_sitemap to use as a cursor. Nothing in core enforces this; nothing in core does it for you.
A minimal example
First, register the extension. In IC5 an extension file that is not listed in data/extensions.json is invisible — IPS\Application::extensions() reads only that JSON file and never scans the directory.
/* applications/myapp/data/extensions.json */ { "core": { "Sitemap": { "Listings": "IPS\\myapp\\extensions\\core\\Sitemap\\Listings" } } } /* applications/myapp/data/settings.json */ [ { "key": "sitemap_myapp_listings", "default": "", "report": "none" } ] /* applications/myapp/dev/lang.php - the __admin__ pack */ 'sitemap_myapp_Listings' => "Listings", /* {appdir}_{extension key} */ 'sitemap_myapp_listings_include' => "Include in sitemap", 'sitemap_myapp_listings_priority' => "Priority", <?php /* applications/myapp/extensions/core/Sitemap/Listings.php */ namespace IPS\myapp\extensions\core\Sitemap; use Exception; use IPS\Db; use IPS\Extensions\SitemapAbstract; use IPS\Helpers\Form\Select; use IPS\Helpers\Form\YesNo; use IPS\Member; use IPS\myapp\Listing; use IPS\Settings; use IPS\Sitemap; use UnderflowException; use function defined; use const IPS\SITEMAP_MAX_PER_FILE; if ( !defined( '\IPS\SUITE_UNIQUE_KEY' ) ) { header( ( $_SERVER['SERVER_PROTOCOL'] ?? 'HTTP/1.0' ) . ' 403 Forbidden' ); exit; } class Listings extends SitemapAbstract { /* Declared as a literal. Do NOT populate this inside settings() - see the trap below. */ public array $recommendedSettings = array( 'sitemap_myapp_listings_include' => true, 'sitemap_myapp_listings_priority' => '0.8' ); /* getFilenames() is called many times per task run - memoise it. */ protected ?array $filenames = null; /* Core constructs extensions with new $class( $member ) where $member is IPS\Member (guest) from the task, or NULL from the ACP. Make it optional. */ public function __construct( ?Member $member = null ) {} protected function config() : array { return Settings::i()->sitemap_myapp_listings ? json_decode( Settings::i()->sitemap_myapp_listings, true ) : array(); } public function settings(): array { $settings = $this->config(); $return = array(); /* Every key here must also exist in $recommendedSettings, and the 8th constructor argument (htmlId) is required for the toggles to work. */ $return['sitemap_myapp_listings_include'] = new YesNo( 'sitemap_myapp_listings_include', $settings['sitemap_myapp_listings_include'] ?? $this->recommendedSettings['sitemap_myapp_listings_include'], false, array( 'togglesOn' => array( 'sitemap_myapp_listings_priority' ) ), null, null, null, 'sitemap_myapp_listings_include' ); /* Deliberately NOT using 'unlimited' => '-1' here - see the priority trap. */ $return['sitemap_myapp_listings_priority'] = new Select( 'sitemap_myapp_listings_priority', $settings['sitemap_myapp_listings_priority'] ?? $this->recommendedSettings['sitemap_myapp_listings_priority'], false, array( 'options' => Sitemap::$priorities ), null, null, null, 'sitemap_myapp_listings_priority' ); return $return; } public function saveSettings( array $values ) : void { /* $values is the ENTIRE form - sitemap_url, every other app's fields, and sitemap_configuration_info, which is always present. Never throw here. */ if ( $values['sitemap_configuration_info'] ) { $save = $this->recommendedSettings; } else { $save = array( 'sitemap_myapp_listings_include' => (bool) $values['sitemap_myapp_listings_include'], 'sitemap_myapp_listings_priority' => (string) $values['sitemap_myapp_listings_priority'] ); } Settings::i()->changeValues( array( 'sitemap_myapp_listings' => json_encode( $save ) ) ); } public function getFilenames(): array { if ( $this->filenames !== null ) { return $this->filenames; } $settings = $this->config(); if ( isset( $settings['sitemap_myapp_listings_include'] ) and !$settings['sitemap_myapp_listings_include'] ) { return $this->filenames = array(); } try { $count = (int) Db::i()->select( 'COUNT(*)', 'myapp_listings', array( 'listing_open=?', 1 ) )->first(); } catch ( Exception $e ) { /* Never throw, and never return array() on a transient failure - core deletes every core_sitemap row that is not in the returned list. Fall back to whatever we have already built. */ return $this->filenames = iterator_to_array( Db::i()->select( 'sitemap', 'core_sitemap', array( "sitemap LIKE 'sitemap_myapp_listings_%'" ) ) ); } $files = array(); for ( $i = 1; $i <= ceil( $count / SITEMAP_MAX_PER_FILE ); $i++ ) { $files[] = 'sitemap_myapp_listings_' . $i; } return $this->filenames = $files; } public function generateSitemap( string $filename, Sitemap $sitemap ) : ?int { $settings = $this->config(); $priority = (string) ( $settings['sitemap_myapp_listings_priority'] ?? $this->recommendedSettings['sitemap_myapp_listings_priority'] ); $exploded = explode( '_', $filename ); $block = (int) array_pop( $exploded ); $entries = array(); $lastId = 0; $where = array( array( 'listing_open=?', 1 ) ); $limitClause = array( ( $block - 1 ) * SITEMAP_MAX_PER_FILE, SITEMAP_MAX_PER_FILE ); /* Cursor: the previous block stored its highest ID in core_sitemap.last_id */ try { $previous = (int) Db::i()->select( 'last_id', 'core_sitemap', array( 'sitemap=?', implode( '_', $exploded ) . '_' . ( $block - 1 ) ) )->first(); if ( $previous > 0 ) { $where[] = array( 'listing_id > ?', $previous ); $limitClause = SITEMAP_MAX_PER_FILE; } } catch ( UnderflowException $e ) {} $guest = new Member; foreach ( Db::i()->select( '*', 'myapp_listings', $where, 'listing_id ASC', $limitClause ) as $row ) { $lastId = (int) $row['listing_id']; /* advance even when skipping */ $item = Listing::constructFromData( $row ); if ( !$item->canView( $guest ) ) { continue; } $entries[] = array( 'url' => (string) $item->url(), 'lastmod' => (int) $row['listing_updated'], 'priority' => $priority /* string, never intval()'d */ ); } /* ALWAYS call this, even with zero entries, or the file is re-queued forever. */ $sitemap->buildSitemapFile( $filename, $entries, $lastId ); return $lastId; } } My extension does not appear on the ACP sitemap page at all
No error, no log entry, no header on the Sitemap tab. Two independent causes, both silent.
First, Application::extensions() in IC5 reads only applications/<app>/data/extensions.json. If your class is not listed under {"core": {"Sitemap": {...}}}, or if class_exists() on the listed name returns false, the loop does continue with no message — the comment in core literally reads "Switching between branches confuses extensions".
Second, Application::allExtensions() caches the resolved class list in Store::i()->extensions and only rebuilds it when the Sitemap key is missing from that store. Adding a new extension to a live site does nothing until the datastore is cleared (ACP → System → Support → Clear cache, or unset( Store::i()->extensions )).
Also note that allExtensions() skips every non-IPS application when \IPS\RECOVERY_MODE is on. If a site is in recovery mode, no third-party URLs are contributed and the sitemap silently shrinks.
My extension is listed but silently disappears from the generated sitemap
Extensions are instantiated by Application::constructExtensionClass():
$obj = new $classToUse( $checkAccess === TRUE ? Member::loggedIn() : ( $checkAccess === FALSE ? NULL : $checkAccess ) ); /* ... */ catch( RuntimeException | OutOfRangeException $e ){} return null; Two consequences. Your constructor is always called with one argument, and it is not the same argument in both call paths: IPS\Sitemap::buildNextSitemap() and RebuildSitemap pass allExtensions( 'core', 'Sitemap', new Member, 'core' ), so you get a guest IPS\Member; the ACP SEO controller passes FALSE, so you get NULL. A constructor declared __construct( Member $member ) works on the task and throws a TypeError in the ACP. Declare it __construct( ?Member $member = null ), or do not declare one at all — SitemapAbstract has no constructor, and PHP tolerates extra arguments to user-defined functions.
And if your constructor throws RuntimeException or OutOfRangeException, the object is dropped and null is returned with no log line anywhere. Your extension simply stops existing. This is the failure mode where nothing at all is written to core_sitemap and there is no evidence to search for.
The ACP shows a header reading "sitemap_myapp_Listings"
And the rebuild progress bar reads "Rebuilding sitemap_myapp_Listings sitemap". The extension key core builds is {application directory}_{extension key from extensions.json}. In applications/core/modules/admin/promotion/seo.php the header is added as $form->addHeader( 'sitemap_' . $extKey ), and in RebuildSitemap::getProgress() the same key is used: $key = 'sitemap_' . $data['extensionKey']. Core ships sitemap_core_Clubs, sitemap_cms_Pages, sitemap_cms_Databases, sitemap_gallery_Images, sitemap_gallery_Videos and sitemap_nexus_Subscriptions in its language packs for exactly this reason. addToStack() on a missing key returns the key itself, so you get the raw string rather than an error.
Define sitemap_{appdir}_{ExtensionKey} in your admin language pack. The header is only rendered when settings() returned at least one field, but the progress-bar key is used regardless.
"Undefined array key" warning on ACP → Promotion → Search Engine Optimisation
The Sitemap tab errors and you cannot reach the settings at all. The form builder does this, with no isset():
foreach ( $extension->settings() as $k => $setting ) { if ( $setting->value != $extension->recommendedSettings[ $k ] ) { $useRecommendedSettings = FALSE; } /* ... */ } Every key returned by settings() must also exist in $recommendedSettings. Returning a numerically-indexed array of form fields — which is the natural thing to do, since the array is only ever iterated for its values afterwards — will produce Undefined array key 0. Every core implementation returns a map keyed identically to its recommendedSettings; gallery's Images and Videos sidestep the issue by returning an empty array from both.
Note where this surfaces: it is a shared screen. A bad key in your app breaks the SEO settings page for every app, and the stack trace names seo.php, not your extension.
"Use recommended settings" saves the wrong values for my app
The admin flips sitemap_configuration_info to Yes, saves, and your extension's recommended defaults are not applied. The cause is statement order in _manageSitemap():
$recommendedSettings = array_merge( $recommendedSettings, $extension->recommendedSettings ); foreach ( $extension->settings() as $k => $setting ) /* <- settings() runs AFTER the merge */ and later, on save, $values = array_merge( $values, $recommendedSettings ). If you populate $this->recommendedSettings inside settings(), the merge has already happened and your defaults never enter the pool. IPS\core\extensions\core\Sitemap\Content does exactly this — it fills recommendedSettings in the body of settings() — and gets away with it only because its own saveSettings() ignores $recommendedSettings entirely and writes json_encode( array() ) in the recommended branch.
Declare public array $recommendedSettings as a literal property, the way Clubs, Pages, Databases and Subscriptions do. If your defaults genuinely have to be computed, do it in the constructor, not in settings().
My settings fields stay visible when "use recommended settings" is switched on
The master YesNo is built as new YesNo( 'sitemap_configuration_info', $useRecommendedSettings, FALSE, array( 'togglesOff' => $toggles ) ), and $toggles is filled with $setting->htmlId. In IPS\Helpers\Form\FormAbstract, public ?string $htmlId = NULL; and it is only set from the eighth constructor argument:
$this->htmlId = $id ? preg_replace( "/[^a-zA-Z0-9\-_]/", "_", $id ) : NULL; Omit it and $toggles collects a NULL, the toggle targets nothing, and your rows stay on screen. Every core Sitemap extension passes the field name again as the eighth argument for this reason. The argument list is ( $name, $default, $required, $options, $validation, $prefix, $suffix, $id ) — you must pass four NULLs to reach it.
Saving the sitemap settings page applies some apps' settings but not others
The save loop has no protection whatsoever:
foreach( Application::allExtensions( 'core', 'Sitemap', FALSE, 'core' ) as $extKey => $extension ) { $extension->saveSettings( $values ); } An exception from your saveSettings() aborts the loop. Extensions iterated before yours have already written their settings; extensions after yours never run; and $form->saveAsSettings() for sitemap_url and the ACP audit-log entry (acplogs__seo_sitemap_settings) are both skipped. The admin sees a suite-level error and a half-saved form.
Remember also that $values is the whole form, not your slice of it: it contains sitemap_url, sitemap_configuration_info and every other app's fields. Only read keys you defined, and always handle the $values['sitemap_configuration_info'] branch — it is present on every save.
Nobody's sitemap updates any more, and nothing in my app logs an error
This is the worst one. IPS\Sitemap::buildNextSitemap() calls your getFilenames() with no try/catch, twice over:
foreach ( $extensions as $extension ) { $files = array_merge( $files, $extension->getFilenames() ); /* no try/catch */ } /* ... */ foreach( $extensions as $extension ) { if( in_array( $toBuild, $extension->getFilenames() ) ) /* no try/catch */ { $extension->generateSitemap( $toBuild, $this ); /* no try/catch */ } } The only handler is one level up, in applications/core/tasks/sitemapgenerator.php:
catch( Exception $e ) { Log::log( $e, 'sitemap_generator' ); $generator->log[] = $e->getMessage(); return FALSE; /* stops runUntilTimeout for this whole run */ } Because the filename union is built before any file is generated, a throw from your getFilenames() means zero sitemap files are built that hour, for every application on the site. And it does not resolve itself: the task runs on a 60-minute schedule (P0Y0M0DT0H60M0S), constructs a fresh IPS\Sitemap each time, and re-derives the list from scratch — so it hits your exception again immediately, forever.
The error surfaces nowhere near its cause. The only trace is a row in ACP → System → Logs under the key sitemap_generator, plus the message attached to the sitemapgenerator task's log. Nothing points at your extension, and the visible symptom is that some other app's sitemap has gone stale. Worse, catch( Exception $e ) does not catch \Error — a TypeError or a call to an undefined method inside getFilenames() is not logged at all and takes the task down.
Wrap the whole body of getFilenames() in your own try/catch ( \Throwable ) and return something sane.
sitemap.php?file=... returns 404 and my previously built files have vanished
The exact response is sitemap_not_found with error code 2C152/1, raised in applications/core/modules/front/sitemap/sitemap.php. The cause is upstream, in the first block of buildNextSitemap():
Db::i()->delete( 'core_sitemap', Db::i()->in( 'sitemap', $files, TRUE ) ); Once per task run, core deletes every row in core_sitemap whose name is not in the union of all extensions' current getFilenames() output. If your implementation returns a shorter list than last hour — because a count query failed, because a cache was cold, because a permission check flipped — those rows are gone, along with their last_id cursors, and the URLs 404 until the file is rebuilt from block 1.
There is a sharper edge here. Db::in() returns the literal string '1=1' when the value array is empty and $reverse is true:
return $reverse ? '1=1' : '1=0'; So if every Sitemap extension returns an empty array on a given run, the delete becomes DELETE FROM core_sitemap WHERE 1=1 and truncates the table.
Make getFilenames() deterministic and total. Return the same list for the same data, never let it throw, and on a transient failure return the filenames already recorded in core_sitemap for your prefix rather than an empty array.
Two apps' entries overwrite each other in the same file
core_sitemap.sitemap is a single flat namespace with a PRIMARY KEY over the first 191 characters — there is no application column. When core picks a file to build it calls every extension that claims it, with no break:
foreach( $extensions as $extension ) { if( in_array( $toBuild, $extension->getFilenames() ) ) { $extension->generateSitemap( $toBuild, $this ); } } and buildSitemapFile() finishes with Db::i()->replace( 'core_sitemap', ... ). Two extensions returning the same string both run, both replace the same row, and the one that runs later wins. No error, no warning — one app's URLs simply never appear. Note that iteration order is not stable in a useful way either: allExtensions() is called with $firstApp = 'core', which only guarantees that core sorts first.
Prefix every filename with your application directory, and keep the whole string under 191 characters. Watch out for prefixes that are substrings of each other, and for per-record prefixes: core's own Clubs extension emits {clubId}_sitemap_clubs_pages_{n}, and Databases emits {databaseId}_sitemap_database_records_{n}.
My file is generated every hour but never appears in the sitemap index
Two separate causes, and they look identical from outside.
If $entries is empty, buildSitemapFile() stores data => NULL, and the index builder selects with array( 'data IS NOT NULL' ). The row exists, its updated timestamp keeps refreshing, and the file never appears in <sitemapindex>. That is intended — an empty file is skipped, not published.
If generateSitemap() returns without calling buildSitemapFile() at all — the early-return pattern that Content, Images, Videos, Pages and Databases all use when their count setting is zero — then no row is ever written. That filename stays in array_diff( $files, ...existing rows... ) on every single run, so core picks it as an unbuilt file first, every hour, forever, and no other file ever gets a turn. The build makes no progress and the log is empty.
Always call buildSitemapFile() before returning, even with array(). If you want a file not to exist, do not return its name from getFilenames().
The <priority> element is missing, or contains -1
Sitemap::$priorities holds strings: '1.0' => '1.0' through '0.1' => '0.1'. The Select field therefore returns a string. Core's content extensions then do:
$priority = ( $item->sitemapPriority() ?: ( intval( $settings["sitemap_{$class::$title}_priority"] ?? 1 ) ) ); if ( $priority !== -1 ) { $data['priority'] = $priority; } intval( '0.8' ) is 0. And buildSitemapFile() only writes the element when the value is truthy (isset( $entry['priority'] ) AND $entry['priority']), so 0 is dropped. The net effect in core is that every priority between 0.1 and 0.9 produces no <priority> element at all — including Content::RECOMMENDED_NODE_PRIORIY, which is 0.6. Only 1.0 survives, as <priority>1</priority>.
The opposite mistake is in Clubs and Subscriptions, which skip intval() but compare strictly:
$priority = $settings['sitemap_club_priority'] ?? 1; if ( $priority !== -1 ) { $data['priority'] = $priority; } Those fields are declared with 'unlimited' => '-1', 'unlimitedLang' => 'sitemap_dont_include', and Select::getValue() returns $this->options['unlimited'] verbatim — the string '-1'. '-1' !== -1 is true, so choosing "Don't include" writes <priority>-1</priority> into the served XML, which is outside the 0.0–1.0 range the sitemap schema allows.
Do neither. Keep the value as a string, and if you offer a "don't include" option compare it loosely or against the string: if ( (string) $priority !== '-1' ).
Returning the last ID from generateSitemap() has no effect
The abstract declares : ?int and the docblock calls it the last built ID, but both callers discard it:
/* IPS\Sitemap::buildNextSitemap() */ $extension->generateSitemap( $toBuild, $this ); /* IPS\core\extensions\core\Queue\RebuildSitemap::run() */ $extension->generateSitemap( $name, $sitemap ); return $offset + 1; The only thing that persists a cursor is the third argument to buildSitemapFile(), which writes core_sitemap.last_id. If you return the ID but pass 0 (or omit the argument, as cms\Pages and cms\Databases do), last_id stays 0 and every subsequent block falls back to LIMIT offset, 500 paging.
Two consequences for batching. First, block n reads block n-1's row, so block 1 always throws UnderflowException and must fall back to offset paging — catch it, as every core implementation does. Second, advance your cursor even for rows you skip: Content updates $lastId before continue-ing on an unviewable item, precisely so that a block of 500 invisible items still moves the cursor forward instead of stalling.
Note too that buildNextSitemap() chooses files by "never built" first, then by updated ASC — it does not guarantee blocks are rebuilt in numerical order. A cursor read from block n-1 may be an hour old.
My image:, video: or news: elements never appear in the XML
Namespaced child elements are only processed inside if( count( $namespaces ) ). If you do not pass the fourth argument to buildSitemapFile(), every extra key in your entry array is skipped in complete silence — no warning, no malformed XML, just missing elements.
Three further undocumented requirements, all readable in the writer loop. The entry key must literally begin with the prefix plus a colon (mb_strpos( $key, $prefix . ':' ) === 0). The value must be an array — a scalar reaches foreach( $value as $k => $v ) and raises "foreach() argument must be of type array|object". And a nested value that is itself an array uses a positional convention: index 0 becomes the element text and every other key becomes an attribute. gallery's Videos extension is the only example of that last form in core:
$data['video:video'] = array( 'video:content_loc' => (string) $url, 'video:title' => $item->mapped('title'), /* array( 0 => element text, 'attribute' => attribute value ) */ 'video:uploader' => array( 0 => $item->author()->name, 'info' => (string) $item->author()->url() ), ); $sitemap->buildSitemapFile( $filename, $entries, $lastId, array( 'video' => 'https://www.google.com/schemas/sitemap-video/1.1' ) ); One more thing about URLs: buildSitemapFile() rewrites $entry['url'] with an anchored preg_replace( '/^' . preg_quote( Settings::i()->base_url, '/' ) . '/', '{base_url}', ... ), and the front controller substitutes it back with str_replace( '<loc>{base_url}', ... ). Only the url key gets this treatment — namespaced values like image:loc are stored absolute. If your URL does not start with base_url character-for-character (different scheme, www mismatch, CDN host) the token is not applied and the absolute URL is baked into the stored XML, surviving any later change of base_url until the file is rebuilt.
Only the first few of my sitemap files ever get rebuilt
Two compounding budget problems.
The sitemapgenerator task runs hourly and calls runUntilTimeout(), which allows roughly 90 seconds (less if max_execution_time or MySQL's wait_timeout is lower). Every sitemap file on the entire site — yours, forums', gallery's, Pages' — competes for that one window. Files are eligible for rebuild once their updated is more than an hour old, ordered updated ASC, so they do rotate, but returning thousands of filenames from getFilenames() starves everybody.
The second problem is that getFilenames() is not called once — it is called for every extension, on every call to buildNextSitemap(), i.e. once per file built per extension. A default install has a Sitemap extension per routed content class (each generated by IPS\Content\ExtensionGenerator), so a run that builds twenty files can make several hundred getFilenames() calls. Core's own Content::getFilenames() runs permission-index joins and per-node $node->_items lookups on every one of them.
Memoise the result on the instance, as the example above does. The same object is reused for the whole run, so a single protected ?array $filenames cache removes the repeat cost entirely.
Clicking "Rebuild Sitemap" does nothing for my app
The ACP action deletes the whole of core_sitemap and then queues one core/RebuildSitemap background task per extension at priority 5. RebuildSitemap::preQueueData() then does:
$files = $extension->getFilenames(); $data['count'] = count( $files ); if( $data['count'] == 0 ) { return NULL; } and Task::queue() treats a NULL return as "nothing to do" — it calls postComplete( [], FALSE ) and inserts no queue row. So if getFilenames() is empty at the moment the button is pressed, your app is silently excluded from the rebuild, with no progress bar and no log line.
Note also that $data['files'] is snapshotted at queue time and run() reads $data['files'][$offset] from that snapshot. If your file list changes while the queue drains, the queue keeps building the old names. And run() calls generateSitemap() with no try/catch: in Task::runQueue(), only IPS\Task\Queue\OutOfRangeException (completion) and OutOfRangeException (rethrown as RuntimeException) are handled, so any other exception escapes before core_queue.offset is updated — the same row is retried on the next queue tick indefinitely.
Verified against
Read from Invision Community 5.0.19 source: system/Extensions/SitemapAbstract.php, system/Sitemap/Sitemap.php, system/Application/Application.php (allExtensions(), extensions(), constructExtensionClass()), system/Db/Db.php (in()), system/Helpers/Form/FormAbstract.php and Select.php, applications/core/tasks/sitemapgenerator.php, applications/core/extensions/core/Queue/RebuildSitemap.php, applications/core/modules/admin/promotion/seo.php, applications/core/modules/front/sitemap/sitemap.php, and all seven shipped implementations under applications/{core,cms,gallery,nexus}/extensions/core/Sitemap/. Schema details come from applications/core/data/schema.json and the SITEMAP_MAX_PER_FILE default from init.php.
Related: The core/ContentRouter extension in Invision Community 5, The core/MetaData extension in Invision Community 5, Extension method signatures are fixed, and a mismatch is a fatal on install
Related application: SEO Toolkit — SEO Toolkit builds on this same extension so a new content type is in the sitemap and the search-engine metadata without a separate list to maintain.

0 comments
38 views

Ernest Defoe

July 26Jul 26

The core/AccountSettings extension in Invision Community 5

The core/AccountSettings extension adds a tab to the member-facing account settings screen — the page at /settings, served by IPS\core\modules\front\system\settings. Core calls it in exactly one place, that controller's manage() method, and it calls it twice per page load: once to find which extension owns the requested area and render its body, and once more (inside _wrapOutputInTemplate()) to build the sidebar list of every tab.
Core ships exactly one implementation of this extension, IPS\core\extensions\core\AccountSettings\Referrals. Everything else on that page — email, password, MFA, devices, username, signature, OAuth apps — is a hard-coded method on the controller, not an extension. That asymmetry is the source of about half the traps below.
The contract
namespace IPS\Extensions; abstract class AccountSettingsAbstract { /* Font Awesome icon name, WITHOUT the "fa-" prefix. The template renders it as: <i class="fa-solid fa-{$icon}"> Must stay static — core reads it as $ext::$icon, never $ext->icon. */ public static string $icon = 'cog'; /* REQUIRED. The tab key. Doubles as: - the URL segment (/settings/<key>) - the array key in the $tabs list - the default language key for the tab label Return NULL to hide the tab AND make its content unreachable. */ abstract public function getTab() : string|null; /* REQUIRED. The HTML body of the tab. Returned, not echoed. Returning '' is NOT the same as returning nothing useful — see below. */ abstract public function getContent() : string; /* OPTIONAL. A LANGUAGE KEY, not a literal title. Default implementation returns $this->getTab(). */ public function getTitle() : string; /* OPTIONAL. TRUE puts a fa-triangle-exclamation badge on the sidebar item. */ public function showWarning() : bool; } No method on this abstract declares a thrown exception, and — this matters — core does not catch any.
The file lives at applications/<app>/extensions/core/AccountSettings/<Class>.php in namespace IPS\<app>\extensions\core\AccountSettings, and must be registered in applications/<app>/data/extensions.json:
{ "core": { "AccountSettings": { "MyTab": "IPS\\myapp\\extensions\\core\\AccountSettings\\MyTab" } } } This is the exact consumer code, from manage() in applications/core/modules/front/system/settings.php:
$area = Request::i()->area ?: 'overview'; $methodName = "_{$area}"; if ( method_exists( $this, $methodName ) ) { $output = $this->$methodName(); } else { foreach( Application::allExtensions( 'core', 'AccountSettings', TRUE, 'core' ) as $ext ) { $tabName = $ext->getTab(); if( $tabName == $area ) { if( isset( Request::i()->action ) AND method_exists( $ext, Request::i()->action ) ) { $method = Request::i()->action; $output = $ext->$method(); } else { $output = $ext->getContent(); } } } } if( !isset( $output ) ) { Output::i()->error( 'node_error', '2C122/2', 404 ); } Note the things that are not there: no try, no break, and no CSRF check.
A minimal example
applications/myapp/extensions/core/AccountSettings/Privacy.php:
<?php namespace IPS\myapp\extensions\core\AccountSettings; use IPS\Extensions\AccountSettingsAbstract; use IPS\Log; use IPS\Member; use IPS\Output; use IPS\Settings; use IPS\Theme; use function defined; if ( !defined( '\IPS\SUITE_UNIQUE_KEY' ) ) { header( ( $_SERVER['SERVER_PROTOCOL'] ?? 'HTTP/1.0' ) . ' 403 Forbidden' ); exit; } class Privacy extends AccountSettingsAbstract { /** * @var string */ public static string $icon = 'shield-halved'; /** * The tab key, or NULL to hide the tab entirely. * * This runs on EVERY request to the settings controller, including * requests for somebody else's tab. Keep it cheap and make it return, * never throw and never redirect. */ public function getTab() : string|null { if ( !Settings::i()->myapp_enabled or !Member::loggedIn()->member_id ) { return NULL; } return 'myapp_privacy'; } /** * The tab label. A LANGUAGE KEY. Omit this method and the key * returned by getTab() is used instead. */ public function getTitle() : string { return 'myapp_privacy_tab'; } /** * The tab body. */ public function getContent() : string { /* Re-check here. getTab() gates access, but a defence in depth check costs nothing and error() is safe in this method. */ if ( !Settings::i()->myapp_enabled ) { Output::i()->error( 'myapp_disabled', '2MYAPP/1', 403, '' ); } try { return Theme::i()->getTemplate( 'settings', 'myapp', 'front' )->privacy( Member::loggedIn() ); } catch ( \Throwable $e ) { /* Contain it. An uncaught throw here takes down the whole settings page, including the core password and email tabs. */ Log::log( $e, 'myapp' ); return Member::loggedIn()->language()->addToStack( 'myapp_privacy_unavailable' ); } } } Then define a language string for the title in applications/myapp/dev/lang.php:
'myapp_privacy_tab' => "Privacy", Members report they can no longer change their password or email address
This is the big one, and the cause is nowhere near the symptom. The loop that collects tabs has no try/catch anywhere in it, and it calls getTab() on every registered extension on every request to the settings controller — including the request for /settings/password, which has nothing to do with your app.
So if your getTab() throws — a bad Db query, a Settings value your install migration never wrote, an OutOfRangeException from a Member::load() — every tab on the account settings page dies with a 500 or a generic EX error, including all of core's. The member (and the admin) will report "I can't change my password". Nothing in that report points at your app.
The same is true of getTitle() and showWarning(), which run inside _wrapOutputInTemplate() for every extension on every full page load.
Fix: treat all four methods as if they run in a shared, unguarded loop, because they do. getTab() should contain nothing that can throw — read a setting, check member_id, return. Wrap anything risky in getContent() in a try/catch ( \Throwable $e ), log it, and return something. Do not return '' from the catch block; see the blank-page trap below.
Every settings tab returns my app's "you do not have permission" error
The mirror image of the previous trap, and it has caught people who moved a permission check into the wrong method. If you call Output::i()->error() or Output::i()->redirect() inside getTab(), it fires during the tab-collection loop for whatever area the member actually requested. A member clicking "Devices" gets your app's 403 instead.
Core's own extension shows the correct split. Referrals::getTab() only returns a value or NULL:
public function getTab() : string|null { return Settings::i()->ref_on ? 'referrals' : null; } and the Output::i()->error( 'referrals_disabled', '2C122/V', 403, '' ) call lives in getContent(), which only runs when the member is actually on that tab.
My tab renders a completely blank page with no sidebar
Returning an empty string from getContent() does not produce an empty tab — it produces an empty page. manage() tests the return value for truthiness, not for isset:
if ( $output ) { Output::i()->output .= $this->_wrapOutputInTemplate( $area, $output ); } An empty string is falsy, so _wrapOutputInTemplate() is never called, the sidebar and the entire settings chrome are never rendered, and the member gets the site header, the footer, and nothing between them. No error is raised, no 404, nothing in the logs.
Note also that '' does not trigger the node_error 404, because isset( $output ) is still TRUE. You get a silent blank page rather than a diagnosable failure.
Fix: never return ''. Return a message, even a bare language string, from every path including your catch blocks.
My tab shows the raw key "myapp_privacy" as its label
Both the sidebar's title attribute and its visible text run the value through the language system:
{{if isset( $tab['title'] )}}{lang="$tab['title']"}{{else}}{lang="$key"}{{endif}} So getTitle() must return a language key, not a title. And because the abstract's default getTitle() is return $this->getTab();, an extension that does not override it needs a language key whose name is exactly the tab key. Core does exactly that — applications/core/dev/lang.php contains 'referrals' => "Referrals".
When the key is missing, IPS\Lang::replaceWords() falls through to its else branch and substitutes the key itself:
else { $values['options']['escape'] = TRUE; $replacement = $values[ 'key' ]; } There is no warning and no log entry. The tab just reads myapp_privacy.
My tab key is ignored and core's Email or Password page renders instead
manage() checks the controller for a method named _<area> before it looks at any extension, and method_exists() in PHP is case-insensitive. If your tab key collides with one of the controller's private area methods, your extension is never consulted at all — getTab() is not even called.
As of 5.0.19 the reserved area keys, taken from the _-prefixed methods on IPS\core\modules\front\system\settings, are:
overview, email, password, devices, mfa, securityquestions, username, links, signature, login, apps plus two that are not tabs at all and will produce an ArgumentCountError if requested: wrapOutputInTemplate and performRedirect This list can grow in any core release, which is the real argument for the fix: prefix your tab key with your app directory, e.g. myapp_privacy rather than privacy. That also protects you from the next trap.
Two apps installed, one tab disappears and clicking it shows the other app's content
The extension loop has no break, and the sidebar builder does a plain array assignment:
foreach( Application::allExtensions( 'core', 'AccountSettings', TRUE, 'core' ) as $ext ) { if( $key = $ext->getTab() ) { $tabs[ $key ] = [ /* ... */ ]; } } If two extensions return the same key, the later one wins in both places — it overwrites the sidebar entry (icon, title, warning) and, because manage() keeps looping after a match, its getContent() overwrites $output too. The earlier extension's getTab() and getContent() still run, they just have their result thrown away. Any side effects or queries they perform still happen.
Extension order is not alphabetical. allExtensions() is called with $firstApp = 'core', which usorts core to the front; everything after that follows application load order. Do not rely on it. Namespace your key.
My tab does not appear at all and nothing is logged
There are four independent silent-drop paths, none of which produce an error:
The class cannot be loaded. Application::extensions() skips any entry that fails is_string( $classname ) or class_exists( $classname ) with the comment "Switching between branches confuses extensions". A namespace that does not match the directory, a typo in extensions.json, or a parse error in your file all land here and are simply continued. Your constructor threw. Application::constructExtensionClass() does catch( RuntimeException | OutOfRangeException $e ){} — an empty catch — and then returns null, which drops the extension from the returned array. Note that the constructor is called as new $class( Member::loggedIn() ), so if you define a constructor it receives a Member. Your app is not accessible to this member. The call passes $checkAccess = TRUE, so $application->canAccess( NULL ) is evaluated against the logged-in member and the whole app's extensions are skipped if it fails. The extension list is cached. allExtensions() reads and writes Store::i()->extensions, keyed only by extension type. It is cleared by the ACP Developer Center when you add or delete an extension (applications/core/modules/admin/developer/extensions.php), but if you hand-edit extensions.json the stale cache persists until something clears it. Also note that RECOVERY_MODE skips every non-IPS application, so no third-party tab appears at all in recovery mode.
My extension vanishes as soon as I add a method called generate()
This one is genuinely surprising. Application::extensions() checks method_exists( $classname, 'generate' ) and, if present, treats your class as a generator of other extensions rather than as an extension:
if ( method_exists( $classname, 'generate' ) ) { $generated = $classname::generate(); foreach( $generated as $k => $v ) { $classes[ $k ] = [ 'generate' => $v::class, 'class' => $v->class ]; } } else { $classes[ $name ] = $classname; } generate is a reserved method name across every IPS extension type, not just this one. If you add a helper called generate() to an AccountSettings class, the class is called statically, the return value is iterated as a list of objects with a ->class property, and your tab simply never appears. Pick another name.
My tab's CSS only works when I load the URL directly, not when I click the tab
The settings sidebar is an ipsTabBar, and dev/js/framework/common/ui/ips.ui.tabbar.js loads each tab over AJAX from its href, replacing only the content area. On that request Request::i()->isAjax() is TRUE, so manage() takes the other branch:
elseif ( $output ) { Output::i()->output .= $output; } _wrapOutputInTemplate() is skipped and no global template — and therefore no global/includeCSS — is rendered. Anything you appended to Output::i()->cssFiles or Output::i()->jsFiles inside getContent() is silently discarded. IPS\Dispatcher\Front::baseCss() confirms this is by design: it does not even load core.css on AJAX requests.
The symptom appears one navigation away from the cause. Landing on /settings/myapp_privacy directly works and looks correct. Landing on /settings and then clicking your tab in the sidebar renders the same HTML unstyled. Testing only by pasting the URL will never reproduce it.
Fix: put your styles in a CSS file loaded on every page of the app, or scope the tab's presentation to inline attributes and existing framework classes rather than a tab-specific stylesheet. Core's Referrals::getContent() does append styles/referrals.css and has the same limitation.
Any public method on my extension class is callable straight from a URL
The action branch is a dynamic method call driven by unvalidated request input, with no allow-list and no CSRF token check:
if( isset( Request::i()->action ) AND method_exists( $ext, Request::i()->action ) ) { $method = Request::i()->action; $output = $ext->$method(); } Three consequences worth planning for:
Anything public is web-reachable. A request for /settings/myapp_privacy?action=deleteAllData calls that method as a GET, with no arguments and no CSRF validation. If you add action methods, validate the request inside them — Request::i()->confirmedDelete(), a ->csrf() check, or a form — and never assume the extension dispatcher did it for you. method_exists() is TRUE for protected and private methods too. Calling one throws an Error, which nothing catches, so it becomes a 500 on the settings page. Non-string returns are not validated. $output here is untyped. Requesting ?action=showWarning assigns a bool: isset( $output ) is TRUE so there is no 404, and if it is FALSE the truthiness test skips rendering entirely and the member gets the blank page described above. The good news is that the action dispatch sits inside the $tabName == $area test, so an extension whose getTab() currently returns NULL is unreachable by this route as well. Gating in getTab() is a real access control, not just a display toggle.
404 "node_error" with code 2C122/2 on a URL that used to work
If no controller method and no extension claims the area, core raises Output::i()->error( 'node_error', '2C122/2', 404 ). On a custom tab that means getTab() returned NULL — or returned a different key — for this particular request.
The subtlety is ordering: getContent() runs before the sidebar is built, so getTab() is evaluated once before your content executes and again afterwards. If getContent() mutates state that getTab() reads — a member setting the tab itself toggles, for instance — the tab can render its content and then vanish from the sidebar in the same response, and the next request 404s. Keep getTab()'s inputs stable across a single request.
Note also that the sidebar test is if( $key = $ext->getTab() ), a truthiness test, while the content test is if( $tabName == $area ), a loose comparison. A tab key of '0' is falsy: it would be reachable by URL but never listed. Another reason to prefix keys with your app directory.
A note on the icon
$icon is declared public static string on the abstract and read as $ext::$icon. Redeclaring it as a non-static property in your subclass is a PHP compile-time fatal, not a silent fallback. The template interpolates it as fa-solid fa-{$icon}, so supply the bare icon name ('key', 'users', 'shield-halved') with no fa- prefix. Core exploits the raw interpolation elsewhere on the same page — the login-method loop sets 'brands fa-' . $icon to reach the brands style — so the same trick is available if you need a brand glyph. If $icon is unset you inherit 'cog'; the template's final fallback, fa-square, is only reached when the key is absent from the array entirely, which cannot happen for extension tabs.
Verified against
Read from an Invision Community 5.0.19 source tree (applications/core/data/versions.json, build 5001908). The files behind every claim above: system/Extensions/AccountSettingsAbstract.php, applications/core/extensions/core/AccountSettings/Referrals.php, applications/core/modules/front/system/settings.php (manage() and _wrapOutputInTemplate()), applications/core/dev/html/front/system/settings.phtml, system/Application/Application.php (allExtensions(), constructExtensionClass(), extensions()), system/Lang/Lang.php (replaceWords()), system/Dispatcher/Front.php (baseCss()) and dev/js/framework/common/ui/ips.ui.tabbar.js. Invision Cloud ships an additional implementation (IPS\cloud\extensions\core\AccountSettings\Expert) which is not present in a self-hosted tree and was not read.
Related: The core/ProfileSteps extension in Invision Community 5, The core/MemberACPProfileTabs extension: AdminCP member view tabs in Invision Community 5, Extension method signatures are fixed, and a mismatch is a fatal on install

0 comments
33 views

Ernest Defoe

July 26Jul 26

The core/ModeratorPermissions extension in Invision Community 5

The core/ModeratorPermissions extension is how an application adds fields to AdminCP → Members → Staff → Moderators → [edit], and how it gets told when a moderator record is created, changed or deleted. Every field you declare becomes one key in a single flat JSON blob stored in core_moderators.perms, which is read back at runtime by IPS\Member::modPermission( 'your_key' ).
The only consumer is applications/core/modules/admin/staff/moderators.php. It calls Application::allExtensions( 'core', 'ModeratorPermissions', FALSE ) six times across three methods: add() (fires onChange), edit() (collects toggles, builds the form, then fires preSave and onChange), and delete() (fires onDelete). Nothing else in the suite calls this extension.
The contract
abstract class IPS\Extensions\ModeratorPermissionsAbstract { /* REQUIRED. Returns the fields for your tab. Key = the permission key, which is ALSO the form field name, the language key for the label, and the toggle element id. Value = either a type string, or an array: [0] type - 'YesNo' | 'Number' | 'Node' (see traps) [1] options - options array for IPS\Helpers\Form\<type> [2] prefix - literal string, NOT a language key [3] suffix - literal string, NOT a language key Return [] and no tab is rendered at all. */ abstract public function getPermissions( array $toggles ): array; /* Called once per extension immediately before the values are saved. $values is BY REFERENCE - you may add/remove/rewrite keys. At this point $values['mod_use_restrictions'] is still present ('yes' or 'no'); core unsets it after the preSave loop. */ public function preSave( array &$values ): void; /* Called after core_moderators is written, and also when a moderator is first created. $changed is the string '*' when the admin picked "All permissions", otherwise an array of only the changed keys. DECLARE IT array|string - see the TypeError trap below. */ public function onChange( array $moderator, array|string $changed ): void; /* Called after the core_moderators row is deleted. $moderator is the raw pre-delete DB row: id, type, perms, updated, show_badge. */ public function onDelete( array $moderator ): void; /* Only reachable via core's ContentGenerator, and only if your app owns a routed content ITEM class - see the trap below. The keys are merged into that content class's tab instead of your own. */ public function getContentPermissions( array $toggles ): array; /* Called from ContentGenerator::onDelete(). */ public function onContentDelete( array $moderator ): void; /* DEAD CODE. Nothing in IC 5.0.19 calls this. */ public function onContentChange( array $moderator, array $changed ): void; } $toggles is an array keyed by 24 fixed action names (view_future, future_publish, pin, unpin, feature, unfeature, edit, hide, unhide, view_hidden, move, lock, unlock, reply_to_locked, assign, delete, split_merge, feature_comments, unfeature_comments, add_item_message, edit_item_message, delete_item_message, toggle_item_moderation, view_reports). Each value is a list of per-content-class element ids such as can_edit_topic. It exists so core's own Content tab can pass them as togglesOff; third-party extensions normally ignore it.
A minimal example
applications/myapp/data/extensions.json — the extension is only discovered from this file, not by scanning the directory:
{ "core": { "ModeratorPermissions": { "Perms": "IPS\\myapp\\extensions\\core\\ModeratorPermissions\\Perms" } } } applications/myapp/extensions/core/ModeratorPermissions/Perms.php:
<?php namespace IPS\myapp\extensions\core\ModeratorPermissions; use IPS\Data\Store; use IPS\Extensions\ModeratorPermissionsAbstract; use IPS\Member; use function array_key_exists; use function defined; use function is_array; if ( !defined( '\IPS\SUITE_UNIQUE_KEY' ) ) { header( ( $_SERVER['SERVER_PROTOCOL'] ?? 'HTTP/1.0' ) . ' 403 Forbidden' ); exit; } class Perms extends ModeratorPermissionsAbstract { public function getPermissions( array $toggles ): array { return array( 'myapp_can_approve' => 'YesNo', 'myapp_can_feature' => array( 'YesNo', array( 'togglesOn' => array( 'myapp_feature_limit' ) ) ), /* index [1] must exist even when empty - see the null-options trap */ 'myapp_feature_limit' => array( 'Number', array(), NULL, Member::loggedIn()->language()->addToStack( 'per_day' ) ), ); } public function preSave( array &$values ): void { if ( $values['mod_use_restrictions'] != 'no' and !$values['myapp_can_approve'] ) { $values['myapp_can_feature'] = FALSE; } } public function onChange( array $moderator, array|string $changed ): void { if ( $changed === '*' or ( is_array( $changed ) and array_key_exists( 'myapp_can_approve', $changed ) ) ) { unset( Store::i()->myapp_approvers ); } } public function onDelete( array $moderator ): void { unset( Store::i()->myapp_approvers ); } } applications/myapp/dev/lang.php — four keys are mandatory, one per field plus the tab:
'modperms__myapp_Perms' => "My App", 'myapp_can_approve' => "Can approve submissions?", 'myapp_can_approve_desc' => "Optional. Rendered under the field.", 'myapp_can_feature' => "Can feature submissions?", 'myapp_feature_limit' => "Features allowed per day", Checking it at runtime:
if ( Member::loggedIn()->modPermission( 'myapp_can_approve' ) ) { ... } The tab is labelled "modperms__myapp_Perms" and every checkbox is labelled with its own key
The ACP shows the literal strings modperms__myapp_Perms, myapp_can_approve and so on instead of readable text.
Three separate language keys are derived from names you never explicitly write down. The tab key is 'modperms__' . $k where $k is the allExtensions() key, i.e. <app directory>_<extension name from extensions.json> — so an extension registered as "Perms" in app myapp needs modperms__myapp_Perms. The field label is the permission key itself, because FormAbstract::rowHtml() falls back to addToStack( $this->name ). The optional description is <key>_desc and is requested with returnBlank, so it is safe to omit.
When a key is missing, Lang::replaceWords() hits its else branch and substitutes $values['key'] — the raw key — rather than throwing. That is why the failure is silent and only visible on screen.
Two things that are not language keys: the prefix and suffix entries at indexes [2] and [3]. They are handed straight to the row template as literal output. Core's Warnings extension therefore writes Member::loggedIn()->language()->addToStack('per_day') for its suffix rather than 'per_day'.
The whole Moderator Permissions screen dies with an error from an app that isn't yours
An admin opens one moderator for editing and gets a fatal error or a blank page mentioning a class in a completely unrelated application.
This error surfaces somewhere other than where it was caused. The edit form is a single shared screen built from every installed application's extension in one uninterrupted loop:
foreach( $extensions as $k => $ext ) { $form->addTab( 'modperms__' . $k ); foreach ( $ext->getPermissions( $toggles ) as $name => $data ) ... There is no try/catch anywhere in that loop, and no per-extension isolation. Any exception, TypeError or SQL failure thrown from any app's getPermissions() takes down the entire moderator editor for the whole suite, and the traceback names the offending app, not the moderator controller.
This is not hypothetical: core's own IPS\forums\extensions\core\ModeratorPermissions\Topic::getContentPermissions() runs three raw counts against forums_forums and forums_topic_mmod every time the form is drawn. Keep getPermissions() cheap and defensive — wrap your own queries, and never let a missing table or setting escape:
public function getPermissions( array $toggles ): array { $return = array( 'myapp_can_approve' => 'YesNo' ); try { if ( Db::i()->select( 'COUNT(*)', 'myapp_queues' )->first() ) { $return['myapp_can_flush_queue'] = 'YesNo'; } } catch ( Exception $e ) {} return $return; } Your tab silently never appears, and nothing is logged
The extension file exists, it is in extensions.json, and there is no error — the tab is simply absent.
There are two distinct causes, both silent by design.
First, an empty return. Form::addTab() writes to $form->tabs, but the moderator template iterates $form->elements (passed in as $tabs by customTemplate()), and elements[$tab] is only created by _insert() when a field is actually added. So returning array() from getPermissions() produces no tab at all. This is deliberate in core: blog\Blogs, downloads\Files and forums\Topic all return [] so their keys land on the Content tab instead. But if all your keys are behind a settings check (Settings::i()->clubs, as in core's Clubs extension), your tab vanishes when the setting is off.
Second, a throwing constructor. Application::constructExtensionClass() does:
try { $obj = new $classToUse( ... ); ... return $obj; } catch( RuntimeException | OutOfRangeException $e ){} return null; A RuntimeException or OutOfRangeException thrown from your __construct() makes the object null, and the caller just skips it. No log entry, no error, no tab. The same happens if class_exists() is false, which catches namespace typos and stale extensions.json entries.
Note also that this extension is constructed with $checkAccess = FALSE, which means your constructor is called with a single argument of NULL. If you declare __construct( Member $member ) without a nullable type you get a TypeError, which is not in the catch list and will fatal the whole screen.
"Argument #4 ($options) must be of type array, null given"
A fatal TypeError pointing at IPS\Helpers\Form\FormAbstract::__construct(), usually preceded by Warning: Undefined array key 1.
You used the array form of a field but only supplied the type:
/* WRONG */ return array( 'myapp_can_approve' => array( 'YesNo' ) ); The controller does $options = is_array( $data ) ? $data[1] : array(); with no isset() guard. Index 1 is missing, $options becomes NULL, and FormAbstract::__construct() declares array $options=array(), so PHP refuses it. Either use the plain string form, or always supply index 1:
/* Both fine */ return array( 'myapp_can_approve' => 'YesNo', 'myapp_feature_limit' => array( 'Number', array() ), ); Existing moderators are denied a permission you just added
You ship an update with a new key. Admins see the new checkbox and it is off, and modPermission() returns FALSE for every existing restricted moderator until somebody re-saves each one by hand.
Permissions are stored as a snapshot, not a schema. edit() writes json_encode( $values ) of whatever fields were on the form at that moment, and Member::modPermission() ends with:
return $permissions[$key] ?? false; An unknown key is false. There is no upgrade hook and no default. The one exception is a moderator whose perms column is the literal string *: modPermission() short-circuits with if ( $permissions === '*' or $key === NULL ) return TRUE;, so unrestricted moderators get every future permission automatically.
The symptom appears in the front end, weeks after the install, with nothing in the moderator record to explain it. If your permission must default to granted for existing restricted moderators, write an upgrade step that decodes core_moderators.perms, inserts your key, re-encodes it, and then unset( Store::i()->moderators ).
A field comes up pre-filled with the previous field's value, or "Undefined variable $currentValue"
Only when the moderator is set to "All permissions". A CheckboxSet, Select, Text or any other type renders with a nonsense default, and in a dev install PHP emits Warning: Undefined variable $currentValue.
The controller computes the "everything is allowed" default with an unguarded switch that has no default branch and never initialises the variable:
if ( $currentPermissions === '*' ) { switch ( $type ) { case 'YesNo': $currentValue = TRUE; break; case 'Number': $currentValue = -1; break; case 'Node': $currentValue = 0; break; } } $currentValue lives in the enclosing foreach scope, so for any other type it silently keeps the value left over from the previous field, or is undefined if yours is the first. Core walks into this itself: core\Content::getPermissions() declares can_recognize_content_options => 'CheckboxSet' directly after a YesNo, so on an unrestricted moderator that checkbox set is constructed with a default value of TRUE.
Restrict yourself to YesNo, Number and Node. Note also that for Number the controller force-injects $options['unlimited'] = -1 after your options, so you cannot choose your own unlimited sentinel.
onContentChange() is never called
You implement onContentChange(), the admin saves the form, and your code never runs. No error.
It is dead code in IC 5.0.19. The only place that could call it is core\ContentGenerator, and its onChange() dispatches to the wrong method:
/* ContentGenerator::onChange() */ foreach ( Application::load( $class::$application )->extensions( 'core', 'ModeratorPermissions' ) as $ext ) { $ext->onChange( $moderator, $changed ); /* not onContentChange() */ } /* ContentGenerator::onDelete() */ foreach ( Application::load( $class::$application )->extensions( 'core', 'ModeratorPermissions' ) as $ext ) { $ext->onContentDelete( $moderator ); /* this one IS the content variant */ } Core is affected by its own bug: forums\Topic::onContentChange() exists to rebuild the search index when can_read_all_topics changes, and it never fires. Put your logic in onChange() instead — content permission keys arrive there too, because they are merged into the same flat $values array. onContentDelete() does work and can be used as documented.
onChange() throws a TypeError, or $changed contains an IPS\Member object
Either TypeError: ...onChange(): Argument #2 ($changed) must be of type array, string given, or your code reads $changed and finds keys named moderators_type, moderators_group and moderators_member that no permission extension ever declared.
onChange() is called from two controller methods with three entirely different payloads.
Caller$moderator$changed edit(), admin picked "Restricted"the pre-save DB rowarray of only the keys whose value differs from the stored JSON edit(), admin picked "All permissions"the pre-save DB rowthe string '*' add(), moderator first createdthe row just inserted, perms is '*'the add form's values: moderators_type, moderators_group, and moderators_member as an IPS\Member object The abstract's signature is array|string $changed, but the older docblocks scattered through core (including forums\Topic) still say @param array $changed. Copy the signature, not the docblock, and guard both shapes:
public function onChange( array $moderator, array|string $changed ): void { if ( $changed === '*' ) { /* switched to unrestricted - assume everything changed */ } elseif ( is_array( $changed ) and array_key_exists( 'myapp_can_approve', $changed ) ) { ... } } Also note that edit() skips the onChange loop entirely when the moderator was already unrestricted and stays unrestricted (if( !( $currentPermissions == '*' AND $changed == '*' ) )), so a no-op save on an unrestricted moderator fires nothing.
preSave() edits are silently discarded
You modify $values inside preSave(), and sometimes the change sticks and sometimes it does not.
The abstract declares preSave( array &$values ) and the controller calls it by reference, so the direct call works. But core\ContentGenerator — which extends IPS\Content\ExtensionGenerator, not the abstract, so PHP never checks the signature — re-dispatches with the reference dropped:
/* ContentGenerator::preSave() */ public function preSave( array $changed ): void /* no ampersand */ { foreach ( Application::load( $class::$application )->extensions( 'core', 'ModeratorPermissions' ) as $ext ) { $ext->preSave( $changed ); } } Your preSave() therefore runs once effectively (from the controller) and once per content item class in your app with the result thrown away. Make preSave() idempotent and free of side effects other than rewriting $values.
One more detail: $values['mod_use_restrictions'] is still set during preSave() and only unset afterwards, which is what core's Warnings::preSave() relies on. Everything that survives preSave() is JSON-encoded into perms, including mod_show_badge, so modPermission( 'mod_show_badge' ) is a real (if unintended) lookup.
onChange() and onDelete() run more than once per save
Your cache-clear or queue-task in onChange() executes two or three times for a single click of Save.
Same root cause as above. The controller loops over all extensions, which includes both your own myapp_Perms entry and one core_Content_<YourItemClass> entry generated by ContentGenerator::generate() for every routed content item class in the suite. Each of those generated instances then calls back into your app's extensions. For an app with two content item classes, onChange() fires three times: once directly, twice via the generator.
Never queue a task or write a log row unconditionally from onChange(). Guard on the keys you care about, and prefer idempotent operations such as unset( Store::i()->yourKey ).
Two apps define the same permission key, and the field appears twice
The same checkbox is rendered on two different tabs, and toggling one has no visible effect on the other.
Permission keys are not namespaced. The only thing that carries the application key is the tab language key, modperms__<app>_<Extension>. The field name, the language key, the toggle id and the JSON key in core_moderators.perms are all the bare string you returned. Form::_insert() stores elements as $this->elements[ $tab ][ $elementKey ], so an identical key in two tabs is not deduplicated — both are rendered, both read the same POST field, and both write the same single JSON entry.
Core's own keys are unprefixed and generic — can_edit_tags, can_manage_alerts, can_manage_sidebar, can_see_emails, can_mod_blogs, can_use_saved_actions — so anything obvious is likely already taken. Prefix every third-party key with your application directory (myapp_can_approve). The same applies to keys returned from getContentPermissions(), which land in the same flat namespace.
getContentPermissions() is never called for your app
You implement getContentPermissions() expecting your keys to appear on a content tab, and nothing appears.
It is not reached from the moderator controller at all. The only caller is core\ContentGenerator::getPermissions(), at the very end of the method, and it is gated three ways:
Your application must own at least one content item class registered through a core/ContentRouter extension. ContentGenerator sets protected static bool $contentItemsOnly = TRUE;, so comment and review classes do not generate a tab. That class must have $canBeModeratedFromFrontend = TRUE. ContentGenerator::getPermissions() returns array() before it ever reaches the getContentPermissions() loop otherwise. IPS\cms\Pages\PageItem and IPS\nexus\Package\Item both set it to FALSE. It looks up extensions with Application::load( $class::$application )->extensions( 'core', 'ModeratorPermissions' ), so the extension must be registered in the extensions.json of the app that owns the content class. If your app has more than one qualifying item class, getContentPermissions() is called once per class and the same keys are added to each of those tabs — producing the duplicate-field behaviour described above. In that situation, return the keys from getPermissions() and accept your own tab instead.
A Node permission is stored as -1 and modPermission() returns TRUE, not an array
You save a Node field with the "all" checkbox ticked, then read it back and find -1 rather than 0, or TRUE rather than a list of ids.
The controller records which fields were of type Node, and on save rewrites a submitted 0 (the zeroVal "all" checkbox) to -1, with the comment "so mod permissions can merge properly". On the next render it translates -1 back to 0. The -1 is what makes Member::modPermissions() group merging treat "all" as beating any explicit list of node ids.
At runtime a node permission has three possible shapes, and every core consumer tests all three:
TRUE — the moderator's perms is '*', so modPermission() short-circuits before it looks at your key at all -1 — restricted, but explicitly granted all nodes an array of node ids — restricted to those containers $allowed = $member->modPermission( $containerClass::$modPerm ); if ( $allowed === TRUE or $allowed === -1 ) { /* everything */ } elseif ( is_array( $allowed ) and in_array( $container->_id, $allowed ) ) { /* this container only */ } Checking only is_array() is the usual mistake and it locks out every unrestricted moderator.
A new extension does not show up until the datastore is cleared, and the tab order is not what you asked for
You add the extension to extensions.json by hand and it does not appear; or your tab lands in a position you did not expect.
Application::allExtensions() caches the resolved class list in Store::i()->extensions, keyed by extension name only. Installing or uninstalling an application through the ACP calls Store::i()->clearAll(), so a normal install is fine, but editing data/extensions.json directly in a dev install is not — the Developer Center's extension controller is what calls unset( Store::i()->extensions ). Clear the datastore after a manual edit.
The same cache is why ordering is unreliable. allExtensions() accepts a $firstApp argument, and the moderator controller passes 'core':
foreach ( Application::allExtensions( 'core', 'ModeratorPermissions', FALSE, 'core' ) as $k => $ext ) but that sort only happens on the branch that builds the cache, and by that point in edit() the cache has already been built by an earlier call on line 330 that passed no $firstApp. The argument is effectively a no-op here — which is exactly why the very next lines re-sort core_General to the front in PHP:
if ( isset( $extensions['core_General'] ) ) { $meFirst = array( 'core_General' => $extensions['core_General'] ); unset( $extensions['core_General'] ); $extensions = $meFirst + $extensions; } There is no supported way for a third-party extension to control its tab position. Everything after the General tab follows application load order.
Verified against
Invision Community 5.0.19, read from source: system/Extensions/ModeratorPermissionsAbstract.php, applications/core/modules/admin/staff/moderators.php, system/Application/Application.php (allExtensions(), extensions(), constructExtensionClass()), system/Member/Member.php (modPermissions(), modPermission()), system/Content/ExtensionGenerator.php, system/Helpers/Form/Form.php, applications/core/dev/html/admin/members/moderatorPermissions.phtml, and every implementation present under applications/*/extensions/core/ModeratorPermissions/ (core: Clubs, Content, ContentGenerator, General, Members, Warnings; forums: Topic; blog: Blogs; cms: Databases; downloads: Files).
Related: The core/Permissions extension and node permission integration in Invision Community 5, The core/ModCp extension in Invision Community 5, Extension method signatures are fixed, and a mismatch is a fatal on install

0 comments
37 views

Ernest Defoe

July 26Jul 26

The core/OutputPlugins extension: how a {tag} in a template is resolved

The core/OutputPlugins extension registers a new {tag="value"} that can be used in theme templates, CSS, email templates, Pages blocks and Pages content. It is unlike almost every other extension in Invision Community 5 in one crucial respect: it does not run when the page is rendered. It runs when the template is compiled, and what it returns is not output — it is a fragment of PHP source code that gets written into the compiled template function and later eval()'d.
The single consumer is IPS\Theme\Theme::_compileTemplate() in system/Theme/Theme.php (the callback starts at line 4263), inside a preg_replace_callback over the pattern /\{([a-z]+?=(['"]).+?\2 ?+)}/. Note that Application::allExtensions( 'core', 'OutputPlugins' ) is never called anywhere in the suite — OutputPlugins is one of the few extension types that is never enumerated. It is looked up by name, one class at a time, via Application::getExtensionClass( $app, 'OutputPlugins', IPS::mb_ucfirst( $plugin ) ). Consequently your extension class is never instantiated, never access-checked, and a generate() method on it will be ignored.
The contract
namespace IPS\Extensions; abstract class OutputPluginsAbstract { /** * @brief Can be used when compiling CSS */ public static bool $canBeUsedInCss = FALSE; /** * @param string $data The initial data from the tag * @param array $options Array of options * @return array|string string with code to eval, or * array( 'pre' => ..., 'return' => ... ) */ abstract public static function runPlugin( string $data, array $options ): string|array; } What core actually does with the result, verbatim from _compileTemplate():
$code = $pluginClass::runPlugin( $value, $options, $functionName, $calledClass ); if( !is_array( $code ) ) { $code = array( 'return' => $code ); } if( !isset( $code['pre'] ) ) { $code['pre'] = ''; } if( !isset( $code['post'] ) ) { $code['post'] = ''; } and then it emits, into the compiled function:
IPSCONTENT; // closes the heredoc holding the literal template text {$code['pre']} $return .= {$code['return']}; // only if $code['return'] is truthy {$code['post']} $return .= <<<IPSCONTENT // reopens the heredoc KeyMust beNotes prePHP statementsOptional. Defaulted to '' by core. Emitted verbatim before the assignment. May legally open a block (IPS\Output\Plugin\Template emits if ( ... ): here). returnA PHP expressionNot defaulted by core. Concatenated as $return .= <expr>;. Emitted only if truthy, so returning '' is the supported way to make a tag produce nothing. postPHP statementsOptional, defaulted to ''. Undocumented in the abstract's docblock but fully supported — Template uses it to emit endif;. $data is the value of the first key="value" pair in the tag; $options is every remaining pair, keyed by attribute name. Both are raw template text, never sanitised, and end up inside PHP source that you generate.
A minimal example
File: applications/myapp/extensions/core/OutputPlugins/Mytag.php. The filename determines the tag: Mytag.php → {mytag="..."}.
<?php namespace IPS\myapp\extensions\core\OutputPlugins; use IPS\Extensions\OutputPluginsAbstract; use function defined; if ( !defined( '\IPS\SUITE_UNIQUE_KEY' ) ) { header( ( $_SERVER['SERVER_PROTOCOL'] ?? 'HTTP/1.0' ) . ' 403 Forbidden' ); exit; } class Mytag extends OutputPluginsAbstract { /** * @brief Can be used when compiling CSS */ public static bool $canBeUsedInCss = FALSE; /** * Run the plug-in * * Extra parameters MUST have defaults - core passes four arguments, * but the abstract only declares two. */ public static function runPlugin( string $data, array $options, ?string $functionName=NULL, string $calledClass='IPS\Theme' ): string|array { /* Core's convention: if the value starts with $ or \ it is PHP, otherwise it is a literal that must be quoted and escaped. */ if ( mb_substr( $data, 0, 1 ) === '$' or mb_substr( $data, 0, 1 ) === '\\' ) { $argument = $data; } else { $argument = "'" . addcslashes( $data, "'\\" ) . "'"; } $mode = isset( $options['mode'] ) ? "'" . addcslashes( $options['mode'], "'\\" ) . "'" : 'NULL'; /* Escape at RUNTIME, in the generated code - core will not escape for you */ return array( 'pre' => '', 'return' => "\\IPS\\Theme\\Template::htmlspecialchars( \\IPS\\myapp\\Formatter::forTemplate( {$argument}, {$mode} ), ENT_QUOTES | ENT_DISALLOWED, 'UTF-8', FALSE )", 'post' => '', ); } } Used in a template as {mytag="hello" mode="short"} or {mytag="$item->title"}. After creating the file you must regenerate applications/myapp/data/extensions.json (ACP → Developer Centre → Extensions writes it via Application::buildExtensionsJson()) and rebuild templates, or nothing will happen at all.
The tag renders as literal text on the page — no error, no log entry
You see {mytag="hello"} printed verbatim in the HTML. This is core's designed behaviour for an unresolvable plugin:
/* Still doesn't exist? */ if( ! class_exists( $pluginClass ) ) { return $matches[0]; } There is no exception, no warning and no log. Every failure mode of name resolution produces exactly this symptom, so check all of them:
The tag name in the template must be all lowercase letters. The outer regex is [a-z]+? — no digits, no underscores, no capitals. {myTag=...}, {my_tag=...} and {tag2=...} are not recognised as tags at all and are left alone by the compiler. The class name must be exactly ucfirst() of that lowercase name. Core computes IPS::mb_ucfirst( $plugin ), which only uppercases the first character (init.php line 1393). The result is used as a case-sensitive array key into extensions.json. A class called MyTag registers the key MyTag, the lookup asks for Mytag, and PHP array keys are case-sensitive, so it never matches. This is not hypothetical: core's own applications/core/extensions/core/OutputPlugins/GraphQl.php registers the key GraphQl and is therefore unreachable from any template tag in 5.0.19. data/extensions.json must contain the entry. getExtensionClass() reads nothing but that file; it does not scan the extensions directory. A newly added file that has not been through the Developer Centre does not exist as far as the template compiler is concerned. Your application must be enabled. getExtensionClass() throws OutOfRangeException if Application::appIsEnabled() is false, and separately if RECOVERY_MODE is on and the app is not one of IPS::$ipsApps. appIsEnabled() additionally returns FALSE for every app while the setup dispatcher is running an install. Because resolution happens at compile time, this bites asymmetrically: templates built while the app was enabled keep working after you disable it, and templates rebuilt while it is disabled bake the literal tag text into the stored template permanently — until they are rebuilt again.
The tag produces nothing at all, silently, when you return an array
You return array( 'pre' => '$x = something();' ) and nothing is output. Core defaults pre and post when they are missing but reads return without an isset():
if ( $code['return'] ) { $return .= <<<PHP $return .= {$code['return']}; PHP; } PHP 8 raises Warning: Undefined array key "return" — and IPS::errorHandler() in init.php explicitly discards it:
/* We don't care about these in production */ if ( in_array( $errno, array( E_WARNING, E_NOTICE, E_STRICT, E_DEPRECATED ) ) ) { return; } So the warning never reaches the screen or the system log. The expression evaluates to NULL, the if is skipped, and your tag compiles to a pre block with no output. Always set 'return', using '' when you genuinely want no output (that is what IPS\core\extensions\core\OutputPlugins\GraphQl does when the query returns no data).
"syntax error, unexpected ..." in eval()'d code, in a template you did not touch
Your return value is spliced into generated PHP as $return .= <your string>;. It must be a valid PHP expression, not a statement, and it must be balanced. Anything you get wrong becomes a parse error inside Theme::runProcessFunction()'s eval(), which reports a file/line of .../system/Theme/Theme.php(3981) : eval()'d code — never your extension file, and usually while a completely different template is being built.
The most common cause is naive quoting of $data. Core does this too and it is genuinely fragile — IPS\cms\extensions\core\OutputPlugins\Media and Pageurl both end with:
return "'" . $url . "'"; A single apostrophe or backslash anywhere in that value terminates the generated string literal early and the template stops compiling. Always run literals through addcslashes( $value, "'\\" ) (or var_export( $value, TRUE )) before wrapping them in quotes.
Two further points on the emitted expression:
A trailing semicolon in your returned string is harmless — you get expr;;, which parses. IPS\cms\...\Widget and IPS\core\...\Backgroundimage both ship one. The declared return type is string|array. Returning an int would work at the source-splicing level but violates the abstract's return type and throws a TypeError. (IPS\Output\Plugin\Theme returns the integer 100 for logo_height, but that class does not extend the abstract, so it is not bound by it.) "Declaration of ...::runPlugin() must be compatible with ...OutputPluginsAbstract::runPlugin()"
The abstract declares two parameters. Core calls the method with four:
$code = $pluginClass::runPlugin( $value, $options, $functionName, $calledClass ); $functionName is the compiled function's name and may be NULL (compileTemplate() types it ?string). $calledClass is get_called_class() from the theme class, normally IPS\Theme. If you want either value you must declare the extra parameters as optional, exactly as IPS\Output\Plugin\Template does:
public static function runPlugin( string $data, array $options, ?string $functionName=NULL, string $calledClass='IPS\Theme' ): array|string Declaring them without defaults is an LSP violation and produces the fatal above. Declaring a required third parameter without extending the abstract at all is worse but legal — IPS\Output\Plugin\Resource::runPlugin( string $data, array $options, string $context ) does exactly that and will throw a TypeError if $functionName is NULL.
The fatal surfaces when PHP loads your class, which happens inside class_exists( $pluginClass ) during template compilation — i.e. on whichever page request or theme build first compiles a template containing your tag, not anywhere near your own code.
An option is missing from $options even though you wrote it in the tag
Option parsing is a second regex run over the inside of the tag:
preg_match_all( '/(.+?)='.$matches[2].'([^' . $matches[2] . ']*)'.$matches[2].'\s?/', $matches[1], $submatches ); where $matches[2] is the quote character used by the first pair. Three consequences that are easy to hit and impossible to diagnose from the symptom:
Only one whitespace character is consumed between pairs (\s?). Write {mytag="x" mode="short"} with two spaces and the captured key is " mode", with a leading space. isset( $options['mode'] ) is then FALSE and your option is silently ignored. Every pair must use the same quote character as the first pair. Mixing {mytag='x' mode="short"} breaks the split. An option value cannot contain the quote character at all — the value class is [^"]*. There is no escape mechanism; use the opposite quote character inside the value. Also note the outer regex has no /s modifier, so . does not match a newline: a tag split across two lines in a template is not recognised as a tag and is emitted literally.
Finally, $options contains only what was written. Always guard with isset(); core does not always bother, which is why IPS\Output\Plugin\Truncate opens with an unguarded if( !$options['length'] ).
Your plugin's output is not HTML-escaped
Ordinary template variables are rewritten by the compiler into Template::htmlspecialchars( $var, ENT_QUOTES | ENT_DISALLOWED, 'UTF-8', FALSE ). Plugin output is not. Your expression is emitted outside the heredoc as a bare $return .= <expr>;, so whatever it evaluates to is concatenated raw.
This is intentional — it is how {template=...} and {lang=...} can emit markup — but it means any plugin that surfaces member-supplied data is an XSS hole unless you emit the escaping call. Follow IPS\Output\Plugin\Expression, which escapes by default and only skips it when the tag passes a truthy raw option, or IPS\Output\Plugin\Setting, which escapes only when the tag sets an escape option. Escape in the generated code, not in runPlugin(), because runPlugin() only sees the template's literal text, not the runtime value.
"InvalidArgumentException: invalid_plugin: ..." — and it takes the front end down
CSS is compiled through the same function with $isCSS = TRUE, and plugins must opt in:
if ( $isCSS AND $pluginClass::$canBeUsedInCss !== TRUE ) { throw new InvalidArgumentException( 'invalid_plugin:' . $functionName . ' - ' . $plugin ); } The default in OutputPluginsAbstract is FALSE, so a new extension is CSS-forbidden until you set public static bool $canBeUsedInCss = TRUE;. Note the comparison is !== TRUE, a strict identity check — a truthy non-boolean will not satisfy it.
This exception is not caught anywhere. Neither Theme::writeCss() nor Theme::compileCustomCss() wraps the call. And compileCustomCss() is reached from Theme::getCustomCssForOutput() during ordinary front-end rendering, so a forbidden tag typed into a theme's Custom CSS box throws on the front end, for every visitor, not in the ACP editor where it was typed. Worse, IPS::exceptionHandler() only shows the exception message when Dispatcher::i()->controllerLocation == 'admin', so front-end visitors get a generic error page and a string like invalid_plugin:css_core_front_custom_custom_css - money is only visible in ACP → System Logs under uncaught_exception.
The language string core_theme_invalid_plugin ("You cannot use the template plug-in '{%s}' here.") exists in applications/core/data/lang.xml but is not referenced by any PHP in 5.0.19 — the exception message is the raw invalid_plugin: string, not a language key, so do not go looking for the friendly wording.
One related fatal: nothing in the consumer checks instanceof OutputPluginsAbstract. If you write a plugin class that does not extend the abstract and does not declare $canBeUsedInCss yourself, the line above raises Error: Access to undeclared static property instead of the exception. IPS\Output\Plugin\Address is in exactly that state in core.
Editing runPlugin() changes nothing on the site
Compiled templates are cached, not recompiled per request. Theme::compileTemplates() writes a whole compiled class per template group into the datastore under a key like template_<themeId>_<hash>_<group>; Theme::writeCss() writes CSS files to disk; Pages blocks, Pages content, custom fields and email templates each cache their compiled function in Store::i()->$functionName. Theme::runProcessFunction() additionally short-circuits entirely if function_exists() already returns true for the function name. First-party application templates skip the datastore entirely outside IN_DEV and are require_once'd from /static/templates/<app>_<location>_<group>.php, which is only regenerated by Theme::compileStatic().
So a change to runPlugin() has no visible effect until the relevant caches are cleared and templates/CSS are rebuilt. It also means every value your plugin computes at compile time is frozen and shared by every visitor and every theme that reuses that compiled function. Never return a literal that depends on the current member, the current request, permissions or the current language: return an expression that computes it at runtime instead. Core is not consistent about this — cms\...\Media and cms\...\Pageurl bake a URL string at compile time, and core\...\GraphQl actually executes a GraphQL query during compilation and bakes the JSON result into the template.
A page that has nothing to do with your app throws a generic error
The call to your extension is completely unguarded:
$code = $pluginClass::runPlugin( $value, $options, $functionName, $calledClass ); No try/catch, no type check on what comes back beyond is_array(). Any exception, TypeError or fatal in your runPlugin() aborts whatever was compiling the template.
This is the failure that surfaces furthest from its cause, because template compilation is lazy and happens during ordinary page rendering. Theme::getTemplate() (system/Theme/Theme.php, line 871) checks whether the datastore key template_<themeId>_<hash>_<group> exists and, if it does not, calls $this->compileTemplates( $app, $location, $group ) inline, on that request. So the visitor who happens to be first to hit a template group containing your tag after a cache clear is the one who gets the error page — on someone else's app's page, with a stack trace that points at IPS\Theme\Theme and never mentions your extension.
The other compilation entry points that your plugin can break are: ACP → Customisation → Themes → build (Theme::compileTemplates(), compileCss()), the "rebuild static" tool (Theme::compileStatic(), first-party apps only), Application::build() in developer mode, and the per-item compiles for Pages blocks (IPS\cms\Blocks\Block), Pages content (IPS\cms\Pages\Page), custom field display templates (IPS\core\ProfileFields\Field, IPS\cms\Fields, IPS\downloads\Field) and email templates (IPS\Email).
Because IPS::exceptionHandler() only reveals the exception message when Dispatcher::i()->controllerLocation == 'admin', the actual message is visible only in ACP → System Logs under uncaught_exception. Wrap everything in runPlugin() defensively and return a safe expression (or '') on failure — this is exactly what cms\...\Media does when Media::load() throws OutOfRangeException.
Another application's plugin wins your tag name
Resolution order is fixed and there is no namespacing:
IPS\Output\Plugin\<Ucfirst> is tried first via class_exists(). Any of the 23 built-in plugins in system/Output/Plugin/ — address, advertisement, customtemplate, datetime, expression, file, filesize, fontsize, hextorgb, insert, lang, member, number, prefix, request, resource, setting, striptags, template, theme, truncate, url, wordbreak — always beats an application extension of the same name. Otherwise every application is walked in Application::applications() order, which is core_applications sorted by app_position, and the loop breaks on the first app that declares the key. So two apps declaring the same tag name silently resolve to whichever is positioned first in the ACP application list, and a site administrator reordering applications changes which one wins on the next template rebuild. Because the tag namespace is global across every template, every CSS file and every Pages block in the suite, choose a distinctive name — and remember that any pre-existing literal text matching {yourname="..."} anywhere in the suite will now be handed to your plugin.
preSaveProcess() and postSaveProcess() are never called
If you copy applications/cms/extensions/core/OutputPlugins/Database.php as a starting point you will inherit two extra static methods, preSaveProcess( string $data, array $options, object $page ) and postSaveProcess( ... ), complete with docblocks describing them as running before and after a Pages page is saved. Grepping the entire 5.0.19 tree for either name returns hits in that one file and nowhere else. They are dead code; nothing invokes them, and the LogicException with cms_err_db_already_on_page that preSaveProcess() throws can never fire. Do not build behaviour on them.
Verified against
Read from an Invision Community 5.0.19 source tree (applications/core/data/versions.json long version 5001908). Primary files: system/Extensions/OutputPluginsAbstract.php; the consumer IPS\Theme\Theme::_compileTemplate() (line 4199) plus compileTemplate(), getTemplate(), makeProcessFunction(), runProcessFunction(), writeCss(), compileCustomCss(), compileStatic() and compileTemplates() in system/Theme/Theme.php; Application::getExtensionClass(), appIsEnabled(), applications(), build() and buildExtensionsJson() in system/Application/Application.php; IPS::mb_ucfirst(), IPS::errorHandler() and IPS::exceptionHandler() in init.php; all eight shipped extensions under applications/{cms,core,nexus}/extensions/core/OutputPlugins/; and the 23 built-in plugins in system/Output/Plugin/.
Related: Pages page content is not a template: why template syntax prints itself, and why removing an element does not stick, Theme hook types are an enum of four, there is no "replace", and how to change the page title anyway, Extension method signatures are fixed, and a mismatch is a fatal on install

0 comments
37 views

Ernest Defoe

July 26Jul 26

The core/Queue extension in Invision Community 5

A core/Queue extension is a resumable batch job. You register it in your app's data/extensions.json, hand it work with IPS\Task::queue(), and core writes one row into core_queue. The queue task (IPS\core\tasks\queue) then calls IPS\Task::runQueue() in a loop until it runs out of time, memory, or rows — each call picks one row, instantiates your class, and calls run() exactly once.
There are only three consumers in the whole suite, and it is worth knowing all three because they behave differently:
IPS\Task::queue() — calls preQueueData(), does duplicate-checking, inserts the row. IPS\Task::runQueue() — calls run(), then postComplete(). Called from applications/core/tasks/queue.php and from the ACP "Run background processes now" screen (applications/core/modules/admin/system/background.php). IPS\core\extensions\core\Dashboard\BackgroundQueue::getBlock() — calls getProgress(). This is the only caller of getProgress(). The contract
IPS\Extensions\QueueAbstract declares three abstract methods and one optional one.
abstract public function preQueueData( array $data ): ?array; // Called ONCE by Task::queue(), before the row is inserted. // Return the array to store as the row's `data` column (JSON-encoded). // Return NULL to abort — nothing is queued, postComplete( [], FALSE ) is called. // Throwing \OutOfRangeException is caught by Task::queue() and treated as NULL. abstract public function run( array &$data, int $offset ): int; // Called once per cycle. $data is BY REFERENCE and is re-saved to the row // if json_encode( $data ) differs from what was read. // Return the new offset (int). It is written to core_queue.offset. // Throw \IPS\Task\Queue\OutOfRangeException to signal "finished". abstract public function getProgress( array $data, int $offset ): array; // Called ONLY by the ACP Dashboard "Background Processes" block. // Must return [ 'text' => string, 'complete' => float|int|null ]. // Throw \OutOfRangeException to have the row omitted from the block. public function postComplete( array $data, bool $processed = TRUE ) : void // Optional. Called after the core_queue row is DELETED. // $data is NOT what you think it is — see the trap below. public int $rebuild = REBUILD_NORMAL; // Batch size hint. Nothing in core reads this property; only your own // run() does. REBUILD_INTENSE=1, REBUILD_SLOW=50, REBUILD_NORMAL=250, // REBUILD_QUICK=500 (defined in init.php). protected function getCountDataFromClass( string $classname ) : array // Helper. Returns [ 'count' => ..., 'realCount' => ... ] for an // ActiveRecord class. Throws GLOBAL \OutOfRangeException on failure. Enqueue with:
IPS\Task::queue( string $app, string $key, mixed $data = NULL, int $priority = 5, mixed $checkForDuplicationKeys = NULL ) : void $key is the key you used in extensions.json, not the class name. priority is a TINYINT UNSIGNED column commented "Values 1 to 5 are allowed, 1 being highest priority."
A minimal example
applications/myapp/data/extensions.json:
{ "core": { "Queue": { "PurgeThings": "IPS\\myapp\\extensions\\core\\Queue\\PurgeThings" } } } applications/myapp/extensions/core/Queue/PurgeThings.php:
<?php namespace IPS\myapp\extensions\core\Queue; use IPS\Db; use IPS\Extensions\QueueAbstract; use IPS\Member; use IPS\Task\Queue\OutOfRangeException as QueueOutOfRangeException; use OutOfRangeException; use function defined; use const IPS\REBUILD_SLOW; if ( !defined( '\IPS\SUITE_UNIQUE_KEY' ) ) { header( ( $_SERVER['SERVER_PROTOCOL'] ?? 'HTTP/1.0' ) . ' 403 Forbidden' ); exit; } class PurgeThings extends QueueAbstract { public int $rebuild = REBUILD_SLOW; public function preQueueData( array $data ): ?array { $data['count'] = (int) Db::i()->select( 'COUNT(*)', 'myapp_things' )->first(); /* Nothing to do — do not create a row at all */ if ( !$data['count'] ) { return null; } return $data; } public function run( array &$data, int $offset ): int { $rows = iterator_to_array( Db::i()->select( '*', 'myapp_things', null, 'thing_id ASC', array( $offset, $this->rebuild ) ) ); if ( !count( $rows ) ) { throw new QueueOutOfRangeException; } foreach ( $rows as $row ) { /* ... do the work. Must tolerate being run twice on the same row. ... */ } $offset += count( $rows ); if ( $offset >= $data['count'] ) { throw new QueueOutOfRangeException; } return $offset; } public function getProgress( array $data, int $offset ): array { return array( 'text' => Member::loggedIn()->language()->addToStack( 'myapp_purging_things' ), 'complete' => $data['count'] ? round( 100 / $data['count'] * $offset, 2 ) : 100 ); } public function postComplete( array $data, bool $processed = TRUE ) : void { /* $data is the core_queue ROW, not your array. Decode it yourself. */ if ( !isset( $data['data'] ) ) { return; } $queueData = json_decode( $data['data'], TRUE ); /* ... teardown ... */ } } Queue it:
IPS\Task::queue( 'myapp', 'PurgeThings', array( 'reason' => 'manual' ), 4 ); "Undefined array key" or json_decode() errors inside postComplete()
The docblock on QueueAbstract::postComplete() says "@param array $data Data returned from preQueueData". That is wrong for the normal completion path. In Task::runQueue() the call is:
catch( Task\Queue\OutOfRangeException $e ) { Db::i()->delete( 'core_queue', array( 'id=?', $queueData['id'] ) ); if ( isset( $class ) AND method_exists( $class, 'postComplete' ) ) { $class->postComplete( $queueData, TRUE ); } } $queueData is the raw core_queue row: id, data, offset, date, app, key, priority, plus a synthetic _originalOffset. Your array is the JSON string in $data['data']. Every core implementation that needs it does the decode by hand — IPS\core\extensions\core\Queue\InstallLanguage, PruneLargeTable, UpdateTaggedItems, ConvertOpenTags and IPS\nexus\extensions\core\Queue\DeletePaymentMethod all open with some form of $data = json_decode( $data['data'], TRUE );.
There is a second shape. When preQueueData() returns NULL, Task::queue() calls $class->postComplete( [], FALSE ) — an empty array, so $data['data'] does not exist at all. Core's own FileCleanup extension guards against exactly this:
public function postComplete( array $data, bool $processed = TRUE ) : void { if( !isset( $data['data'] ) ) { return; } $data = json_decode( $data['data'], TRUE ); ... } Copy that guard. Note also that postComplete() runs after the row is deleted — RebuildAchievements carries the comment "The core_queue row is deleted before this is run" and counts remaining rows accordingly. And postComplete() is never called at all if the app is uninstalled: Application::delete() does a bare Db::i()->delete( 'core_queue', array( 'app=?', $this->directory ) ).
RuntimeException with an empty message, and the job never finishes
This is the big one. Two classes named OutOfRangeException are in play, and IPS\Task\Queue\OutOfRangeException extends \OutOfRangeException, so the catch order in Task::runQueue() decides everything:
/* This means the task is done */ catch( Task\Queue\OutOfRangeException $e ) { Db::i()->delete( 'core_queue', array( 'id=?', $queueData['id'] ) ); ... } /* Catch any OORE in the task and transform the exception so that we can display an error */ catch( OutOfRangeException $e ) { Log::log( $e, 'queue_oore'); throw new RuntimeException( $e->getMessage() ); } Only IPS\Task\Queue\OutOfRangeException means "complete". A plain global \OutOfRangeException is treated as a fatal error: the row is not deleted, the offset is not updated, and a RuntimeException is thrown out of runQueue(). The next cycle picks the same row and does the same thing forever.
You almost never throw a global \OutOfRangeException deliberately — you inherit one. ActiveRecord::load() throws it when a record is missing, so a single deleted member or content item inside your run() loop poisons the whole queue. This is why IPS\core\extensions\core\Queue\Follow wraps its load and converts:
try { $item = $classname::load( $data['item'] ); } catch( OutOfRangeException $e ) { /* Item no longer exists, so we're done here. */ throw new \IPS\Task\Queue\OutOfRangeException; } The error surfaces nowhere near where you caused it. IPS exceptions are usually constructed with no message, so $e->getMessage() is '' and you get a RuntimeException with a blank message and a stack trace rooted in IPS\Task::runQueue(), containing no reference to your app. The only record of the real exception is the Log::log( $e, 'queue_oore') line above it — look in the ACP System Logs (core_log) filtered to category queue_oore. That entry has the original class, file and line.
The safest habit: alias the import so the two can never be confused, as UpdateWidgetAreas does — use IPS\Task\Queue\OutOfRangeException as QueueOutOfRangeException; — and wrap any ::load() call in your run().
ACP notification "Locked Task: queue", and background processing stalls for 15 minutes
This is the downstream effect of the previous trap, and it is what an admin actually reports to you.
Task::run() sets running = 1 in core_tasks before calling execute(), and only clears it afterwards:
$output = $this->execute(); $this->running = 0; $this->lock_count = 0; ... $this->save(); If anything escapes execute(), those lines never run. Task::runAndLog() only catches IPS\Task\Exception, and queue.php's callback only catches UnderflowException, so a RuntimeException from your run() escapes both. The queue task stays locked until Task::queued() sees it is more than 15 minutes stale ($fifteenMinutesAgo = ( time() - 900 )), unlocks it, and increments lock_count. On the third unlock, Task::unlock() fires AdminNotification::send( 'core', 'ConfigurationError', "taskLock-{$this->id}" ), which renders using the language string dashboard_tasks_broken — "Locked Task: %s", i.e. "Locked Task: queue".
It gets worse under the traditional (non-cron) task method. IPS\Dispatcher\Standard::__destruct() runs tasks on page loads inside:
try { $task = Task::queued(); if ( $task ) { $task->runAndLog(); } } catch( Exception $e ) { } That is a completely silent swallow — no log row, no output. The only symptom is the locked-task notification and a queue that stops moving. If the site uses the real cron entry point (applications/core/interface/task/task.php) you at least get an uncaught_exception log row — but note that handler catches Exception, not Throwable, so a PHP Error (a TypeError, a call to an undefined method) escapes even that and appears as a raw fatal in cron output.
The ACP Dashboard is blank or 500s after installing my app
The Background Processes dashboard block iterates up to 100 core_queue rows from every enabled app and calls each one's getProgress():
$class = new $extensions[ $queueData['key'] ]; $rows[] = $class->getProgress( json_decode( $queueData['data'], TRUE ), $queueData['offset'] ); } catch ( OutOfRangeException $e ) { } Only OutOfRangeException is caught. Anything else — a TypeError, an IPS\Db\Exception, an undefined index promoted to an error, a Theme failure — propagates out of getProgress(), out of getBlock(), and into IPS\core\modules\admin\overview\dashboard::manage(), where $blocks[ $key ] = $extension->getBlock(); has no try/catch at all.
Your app's bug takes down the shared AdminCP Dashboard for every administrator, and the error names IPS\core\modules\admin\overview\dashboard, not your app. core_BackgroundQueue is one of the default main blocks for every admin, so this is not a corner case. Treat getProgress() as hostile-input code: it receives whatever JSON is in the row, including rows written by an older version of your app.
The idiom core uses is to throw a global \OutOfRangeException to bail out safely — note this is the opposite convention from run():
public function getProgress( array $data, int $offset ): array { if ( !isset( $data['count'] ) ) { throw new OutOfRangeException; /* GLOBAL one — swallowed by the block */ } ... } Progress bar is stuck at "indeterminate", or the block shows an undefined-key warning
The backgroundQueue template reads exactly two keys and nothing else:
{{if $row['complete'] === NULL OR $row['complete'] > 100}} ... indeterminate bar, {lang="progress_bar_percent_not_available"} ... {{else}} <progress ... value="{expression="number_format( $row['complete'], 2)" }" max="100"> {{endif}} ... <strong>{$row['text']|raw}</strong> Omit complete and you get an undefined-array-key warning plus a permanently animated indeterminate bar — which reads as "stuck" to an admin. Return a value over 100 (easy if you use a count captured at queue time and rows were added since) and you get the same indeterminate bar. Core's RebuildPosts deliberately returns 'complete' => null for very large tables rather than a wrong number.
Also note {$row['text']|raw}: the text is printed unescaped. If you interpolate a user-supplied title into it, you have stored XSS in the AdminCP. Core passes user content through htmlsprintf and a template — see Follow::getProgress(), which builds its label with Theme::i()->getTemplate( 'global', 'core', 'global' )->basicUrl(...) rather than string concatenation.
Task::queue() throws InvalidArgumentException with no message
Task::queue() resolves your class through Application::extensions( 'core', 'Queue', FALSE ), which reads data/extensions.json and applies this filter:
if( !is_string( $classname ) or !class_exists( $classname ) ) { /* Switching between branches confuses extensions */ continue; } A class that does not exist is silently dropped from the list. Then:
if ( !isset( $extensions[ $key ] ) ) { throw new InvalidArgumentException; } No message, no key name, no app name. The three causes, in order of likelihood: the key is missing from applications/<app>/data/extensions.json; the fully-qualified class name in the JSON does not match the file's namespace and class (remember the JSON needs doubled backslashes); or the file is not on disk in the built package. The same InvalidArgumentException is thrown if Application::load( $app ) fails, so a typo'd app directory looks identical.
Two further consequences of that construct=FALSE call. First, core does new $extensions[ $key ] with no constructor arguments — unlike most extension types, a Queue extension must be constructible with zero args. Second, if the extension key disappears while rows still exist (you renamed it, or shipped a version that removed the class), runQueue() hits:
if ( !isset( $extensions[ $queueData['key'] ] ) ) { throw new Task\Queue\OutOfRangeException; } …which lands in the "completed successfully" catch. The row is deleted and postComplete() is skipped — that is what the isset( $class ) guard in that catch block is for. Renaming a Queue key silently discards every pending job under the old name.
Task::queue() returns without queueing anything
Task::queue() has two silent no-op paths, both before the INSERT:
try { $data = $class->preQueueData( $data ); } catch( OutOfRangeException $e ) { $data = NULL; } if ( $data === NULL ) { $class->postComplete( [], FALSE ); return; } Returning NULL from preQueueData() is the intended "there is nothing to do" signal and core uses it heavily (RebuildPosts and RebuildItems both return null when $data['count'] == 0; MemberContent returns NULL when the app is disabled or the class lacks the required feature).
The path that catches people is the catch. Any global \OutOfRangeException raised anywhere inside preQueueData() — including from the inherited helper getCountDataFromClass(), which converts any DB failure into throw new OutOfRangeException — is swallowed and turned into "nothing to queue". No log entry is written. If your job mysteriously never appears in core_queue, put a Log::debug() inside preQueueData() before assuming Task::queue() was never reached.
Also: preQueueData() is typed array $data, but Task::queue()'s third parameter defaults to NULL. Calling Task::queue( 'myapp', 'PurgeThings' ) with no data is a TypeError, not a no-op. Core always passes at least array() — e.g. Task::queue( 'core', 'RecountMemberContent', array(), 4 ).
Returning NULL from run() to signal completion no longer works
runQueue() still contains this branch:
if ( is_null( $newOffset ) ) { Log::log( $queueData['key'] . " returned a NULL offset - tasks should throw \\IPS\\Task\\Queue\\OutOfRangeException when they are finished", 'runQueue_log' ); Db::i()->delete( 'core_queue', array( 'id=?', $queueData['id'] ) ); ... } It is dead code in IC5. QueueAbstract::run() declares a return type of : int, and PHP return types are covariant — a subclass cannot widen it to ?int. Returning null produces Return value must be of type int, null returned from inside your own class, which then escapes as an uncaught TypeError (see the "Locked Task" trap above). Throw IPS\Task\Queue\OutOfRangeException; there is no other completion signal.
Related: the log category runQueue_log, which also carries the useful "returned a new offset of N" debug lines, is listed in Log::$excludeFromSettings. Those lines are written only when the \IPS\DEBUG_LOG constant is enabled in constants.php. If you are debugging a queue extension, turn it on first.
My changes to $data disappear when the job finishes
$data is passed by reference and core persists it — but only on the success branch:
$newOffset = $class->run( $json, $queueData['offset'] ); $queueData['offset'] = $newOffset; ... else { Db::i()->update( 'core_queue', array( 'offset' => $newOffset ), array( 'id=?', $queueData['id'] ) ); $newData = json_encode( $json ); /* Did it change?? */ if ( $newData !== $queueData['data'] ) { $queueData['data'] = $newData; Db::i()->update( 'core_queue', array( 'data' => $newData ), array( 'id=?', $queueData['id'] ) ); } } That entire block is skipped when you throw IPS\Task\Queue\OutOfRangeException. Anything you wrote into $data during the final cycle is lost, and postComplete() receives the row as it was before that cycle. If postComplete() needs a result accumulated by run() — a list of failures, a count — write it during a cycle that returns an offset normally, and only throw on the following, empty cycle. IPS\nexus\extensions\core\Queue\DeletePaymentMethod relies on exactly this, reading $_data['failures'] in postComplete().
The comparison is a string comparison against the row's original JSON, so a change that round-trips to identical JSON is not re-saved — harmless, but it means you cannot use a no-op write to force a row touch.
A queued job restarts from offset 0 and its cleanup never runs
$checkForDuplicationKeys does not prevent a duplicate. It deletes the existing row:
if ( $got === count( $checkForDuplicationKeys ) ) { /* Ok, so we have a duplicate queue item, lets remove it so the new one which is set with the correct count is used and offset is returned to 0 to start over */ Db::i()->delete( 'core_queue', array( 'id=?', $row['id'] ) ); } It is a raw DELETE. The old job's progress is discarded no matter how far along it was, and — because this is not the runQueue() completion path — postComplete() is never called for the row that was thrown away. Any state preQueueData() set up for that job (a temp table, a flag setting, an uploaded file) that postComplete() was supposed to tear down is leaked. If your job allocates anything, do not use $checkForDuplicationKeys; query core_queue yourself and decide.
Three more things about the matching, all silent when they go wrong:
The guard is if ( is_array( $checkForDuplicationKeys ) and is_array( $data ) ). Passing a single key as a string — Task::queue( 'myapp', 'Foo', $data, 5, 'thing_id' ) — is ignored entirely, with no error. Always pass array( 'thing_id' ). The keys are checked against $data after preQueueData() has run, and against the stored (already-preQueueData'd) JSON of existing rows. If preQueueData() renames or removes the key, dedup silently stops working. A row matches only if every key is present on both sides and equal (loose ==). Core exploits this by inserting a marker key purely to enable the mechanism — applications/core/modules/admin/overview/files.php queues array( 'delete' => true, ... ) with the comment "We use a key in the data array just to trigger the code that deletes duplicate tasks". Two jobs run out of order, or a job's work is done twice
runQueue() selects one row like this:
$queueData = Db::i()->select( '*', 'core_queue', array( Db::i()->in('app', $enabledApps ) ), 'priority ASC, RAND()', 1, NULL, NULL, Db::SELECT_FROM_WRITE_SERVER )->first(); RAND(). Rows at the same priority are picked in random order every cycle, so two jobs of equal priority interleave arbitrarily and neither is guaranteed to finish first. The only ordering tool you have is the priority integer: all priority-1 rows drain before any priority-2 row runs. If job B depends on job A, either give A a lower number or queue B from A's postComplete() — which is what ConvertOpenTags::postComplete() does when it calls Task::queue( 'core', 'UpdateTaggedItems', ... ).
There is also no row-level lock on core_queue — the SELECT takes no lock and nothing marks the row as in-flight. The only serialisation is the lock on the queue row in core_tasks, and the ACP "Run background processes now" screen has to defend that lock by hand on every redirect cycle:
/* Make sure the task is locked */ $task = Task::load('queue', 'key'); $task->running = TRUE; $task->next_run = time() + 900; $task->save(); If an admin opens that screen while a cron run is in progress, or if a run times out mid-cycle and the row is retried at the same offset, your run() will process the same batch twice. Write run() to be idempotent. Note also Db::SELECT_FROM_WRITE_SERVER on the queue row itself — core does not trust a read replica here, and Follow::run() goes further and temporarily sets Db::i()->readWriteSeparation = FALSE around its work.
The job stops moving while my app is disabled
Both the runner and the dashboard filter by enabled applications — runQueue() with Db::i()->in( 'app', $enabledApps ), the dashboard block with a join on core_applications and app_enabled=1. Rows for a disabled app are invisible to both: they are not lost, but they neither run nor appear anywhere in the ACP.
If yours were the only rows left, the SELECT ... ->first() underflows, and queue.php reacts by turning the shared task off:
catch ( UnderflowException $e ) { $this->enabled = FALSE; $this->save(); return FALSE; } Only two things switch it back on: Task::queue() (its last line is Db::i()->update( 'core_tasks', array( 'enabled' => 1 ), array( '`key`=?', 'queue' ) )) and re-enabling an application, which Application::set__enabled() handles with the comment "Enable queue task in case there are pending items". Uninstalling the app deletes its rows outright.
The "Run background processes now" screen shows a raw class name, not my translated text
getProgress() is not involved in that screen. applications/core/modules/admin/system/background.php builds its own label straight from the row:
$lang = array( $queueData['key'] ); if ( isset( $json['class'] ) ) { $lang[] = $json['class']; } else if ( isset( $json['extension'] ) ) { $lang[] = $json['extension']; } else if ( isset( $json['storageExtension'] ) ) { $lang[] = $json['storageExtension']; } if ( isset( $json['count'] ) ) { $lang[] = " " . $offset . ' / ' . $json['count']; } ... addToStack( 'background_processes_processing', FALSE, array( 'sprintf' => array( implode( ' - ', $lang ) ) ) ) background_processes_processing is "Processing %s", and Lang applies sprintf replacements literally (they are htmlspecialchars'd, never translated). So the admin sees "Processing PurgeThings - IPS\myapp\Thing - 500 / 12000" — your extension key and a raw class name. The only way to influence it is to name your extension key readably and to put class and count keys in your data array. Your translated getProgress() text appears only on the ACP Dashboard block.
The percentage on that screen comes from $data['done'] + ( $queueData['offset'] - $queueData['_originalOffset'] ) against a total that background::getCount() computed by summing realCount, falling back to count, falling back to 1 per row. If your run() returns something that is not a monotonically increasing row offset — RebuildPosts returns $data['indexed'], a running total — the delta is still positive and the bar behaves. Return anything that can decrease and the bar goes backwards.
Verified against
Read from Invision Community 5.0.19 source (applications/core/data/versions.json long version 5001908): system/Extensions/QueueAbstract.php, system/Task/Task.php, system/Task/Queue/OutOfRangeException.php, system/Application/Application.php, system/Dispatcher/Standard.php, system/Lang/Lang.php, system/Log/Log.php, applications/core/tasks/queue.php, applications/core/modules/admin/system/background.php, applications/core/modules/admin/overview/dashboard.php, applications/core/extensions/core/Dashboard/BackgroundQueue.php, applications/core/dev/html/admin/dashboard/backgroundQueue.phtml, and all 97 core/Queue implementations shipped in applications/*/extensions/core/Queue/. Nothing here is inferred from IPS 4.x behaviour.
Related: Your background task is running inside a visitor's page load, Running code on every request, after the page has been sent, A realtime gateway will let any member join any channel unless you sign the channel into the token

0 comments
36 views

Ernest Defoe

July 26Jul 26

The core/FileStorage extension in Invision Community 5

A core/FileStorage extension is how an application tells the suite that it owns a bucket of files. Each extension becomes one row on AdminCP → System → Files → Storage Settings, one entry in the upload_settings setting, and one unit of work for the MoveFiles background task. Core calls it in exactly four situations: when the admin repoints that bucket at a different storage configuration (count() then repeated move()), when the admin runs Find orphaned files (isValidFile() for every file in a configuration), when the owning application is uninstalled (delete()), and — for the string form of a storage key — every time anything in your app calls File::create() or File::get().
The contract
IPS\Extensions\FileStorageAbstract declares four abstract methods and nothing else. All four are required.
abstract class FileStorageAbstract { /* Progress denominator for the move task. Must be > 0 if you have files, or nothing will ever be moved. Need NOT be an accurate file count. Must never throw. */ abstract public function count(): int; /* Move one "step" of files. $offset starts at 0 and increases by 1 per call, forever, until you throw UnderflowException. Return type is void — you cannot return a new offset. */ abstract public function move( int $offset, int $storageConfiguration, int $oldConfiguration=NULL ) : void; /* TRUE if this container-relative path is a file your app still references. FALSE means "orphan" and the file is queued for deletion. Called with a string in practice, never a File. */ abstract public function isValidFile( File|string $file ): bool; /* Delete every file this extension owns. Called on app uninstall, while your tables still exist. */ abstract public function delete() : void; } Three further members are read by core but are not declared on the abstract, so nothing reminds you they exist:
MemberRead byEffect public static bool $isPrivateIPS\File\Amazon::isPrivate()When TRUE, objects are PUT with X-Amz-Acl: bucket-owner-read instead of public-read, and API output returns a signed temporary URL. public static array $storeGzipExtensionsIPS\File\Amazon::getGzipExtensions()File extensions for which a second, gzipped copy is stored (core uses array( 'css', 'js' ) for core_Theme). public static function settingsUpdated()MoveFiles and IPS\File::settingsUpdated(), both via method_exists()Called once when a move to a new configuration completes, so you can flush caches that embed file URLs. The abstract writes int $oldConfiguration=NULL, which is an implicitly-nullable parameter. Declaring ?int $oldConfiguration=NULL in your subclass is the identical type and is signature-compatible.
A minimal example
Two files. First, register the class in applications/myapp/data/extensions.json — in IC5 this file is the extension registry; the directory is never scanned.
{ "core": { "FileStorage": { "Photos": "IPS\\myapp\\extensions\\core\\FileStorage\\Photos" } } } Then applications/myapp/extensions/core/FileStorage/Photos.php. The storage key this produces is myapp_Photos (application directory, underscore, extensions.json key), and that is the string you pass to File::create() and File::get() everywhere else in your app.
<?php namespace IPS\myapp\extensions\core\FileStorage; use Exception; use IPS\Db; use IPS\Extensions\FileStorageAbstract; use IPS\File; use UnderflowException; use function defined; if ( !defined( '\IPS\SUITE_UNIQUE_KEY' ) ) { header( ( $_SERVER['SERVER_PROTOCOL'] ?? 'HTTP/1.0' ) . ' 403 Forbidden' ); exit; } class Photos extends FileStorageAbstract { public function count(): int { /* Guard: this runs on a shared AdminCP screen and must not throw */ if ( !Db::i()->checkForTable( 'myapp_photos' ) ) { return 0; } return (int) Db::i()->select( 'COUNT(*)', 'myapp_photos', 'photo_location IS NOT NULL' )->first(); } public function move( int $offset, int $storageConfiguration, ?int $oldConfiguration=NULL ) : void { /* NOTE: deliberately OUTSIDE the try. When the offset runs past the last row, ->first() throws UnderflowException and that is the ONLY thing that ends the background task. */ $row = Db::i()->select( '*', 'myapp_photos', 'photo_location IS NOT NULL', 'photo_id', array( $offset, 1 ) )->first(); try { $file = File::get( $oldConfiguration ?: 'myapp_Photos', $row['photo_location'] )->move( $storageConfiguration ); if ( (string) $file != $row['photo_location'] ) { Db::i()->update( 'myapp_photos', array( 'photo_location' => (string) $file ), array( 'photo_id=?', $row['photo_id'] ) ); } } catch( Exception $e ) { /* Logged by IPS\File; skip this row and let the task continue */ } } public function isValidFile( File|string $file ): bool { try { Db::i()->select( 'photo_id', 'myapp_photos', array( 'photo_location=?', (string) $file ) )->first(); return TRUE; } catch ( UnderflowException $e ) { return FALSE; } } public function delete() : void { foreach( Db::i()->select( '*', 'myapp_photos', 'photo_location IS NOT NULL' ) as $row ) { try { File::get( 'myapp_Photos', $row['photo_location'] )->delete(); } catch( Exception $e ){} } } } My extension never appears in the Storage Settings list
The AdminCP form at applications/core/modules/admin/overview/files.php builds one Select per entry returned by Application::allExtensions( 'core', 'FileStorage', FALSE, NULL, NULL, TRUE ). That walks Application::extensions(), which reads only applications/<app>/data/extensions.json. Dropping a file into extensions/core/FileStorage/ registers nothing.
Two silent filters sit inside that loader. extensions() skips any entry where !is_string( $classname ) or !class_exists( $classname ) — the comment in core is "Switching between branches confuses extensions" — so a namespace typo produces no error at all, just an absent row. And constructExtensionClass() wraps new $class in catch( RuntimeException | OutOfRangeException $e ){} and returns null, so a constructor that throws either of those also just disappears from the list. Neither failure is logged anywhere. If the row is missing, check extensions.json first and your namespace second.
Also note the results are cached: allExtensions() stores the whole map in Store::i()->extensions and extensions() memoises in a static. After editing extensions.json, clear the data store.
RuntimeException: NO_STORAGE_EXTENSION — but only on the dev install
Your storage key is composed from two different sources depending on who is asking, and they must agree.
File::getClass() contains an IN_DEV-only sanity check. When the key you passed has no entry in upload_settings, it loops Application::allExtensions( 'core', 'FileStorage', FALSE ), splits each object's namespace, and rebuilds the key as $extensionNamespace[1] . '_' . $extensionName — that is, application directory plus the class basename. If nothing matches it throws RuntimeException( 'NO_STORAGE_EXTENSION' ). Everywhere else — MoveFiles, Amazon::isPrivate(), Amazon::getGzipExtensions(), File::settingsUpdated() — resolution goes through Application::getExtensionClass(), which looks the key up as a key in extensions.json.
So if you register "Photos": "IPS\\myapp\\extensions\\core\\FileStorage\\PhotoStorage", the key myapp_Photos works on a production install and throws NO_STORAGE_EXTENSION on a dev install, while the key myapp_PhotoStorage does the reverse. Keep the extensions.json key identical to the class basename and the file name.
The dropdown for my extension is greyed out forever: "This setting can not be changed as there is a move in progress."
This one is worth reading twice, because the symptom appears in the AdminCP and the cause is in a background task that reported success.
When you change a storage configuration, the AdminCP does not simply write the new ID. It writes a two-element array into upload_settings — $values[$k] = array( $newId, $oldId ) — as a lock, so that File::getClass() can still find not-yet-moved files. The only code that ever collapses that array back to a plain integer is the UnderflowException branch of IPS\core\extensions\core\Queue\MoveFiles::run(). While the array is present, files.php sets 'disabled' => $disabled on the Select and injects the language string file_storage_move_in_progress as the field warning.
MoveFiles::run() starts like this:
$exploded = explode( '_', $data['storageExtension'] ); // "filestorage__myapp_Photos" try { $classname = Application::getExtensionClass( $exploded[2], 'FileStorage', $exploded[3] ); } catch( OutOfRangeException ) { throw new \IPS\Task\Queue\OutOfRangeException; } The setting key is filestorage__ + app + _ + extension key, so the naive explode() gives [2] = app and [3] = extension key. Put an underscore anywhere in your extension key (Photos_v2) or your application directory and $exploded[3] is a fragment, getExtensionClass() throws OutOfRangeException, and the handler rethrows it as \IPS\Task\Queue\OutOfRangeException — which Task::runQueue() interprets as "the task is done" and deletes the queue row, logging "completed successfully" to runQueue_log. No files move, no error is raised, and the lock array stays in upload_settings permanently. The admin sees a permanently disabled dropdown and there is nothing in the logs connecting the two.
The same parse also disables the extension when the application is disabled or when RECOVERY_MODE is on, since getExtensionClass() throws OutOfRangeException in both cases. The fix is simply: no underscores in FileStorage extension names. Use PhotosV2, never Photos_V2. Recovering an already-stuck site means editing the upload_settings row in core_sys_conf_settings by hand to replace the two-element array with a single ID.
Files that are supposed to be private are uploaded to S3 as public-read
IPS\File\Amazon::isPrivate() is the only thing that decides the ACL on every PUT:
$bits = explode( '_', $this->storageExtension ); try { $class = Application::getExtensionClass( $bits[0], 'FileStorage', $bits[1] ); if ( isset( $class::$isPrivate ) ) { return $class::$isPrivate; } } catch( OutOfRangeException ){} return false; Note $bits[0]/$bits[1] here versus $exploded[2]/$exploded[3] in MoveFiles — same string, two different offsets, because this one has no filestorage__ prefix. The failure mode is the important part: any underscore in your extension name, or a class not present in extensions.json, makes getExtensionClass() throw, the empty catch swallows it, and the method returns false — meaning your private files are written with X-Amz-Acl: public-read. Nothing is logged, nothing fails, and the file uploads perfectly. You only find out by inspecting object ACLs in the bucket. getGzipExtensions() degrades the same way, harmlessly.
Because $isPrivate is read statically off the class name, it must be public static bool $isPrivate = true; — an instance property will not be seen. Core's only example is IPS\downloads\extensions\core\FileStorage\Files.
The move task runs forever and the progress bar passes 100%
MoveFiles::run() has no upper bound derived from count(). It runs a fixed REBUILD_SLOW iterations per cycle, calling move( $offset, ... ) with $offset incremented by one each time, and the only exits are $data['count'] === 0 or an UnderflowException propagating out of your move():
catch ( UnderflowException $e ) { /* collapse the lock in upload_settings, call settingsUpdated(), then */ throw new \IPS\Task\Queue\OutOfRangeException; } catch ( Exception $e ) { Log::log( $e, 'move_files_bgtask' ); $offset++; continue; } The classic mistake is wrapping the whole body of move() in try { ... } catch ( Exception $e ) {} to be defensive. UnderflowException extends RuntimeException extends Exception, so you swallow your own terminator and the task never finishes — it keeps incrementing the offset every cron run indefinitely, and the lock in upload_settings is never released (see the trap above). Every core implementation puts the terminating query outside its try block for exactly this reason; look at core/Attachment::move().
Ordering matters too: the UnderflowException catch is listed first. If you throw one from deep inside a nested try that catches Exception, it is gone.
Any other exception costs you one row: it is logged to the AdminCP system log under the key move_files_bgtask, the offset advances, and the loop continues. That is where to look when "some of my files moved and some didn't".
Fatal error: A void function must not return a value
In IC4, move() could return an integer to jump the offset. IC5 typed the signature as : void, but the surrounding code was never cleaned up, and the stale documentation is actively misleading. Core's own ClubField, Imageproxycache and Theme extensions carry the docblock line:
* @return void An offset integer to use on the next cycle, or nothing And MoveFiles::run() still contains:
$return = $extension->move( $offset, (int) $data['newConfiguration'], (int) $data['oldConfiguration'] ); $offset++; if ( is_numeric( $return ) ) { $offset = $return; } That block is dead code in IC5 — $return is always NULL. If you follow the docblock and return $newOffset;, PHP raises Fatal error: A void function must not return a value inside the background task, which surfaces as a stalled queue row and an uncaught_exception log entry, not as anything the admin can connect to your app. The offset is a plain counter you do not control; design your move() around a LIMIT $offset, 1 cursor or around fixed step numbers, as core/Theme does with its switch ( $offset ).
Nothing moves at all, and the progress bar sits at 100%
count() is not a file count — it is a permission slip and a progress denominator, and both consumers treat zero as "nothing to do".
In files.php: $count = $extension->count(); if ( $count ) { ... Task::queue( 'core', 'MoveFiles', ... ) ... }. No task is queued at all if you return 0. In configurationMove() the test is if( $count > 0 ). And inside MoveFiles::run(), if( $data['count'] === 0 ) { throw new UnderflowException; } ends the task immediately. Return 0 and your files stay exactly where they are, permanently, while the AdminCP reports the move as finished.
Accuracy, on the other hand, does not matter at all. getProgress() computes round( 100 / $data['count'] * $offset, 2 ) and nothing clamps it. Core is wildly inconsistent here and it is harmless: core/Attachment::count() returns MAX(attach_id) while its move() uses LIMIT $offset, 1; core/Theme::count() returns the literal 6 with the comment "it's the number of steps this will take to move them, which is all it's used for"; cms/Pages::count() returns 1; core/ClubField::count() returns a real row count but its move() processes every field on offset 0 and then throws UnderflowException immediately. Any positive integer is a valid answer. Zero is not.
Saving the Storage Settings form dies with my app's database error
In files.php, the settings save loop protects the class lookup but not the call:
try { $classname = Application::getExtensionClass( $exploded[2], 'FileStorage', $exploded[3] ); } catch( OutOfRangeException ) { continue; } $extension = new $classname; ... $count = $extension->count(); That count() has no try/catch around it. If your table was dropped by a botched upgrade, or your count() queries a column that a migration has not added yet, the exception escapes and takes down the entire Storage Settings save for every application on the site. The admin cannot change any storage setting until your app is disabled. This is why core/Imageproxycache caches a Db::i()->checkForTable( 'core_image_proxy' ) result and returns 0 when the table is missing — treat count() as code that must never throw.
The other AdminCP path, Move all files off this configuration (configurationMove()), inverts the risk: the whole per-extension block including count() sits inside catch( Exception $e ){}. If your count() throws there, the line $settings[ $k ] = $newStorageId; never runs, so your extension keeps pointing at the old configuration — which the same request then queues DeleteMovedFiles against with 'storageToDelete' => $current['id']. Silent skip, followed by a deletion pass over the storage you were skipped onto.
One further hazard in that same loop: it iterates every key in upload_settings testing if ( $v == $current['id'] ), but upload_settings also contains the boolean filestorage_move key, written by the settings form's own json_encode( $values ). On a site whose first storage configuration has id = 1, true == 1 matches, explode( '_', 'filestorage_move' ) has no index 2 or 3, and getExtensionClass( null, 'FileStorage', null ) raises a TypeError — which is an Error, not an Exception, so the surrounding catch( Exception $e ){} does not catch it and the whole action fatals before reaching later extensions. The settings-form loop excludes that key explicitly ($k != 'filestorage_move'); this one does not.
My files were deleted after an admin ran "Find orphaned files"
isValidFile() is the only thing standing between your files and deletion, and the deletion is not reviewed by a human. FindOrphanedFiles calls File::orphanedFiles(), which calls removeOrphanedFiles( $fileIndex, Application::allExtensions( 'core', 'FileStorage', FALSE ) ) on the storage handler. The handler walks every object in the configuration, asks every FileStorage extension in the suite isValidFile( $path ), and logs anything nobody claims via logOrphanedFile() into core_file_logs with log_type = 'orphaned'. As soon as the scan reports _done, FindOrphanedFiles itself queues DeleteOrphanedFiles, which loads each logged row and calls ->delete(). There is no confirmation screen between the scan and the deletion.
The consequences are worth spelling out:
An empty or wrong isValidFile() deletes your users' files. A stub that returns FALSE, or one that compares against the wrong column, is not a no-op. The argument is a container-relative path string, not a URL and not a File. FileSystem passes SplFileInfo::getSubPathname(), Database passes $file['container'] . '/' . $file['filename'], Ftp passes the pathname with the configured root stripped, and Amazon passes the S3 key with bucket_path stripped — as a SimpleXMLElement, not a string. Always (string) $file before comparing, exactly as every core implementation does. If your application is disabled, your extension is not in the list. allExtensions() skips applications failing appIsEnabled(), and skips all non-IPS applications entirely when RECOVERY_MODE is on. Running the orphan scan while your app is disabled means nobody claims your files and all of them are queued for deletion. Throwing from isValidFile() aborts the scan for everybody. Only FileSystem guards the call, and only against InvalidArgumentException (where it treats the file as valid and moves on). Amazon, Database and Ftp call $engine->isValidFile( ... ) bare. One badly written third-party extension takes the whole orphan scan down. Find orphaned files fails with "Call to undefined method ...::getSubPathname()"
In 5.0.19, IPS\File\FileSystem::removeOrphanedFiles() contains a variable-name bug:
foreach( $engines as $engine ) { try { if( $engine->isValidFile( $engine->getSubPathname() ) ) { continue 2; } } catch( InvalidArgumentException $e ) { continue 2; } } $engine is your FileStorage extension object; the intended receiver is $f, the SplFileInfo from the directory iterator. FileStorage extensions have no getSubPathname(), so this raises Error: Call to undefined method IPS\...\FileStorage\<Name>::getSubPathname() on the first file examined. Error is not InvalidArgumentException and not RuntimeException, so neither the local catch nor FindOrphanedFiles's catch( RuntimeException $ex ) intercepts it — it escapes Task::runQueue() and lands in the uncaught_exception log, named after whichever extension happened to be first in the list. The class named in that error is almost never the class at fault; it is simply the first entry of allExtensions( 'core', 'FileStorage' ).
The practical consequence is that on a FileSystem storage configuration the orphan scan cannot run at all in this version, so isValidFile() is effectively exercised only on Amazon, Database and Ftp configurations — which is where the deletion risk described above is real. Do not conclude from a green run on a local filesystem install that your isValidFile() has been tested.
After a storage move, the old copies are never deleted
IPS\File::move() does not delete the source. Unless you pass File::MOVE_DELETE_NOW, it copies the file and then writes a row into core_file_logs with log_type = 'move' and log_configuration_id set to the source configuration. Cleanup is a separate background task, DeleteMovedFiles, which drains those log rows.
That task is queued once, after the settings loop, gated on a single variable:
foreach ( $values as $k => $v ) { ... if ( ... !$newClass::moveCheck( ... ) ) { $rebuild = FALSE; } else { $rebuild = TRUE; } if ( $rebuild ) { /* queue MoveFiles for this extension */ } } if( $rebuild ) { Task::queue( 'core', 'DeleteMovedFiles', ... ); } $rebuild is reassigned on every iteration and only its final value is tested. If the last changed extension in the loop resolves to "same handler, same configuration, no move needed", $rebuild ends as FALSE and DeleteMovedFiles is never queued — even though earlier extensions were queued and are busy copying files. The move succeeds, the database points at the new locations, and a full duplicate set of files is left behind on the old configuration with rows still sitting in core_file_logs. Nothing reports this. If you are debugging disk usage after a move, check SELECT COUNT(*) FROM core_file_logs WHERE log_type = 'move'.
Files uploaded while a move was running 404 afterwards
Both File::create() and File::get() call static::getClass( $storageExtension, TRUE ) — that second argument is $tryOldFirst. When upload_settings holds the two-element lock array, getClass() does array_pop() and returns the old configuration. That is correct for reads, since unmoved files are still there. It also means new uploads created during the move are written to the storage you are moving away from.
Whether they survive is a race. If your move() uses a LIMIT $offset, 1 cursor over a table, a row inserted before the task reaches the end will still be picked up. A row inserted after it has passed will not, and once the lock collapses to the new configuration ID, File::get() resolves against the new storage and the file is unreachable. Extensions like core/Theme that move fixed sets in numbered steps have no chance of catching late arrivals at all. This is what the AdminCP message filestorage_move_info means by "you may experience broken links/images until the process has completed" — except some of them do not heal. Prefer moving storage during a maintenance window.
settingsUpdated() throws a TypeError, or is never called
This optional hook is discovered by method_exists() and called from two places that disagree about its signature.
/* MoveFiles::run() — passes an int configuration ID */ $extension::settingsUpdated( $data['newConfiguration'] ); /* IPS\File::settingsUpdated() — passes the decoded configuration array */ $classname::settingsUpdated( $this->configuration ); Core's only implementation, core/Theme::settingsUpdated(), declares no parameters at all, which is what makes both call sites work — PHP silently discards extra arguments to userland functions. Declare a typed parameter and one of the two callers will hand you the wrong type: int breaks the File path with a TypeError, array breaks the MoveFiles path. Declare it with no parameters.
It must also be static. Both call sites use the :: form on a class name or an object, and in PHP 8 calling a non-static method that way is a fatal Error. In MoveFiles, that call sits inside the catch ( UnderflowException ) block, after Settings::i()->changeValues() has already released the lock but before throw new \IPS\Task\Queue\OutOfRangeException — so the fatal leaves the queue row undeleted and the task re-runs and re-fatals on every cron cycle.
Finally, note when it does not fire: only on the UnderflowException completion of MoveFiles. If count() returned zero, or the extension key failed to parse, no move task ran and settingsUpdated() is never called, so any cache you flush there stays stale.
Uninstalling my app stops halfway and leaves it half-removed
Application::delete() calls your delete() like this:
foreach( $this->extensions('core', 'FileStorage' ) as $extension ) { try { $extension->delete(); } catch( PHPException $e ){} } PHPException is Exception aliased at the top of the file. A TypeError, an ArgumentCountError, or any other Error is not caught and aborts the uninstall in place — after language strings, theme resources, modules, permissions and settings rows have already been deleted, and before the database tables are dropped. The application is left in a state no screen can finish cleaning up.
Two facts about ordering are useful when writing delete(). Your tables still exist at this point (schema.json is processed roughly sixty lines later), so querying them is correct and expected. But core_sys_conf_settings rows belonging to your app have already been deleted, so do not read your own settings there.
Also be aware that nothing removes your filestorage__<app>_<Name> keys from upload_settings on uninstall. installExtensions() writes them; delete() has no counterpart. They linger as harmless dead keys, invisible in the AdminCP (which lists extensions, not settings keys) but still iterated by configurationMove().
A FileStorage extension added in an app update never gets a setting
Application::installExtensions() — the method that writes $settings['filestorage__' . $this->directory . '_' . $key] = $fileSystem['id'] for each of your extensions — is called from exactly two places: the setup installer and the AdminCP "install application" flow. It does not run on a version upgrade. Add a new FileStorage extension in version 2 of your app and no upload_settings key is ever created for it.
This does not fail loudly. The first time anything calls File::create( 'myapp_NewThing', ... ), File::getClass() notices the missing key, loops core_file_storage, calls testSettings() on each configuration in id order, takes the first that does not throw a LogicException, and persists that choice back into upload_settings. On a single-configuration site the result is right by accident. On a site with two configurations, your new bucket silently lands wherever the first row points — typically the local filesystem — while the rest of your app is on S3, and the admin never chose it. Write the key explicitly in an upgrade step:
$settings = json_decode( Settings::i()->upload_settings, TRUE ); if ( !isset( $settings['filestorage__myapp_NewThing'] ) and isset( $settings['filestorage__myapp_Photos'] ) ) { $settings['filestorage__myapp_NewThing'] = $settings['filestorage__myapp_Photos']; Settings::i()->changeValues( array( 'upload_settings' => json_encode( $settings ) ) ); } While you are there, add the two language strings the AdminCP form needs. The Select is built as new Select( 'filestorage__' . $name, ... ), so it looks for filestorage__myapp_Photos as its label and filestorage__myapp_Photos_desc as its description. Without them the admin sees the raw key.
Verified against
Everything above was read from the source of Invision Community 5.0.19: system/Extensions/FileStorageAbstract.php, system/File/File.php, system/File/FileSystem.php, system/File/Amazon.php, system/File/Database.php, system/File/Ftp.php, system/Application/Application.php, system/Task/Task.php, applications/core/modules/admin/overview/files.php, the MoveFiles, DeleteMovedFiles, FindOrphanedFiles and DeleteOrphanedFiles queue extensions, and all 34 core core/FileStorage implementations across the blog, calendar, cms, core, downloads, forums, gallery and nexus applications. Line-level details — particularly the getSubPathname() bug, the $rebuild overwrite, and the dead is_numeric( $return ) block — are specific to this release and may be fixed without notice.
Related: S3 and R2 storage break in any CLI script without HTTPS=on, Dumping an Invision Community database from PHP: ANSI_QUOTES, multi-member gzip, and the 0x trap

0 comments
39 views

Ernest Defoe

July 26Jul 26

The core/Uninstall extension in Invision Community 5

The core/Uninstall extension is the only hook an application gets into its own removal, and the only way one application can react to a different application being removed. Core calls it from IPS\Application::delete() (system/Application/Application.php) — preUninstall() before any data is touched, onOtherUninstall() on every enabled application immediately after, and postUninstall() near the end, after your database tables have already been dropped.
Despite the docblock wording, there is no plugin system in IC5. There is no plugins/ directory and no IPS\Plugin class in 5.0.19; onOtherUninstall() only ever fires for applications.
The contract
system/Extensions/UninstallAbstract.php declares three methods. None of them are abstract — every one has an empty body, so you only override the ones you need. None return a value, and none declare a thrown exception.
namespace IPS\Extensions; abstract class UninstallAbstract { /* Called on the app being removed, BEFORE core deletes anything. $application is always your own app directory here. */ public function preUninstall( string $application ) : void {} /* Called on the app being removed, AFTER core has dropped your tables, deleted your settings, lang words, themes and widgets. */ public function postUninstall( string $application ) : void {} /* Called on EVERY enabled application (including the one being removed) when any application is uninstalled. $application is the directory of the app going away. */ public function onOtherUninstall( string $application ) : void {} } The order core executes them in, taken from Application::delete():
StepWhat happens 1$this->extensions( 'core', 'Uninstall' ) — your extension objects are constructed once and kept in an array 2preUninstall( $this->directory ) on each of your extensions 3onOtherUninstall( $this->directory ) on Application::allExtensions( 'core', 'Uninstall', FALSE ) — every enabled app 4Profile steps, front navigation, club node maps, search index, then ~25 DELETEs from shared core_* tables 5ContentRouter class cleanup, attachment maps, tasks, lang words, email templates, theme templates/CSS/resources 6FileStorage extension delete(), upload settings, notification defaults, login handlers 7Tables in data/schema.json are dropped; addColumn/addIndex from setup/install/queries.json are reverted 8Widget rows, widget areas, widget trash 9postUninstall( $this->directory ) on the same objects from step 1 10FURL definitions, parent::delete() (the core_applications row), data store flush, then the applications/<dir>/ folder is deleted from disk A minimal example
File: applications/myapp/extensions/core/Uninstall/Cleanup.php
<?php namespace IPS\myapp\extensions\core\Uninstall; use IPS\Data\Store; use IPS\Db; use IPS\Extensions\UninstallAbstract; use function defined; if ( !defined( '\IPS\SUITE_UNIQUE_KEY' ) ) { header( ( $_SERVER['SERVER_PROTOCOL'] ?? 'HTTP/1.0' ) . ' 403 Forbidden' ); exit; } class Cleanup extends UninstallAbstract { /** * Before anything is removed - your tables still exist here. */ public function preUninstall( string $application ) : void { /* Rows we put in a core table that core will not match on app= */ if ( Db::i()->checkForTable( 'myapp_widget_map' ) ) { Db::i()->delete( 'core_item_markers', array( 'item_app=?', 'myapp' ) ); } } /** * After our tables are gone. Guard EVERY table access. */ public function postUninstall( string $application ) : void { /* Columns we added in an UPGRADE step are not reverted by core */ foreach ( array( 'myapp_score', 'myapp_last_seen' ) as $column ) { if ( Db::i()->checkForColumn( 'core_members', $column ) ) { Db::i()->dropColumn( 'core_members', $column ); } } /* Our own datastore keys */ if ( isset( Store::i()->myapp_config ) ) { unset( Store::i()->myapp_config ); } } /** * Someone else is being uninstalled. Never throw from here. */ public function onOtherUninstall( string $application ) : void { /* We are also called with our OWN directory - skip that case */ if ( $application === 'myapp' ) { return; } try { if ( Db::i()->checkForTable( 'myapp_integrations' ) ) { Db::i()->delete( 'myapp_integrations', array( 'int_app=?', $application ) ); } } catch ( \Exception $e ) { /* Swallow: an exception here aborts the OTHER app's uninstall */ } } } And the registration in applications/myapp/data/extensions.json:
{ "core": { "Uninstall": { "Cleanup": "IPS\\myapp\\extensions\\core\\Uninstall\\Cleanup" } } } My uninstall code never ran and the ACP said "Deleted"
The most likely cause is that the application was flagged app_requires_manual_intervention. Every suite upgrade sets this on all non-IPS applications — system/Dispatcher/Setup.php and applications/core/modules/setup/upgrade/applications.php both run Db::i()->update( 'core_applications', [ 'app_enabled' => 0, 'app_requires_manual_intervention' => 1 ], Db::i()->in( 'app_directory', IPS::$ipsApps, true ) ).
When the admin then deletes that application, applications/core/modules/admin/applications/applications.php::delete() checks the flag and reroutes to static::_deleteLegacyApp( $application['app_directory'] ). That method reimplements most of Application::delete() by hand and never calls preUninstall() or postUninstall() at all — it only calls onOtherUninstall() on the other apps. It also skips the search index entirely (the source literally reads // @todo Search index), skips your FileStorage extensions, skips your Notifications and LoginHandler cleanup, and does not run Store::i()->clearAll().
There is a second, related consequence: Application::applications() skips any row where app_requires_manual_intervention is set and the directory is not in IPS::$ipsApps. So a flagged third-party app is invisible to allExtensions() — it will not receive onOtherUninstall() when some other app is removed either.
There is no fix you can apply from inside the extension. Design for it: anything that absolutely must be cleaned up should also be safe to leave behind, and your install routine should be idempotent so a later reinstall tolerates leftovers.
The ACP reports "Deleted" but nothing was removed
This is the single most dangerous behaviour of this extension point. In the ACP controller, $node->delete() sits inside a try block whose first catch is:
/* Legacy */ catch ( UnexpectedValueException $e ) { if( !Member::loggedIn()->hasAcpRestriction( 'core', 'applications', 'app_delete' ) ) { Output::i()->error( 'node_noperm_delete', '2C133/J', 403, '' ); } Db::i()->delete( 'core_applications', array( 'app_directory=?', Request::i()->id ) ); } That catch exists to handle an application row with no loadable Application.php. But it wraps the entire delete() call, so if your preUninstall(), postUninstall() or onOtherUninstall() throws UnexpectedValueException — and a great deal of IPS code throws it, including Application::constructFromData() — core interprets it as "this is a legacy app", deletes the single core_applications row, and redirects with the deleted language string, which renders as "Deleted". Every table, setting, language word, theme template and file is left in place, and the app has no row so it can never be uninstalled properly again.
The error surfaces nowhere near its cause. There is no log entry, no error code, and no indication that your extension threw. Wrap every method body in try { … } catch ( \Exception $e ) {}, or at minimum guard against UnexpectedValueException:
public function preUninstall( string $application ) : void { try { $this->doCleanup(); } catch ( \Exception $e ) { /* Never let this escape - core would treat it as a legacy app */ } } "We could not locate the item you are trying to view." with code 2C133/I
Same try block, second catch: catch ( OutOfRangeException $e ) { Output::i()->error( 'node_error', '2C133/I', 404, '' ); }. OutOfRangeException is what ActiveRecord::load(), Application::load() and Settings lookups throw when a record is missing — exactly the sort of thing that happens in postUninstall(), where half your data is already gone.
The visible result is a 404 error page in the ACP with code 2C133/I, and the uninstall stops at whatever step you were on. If you threw from postUninstall(), the tables have already been dropped but core_applications still has the row, the FURL definitions are still registered and the app folder is still on disk — a half-uninstalled application that the ACP will still list. Catch OutOfRangeException anywhere you call ::load() in an Uninstall method.
Uninstalling someone else's app fails with an error from my code
Application::delete() loops the onOtherUninstall() calls with no error handling whatsoever:
foreach( static::allExtensions( 'core', 'Uninstall', FALSE ) as $extension ) { $extension->onOtherUninstall( $this->directory ); } Compare this with the FileStorage loop a few hundred lines later, which is wrapped in try { $extension->delete(); } catch( PHPException $e ){}. The Uninstall loop has no such protection.
An exception thrown from your onOtherUninstall() aborts the uninstall of an application you have nothing to do with, at step 3 of 10 — before core has deleted a single row. The admin sees an error while removing, say, Gallery, and there is nothing on the screen connecting it to your app. Your extension is only identifiable from the stack trace in core_error_logs. This is the classic "one bad app breaks a shared screen" case, and it is why the example above swallows exceptions in onOtherUninstall() unconditionally.
The same loop also means onOtherUninstall() runs on every enabled application on the site on every application uninstall. Keep it cheap and keep it defensive.
"Table 'x.myapp_something' doesn't exist" during postUninstall
By the time postUninstall() is called, core has already looped data/schema.json and run Db::i()->dropTable( $tableName, TRUE ) on every table in it. Your own tables are gone. Core's own Pages extension documents this with a bare comment at the top of the method:
public function postUninstall( string $application ) : void { /* cms_databases has been removed */ $tables = array(); … } Anything that needs to read your data must happen in preUninstall(). Anything in postUninstall() that touches a table must be guarded with Db::i()->checkForTable() — including tables you think still exist, because a partially-failed earlier uninstall attempt may have already removed them. Note that Databases::preUninstall() guards even the pre-hook with if ( Db::i()->checkForTable( 'cms_databases' ) ).
The reverse is also true and useful: at postUninstall() time the core_applications row still exists and applications/<dir>/ is still on disk, because both are removed afterwards. You can still read your own data/*.json files there.
onOtherUninstall fires with my own app's directory
allExtensions() iterates static::applications(), and at step 3 the application being removed is still enabled and still in core_applications — parent::delete() does not run until step 10. So your own onOtherUninstall() is invoked with your own directory as $application, in addition to the preUninstall() you already received.
Core's Pages extension relies on this (cms\extensions\core\Uninstall\Widgets has only an onOtherUninstall() and it is what cleans up Pages' own widget areas when Pages itself is removed). But if your onOtherUninstall() writes rows, re-caches, or rebuilds something on behalf of the surviving apps, it will do that work for an app that is about to cease to exist. Test $application against your own directory and return early if you do not want that.
A disabled app never gets a chance to clean up
allExtensions() contains two hard filters before it collects any extension:
if( RECOVERY_MODE and !in_array( $application->directory, IPS::$ipsApps ) ) { continue; } if ( !static::appIsEnabled( $application->directory ) ) { continue; } So onOtherUninstall() is skipped for every disabled application, and — if the site is in recovery mode — for every third-party application. Rows that a disabled app holds pointing at the removed app stay there forever, and there is no retry when the app is re-enabled.
Note the asymmetry: preUninstall()/postUninstall() come from $this->extensions( 'core', 'Uninstall' ), an instance method that checks neither appIsEnabled() nor recovery mode. Your own uninstall code therefore runs even when your app is disabled — but your own onOtherUninstall() does not, because that comes from the filtered static call.
My extension is never constructed and no error appears
Application::extensions() reads data/extensions.json and skips silently on two conditions:
foreach ( $json[ $app ][ $extension ] as $name => $classname ) { if( !is_string( $classname ) or !class_exists( $classname ) ) { /* Switching between branches confuses extensions */ continue; } … } and constructExtensionClass() returns null — again silently — if class_exists() fails or if the constructor throws RuntimeException or OutOfRangeException. Nothing is logged in any of those paths.
So a typo in the class name, a namespace that does not match the directory, a missing data/extensions.json entry, or a fatal-free parse problem all present identically: the uninstall completes cleanly and your cleanup simply did not happen. In dev mode the JSON is regenerated by Application::buildExtensionsJson(), which scans applications/<dir>/extensions/ and skips any class that does not exist or that defines a deprecated() method — visit the Developer Center once after adding the file, and verify the entry is actually in the built data/extensions.json you ship.
Fatal TypeError when the uninstall starts
UninstallAbstract declares no constructor, but core always instantiates extensions with one argument:
$obj = new $classToUse( $checkAccess === TRUE ? Member::loggedIn() : ( $checkAccess === FALSE ? NULL : $checkAccess ) ); For Uninstall extensions $checkAccess is FALSE in both call sites, so your class is constructed as new Cleanup( NULL ). If you copy a constructor signature from another extension type and write public function __construct( Member $member ), that is a TypeError. constructExtensionClass() only catches RuntimeException and OutOfRangeException, so a TypeError escapes it, escapes the ACP's two catches, and produces a hard error page. Either declare no constructor at all, or accept a nullable first argument: public function __construct( ?Member $member = NULL ).
Columns and rows my installer added are still in core tables
On uninstall core reverts setup/install/queries.json, but the switch only has two cases and no default:
foreach( $schema as $instruction ) { switch ( $instruction['method'] ) { case 'addColumn': /* dropColumn, ignoring error 1091 */ break; case 'addIndex': /* dropIndex, ignoring error 1091 */ break; } } Two consequences, both verifiable against IPS's own apps:
insert is never reverted. calendar, downloads, forums and gallery all use "method": "insert" in their install queries. Those rows survive the uninstall permanently. Only setup/install/queries.json is read. The per-version setup/upg_<version>/queries.json files are never consulted on delete. A column you added to core_members in an upgrade step, rather than at install, is not dropped. Drop those yourself in postUninstall() with a Db::i()->checkForColumn() guard, as in the example above, and delete your own inserted rows in preUninstall().
One of my tables survived the uninstall
Table dropping is driven entirely by data/schema.json — core iterates its keys and drops each one, swallowing only MySQL error 1051 ("unknown table"). A table you created directly with an addTable instruction in an upgrade step, or with raw SQL, and never added to schema.json is invisible to that loop and will remain in the database after the app is gone.
Pages has exactly this problem and solves it in postUninstall() by pattern-matching the table names, which is the general fix:
try { $databaseTables = Db::i()->query( "SHOW TABLES LIKE '" . Db::i()->prefix . "cms_custom_database_%'" )->fetch_assoc(); … } catch( Exception $ex ) { } foreach( $tables as $table ) { if ( Db::i()->checkForTable( $table ) ) { Db::i()->dropTable( $table ); } } Note that Pages defensively handles fetch_assoc() returning either a scalar or an array per row (if( is_array( $row ) ) { $tables[] = array_pop( $row ); } else { $tables[] = $row; }) — the shape of a SHOW TABLES LIKE result is not stable enough for core to rely on.
A Pages block still points at my widget after my app is uninstalled
cms\extensions\core\Uninstall\Widgets::onOtherUninstall() deletes cms_blocks rows whose block_plugin_app matches the removed app — but that loop is nested inside the loop over cms_page_widget_areas:
foreach ( Db::i()->select( '*', 'cms_page_widget_areas' ) as $row ) { … /* Remove blocks using a widget from the uninstalled application */ foreach( new ActiveRecordIterator( Db::i()->select( '*', 'cms_blocks', array( "block_plugin_app=?", $application ) ), Block::class ) AS $block ) { $block->delete(); } } If cms_page_widget_areas is empty — a site with Pages installed but no widgets placed on any page — the inner loop never executes and your blocks are left behind. If it has many rows, the same delete is re-run once per row. This is core's code, not yours; you cannot fix it from your own extension, but you can clean up after it in your own preUninstall() by deleting from cms_blocks yourself, guarded with Db::i()->checkForTable( 'cms_blocks' ) since Pages may not be installed.
A property I set in preUninstall is missing in onOtherUninstall
Core constructs your extension twice. Step 1 builds the objects used for preUninstall() and postUninstall() and deliberately keeps them — the source comment reads "They are stored in an array so that we only create one object per extension, instead of one each time we loop". Step 3 calls allExtensions(), which constructs a completely separate set of objects for onOtherUninstall().
So state stored on $this in preUninstall() is visible in postUninstall() — that is a supported way to hand data across the destructive middle of the uninstall — but it is not visible in onOtherUninstall(), even though that method is on the same class and runs in between. Use a static property or a datastore key if you need it in all three.
What core cleans up for you, and what it leaves behind
Before writing any cleanup code, check this list — most of it is already done. Core deletes, keyed on your app directory: core_modules, core_permission_index, core_dev, core_item_markers, core_reputation_index, core_upgrade_history, core_admin_logs, core_sys_conf_settings (your settings), core_queue, core_follow, core_follow_count_cache, core_item_statistics_cache, core_view_updates, core_moderator_logs, core_member_history, core_acp_notifications, core_solved_index, core_notifications, core_javascript, core_theme_templates_custom, core_sys_lang_words, core_email_templates, core_tasks and core_tasks_log, plus core_achievements_rules whose action starts with <yourapp>_.
Keyed on classes returned by your ContentRouter extensions it also clears core_approval_queue, core_deletion_log, core_content_promote, core_ratings, core_item_redirect, core_item_member_map, core_rss_import, core_soft_delete_log, core_post_before_registering, core_anonymous_posts, core_polls/core_voters, core_assignments and the core_rc_* report tables. Profile steps, front navigation, club node maps, notification defaults and preferences, login handler rows, theme templates/CSS/resources/editor settings, widgets, widget areas, widget trash and FURL definitions are all handled too.
What it does not do:
Delete attachment files. Only core_attachments_map rows for your EditorLocations keys are removed; the comment says "if the attachment is unused, the regular cleanup task will remove the file later". Guarantee your stored files are gone. FileStorage::delete() is called inside try { … } catch( PHPException $e ){} — any failure is swallowed with no log. Remove custom language words saved under a different word_app. Only word_app = <yourapp> is deleted, so node titles you saved with Lang::saveCustom( 'core', … ) survive. Remove the search index for classes not declared by a core/SearchContent extension. Index::i()->removeApplicationContent() iterates $application->extensions( 'core', 'SearchContent' ) and calls removeClassFromSearchIndex() for each supportedClasses() entry. Pages' preUninstall() exists purely to work around this and explains why: "The content router only returns databases linked to pages… so the method to remove all app content from the search index fails". Remove your IPS\Data\Store keys, in the legacy delete path. The normal ACP path calls Cache::i()->clearAll() and Store::i()->clearAll() after delete() returns; _deleteLegacyApp() only unsets modules, applications, widgets and furl_configuration. Pages unsets Store::i()->cms_menu in its own postUninstall(); do the same for yours. Delete your files on a developer install. The applications/<dir>/ removal at the end is wrapped in if ( !CIC2 AND !IN_DEV AND !DEMO_MODE AND file_exists( … ) ), and every rmdir/unlink is error-suppressed with @. Verified against
Read from Invision Community 5.0.19 source: system/Extensions/UninstallAbstract.php, system/Application/Application.php (delete(), extensions(), allExtensions(), constructExtensionClass(), applications(), appIsEnabled(), buildExtensionsJson()), applications/core/modules/admin/applications/applications.php (delete(), _deleteLegacyApp()), system/Content/Search/Index.php and system/Content/Search/Mysql/Index.php, and the three shipped implementations: cms/extensions/core/Uninstall/Databases.php, cms/extensions/core/Uninstall/Widgets.php and nexus/extensions/core/Uninstall/DisablePOP.php (which is an empty postUninstall() stub and does nothing). No plugin system exists in this version; onOtherUninstall() fires for applications only.
Related: Application versions: why re-uploading the same version does nothing, The core/Build extension in Invision Community 5, Extension method signatures are fixed, and a mismatch is a fatal on install

0 comments
43 views

Ernest Defoe

July 26Jul 26

The ?: fallback silently breaks any setting an admin can set to zero

Reading settings with ?: is idiomatic and, for anything numeric, wrong:
$perPage = Settings::i()->myapp_per_page ?: 25; PHP treats 0, '0' and '' as falsy. If an administrator sets that field to 0 — a perfectly reasonable way to say "none" or "unlimited" — they get 25 instead. The setting appears to do nothing, and no amount of re-saving it helps.
Where this actually bites
Anything meaning "unlimited", where 0 is the natural way to express it. Anything meaning "off", where 0 disables a feature. A delay or threshold where 0 means "immediately" or "always". It is invisible in testing because nobody tests with zero — they test with a sensible number.
The fix
public static function number( mixed $value, int $default ): int { if ( $value === NULL or $value === '' or !is_numeric( $value ) ) { return $default; } return (int) $value; } Then clamp separately, so the intent stays readable:
$perPage = max( 1, min( 500, static::number( Settings::i()->myapp_per_page, 25 ) ) ); The same trap with ?? and empty arrays
?? has the mirror-image problem: it only catches a key that is missing, not one that is empty. A widget's configuration is an empty array both before it has ever been configured and after being saved with nothing selected, so:
$show = $this->configuration['show'] ?? $defaults; // WRONG: empty array wins gives you an empty selection and a permanently blank block. Fall through to defaults when the value is empty, not merely absent:
$show = (array) ( $this->configuration['show'] ?? array() ); if ( !$show ) { $show = $defaults; } Verified against
Invision Community 5.0.19. The behaviour is plain PHP truthiness rather than anything Invision-specific, but the widget-configuration case is specific to how blocks store their settings.
Related: The report that shows nothing: only_full_group_by, and why catching the exception is the real bug, Db::insert() third and fourth arguments do very different things

0 comments
24 views

Ernest Defoe

July 26Jul 26

Application versions: why re-uploading the same version does nothing

An application's versions live in data/versions.json, keyed by a long integer version and valued with the human one:
{ "10000": "1.0.0", "10001": "1.0.1", "10002": "1.0.2" } Uploading the same version number is silently a no-op
Invision Community compares the long version in the uploaded file against the installed one. If it is not higher, the upload is accepted and nothing happens. No error, no warning, no changed files.
This is the single most wasteful mistake in application development, because the symptom is "my fix did not work" and the instinct is to go back and debug code that was never installed. If you have built a tar and it has left your machine, the next change gets a new version. Rebuilding the same number for a quick fix does not work.
Check what is actually installed before assuming otherwise:
SELECT app_directory, app_version, app_long_version FROM core_applications WHERE app_directory LIKE 'myapp%'; Every version needs an upgrade step
A new key in versions.json needs a matching setup/upg_<long>/ directory, at minimum a data.json declaring which upgrade steps to run:
{ "id": 10002, "name": "1.0.2", "steps": { "queries": false, "lang": true, "settings": false, "tasks": false, "modules": false, "widgets": false, ... }, "forceMainUpgrader": false } Set lang to true whenever you added or changed language strings, settings when you added settings, tasks when you added a scheduled task, widgets when a block changed. Leaving them all false means the code ships but its supporting data does not, which produces an application that installs cleanly and then shows raw language keys.
Language changes need a lang.json too
{ "normal": { "added": [ "myapp_new_setting", "myapp_new_setting_desc" ], "edited": [], "removed": [] }, "js": { "added": [], "edited": [], "removed": [] } } Verify the built file rather than trusting the build
A tar named myapp-1.0.2.tar can easily contain 1.0.1 — the filename is whatever you typed, the contents are whatever was on disk. Check the two agree before uploading:
tar -xOf dist/myapp-1.0.2.tar data/versions.json tar -xOf dist/myapp-1.0.2.tar --wildcards '*lang.xml' | grep -c 'key=' And check the dev/ directory did not get included, which happens easily and ships your working files to customers:
tar -tf dist/myapp-1.0.2.tar | grep -c '^myapp/dev/' # should be 0 Verified against
Invision Community 5.0.19, by reading the application upgrade handling and by shipping applications through it.
Related: A malformed manifest silently disables your whole application, The core/Build extension in Invision Community 5, What you can and cannot test from the command line

0 comments
28 views

Ernest Defoe

July 26Jul 26

What you can and cannot test from the command line

Loading init.php from a CLI script gives you most of the framework, and it is by far the fastest way to test an application. Some things genuinely cannot run there, and knowing which saves a lot of time spent debugging the harness instead of the code.
The basic harness
<?php require_once '/var/www/html/init.php'; use IPS\Db; /* IPS installs its own error handlers. Under CLI a fatal can end the process with NO OUTPUT AT ALL, which reads as the script silently stopping. Write to a file so failures are visible. */ const LOG = '/tmp/mytest.log'; @unlink( LOG ); function say( string $line = '' ): void { file_put_contents( LOG, $line . PHP_EOL, FILE_APPEND ); } set_exception_handler( function ( $e ) { say( 'FATAL ' . get_class( $e ) . ': ' . $e->getMessage() ); say( $e->getTraceAsString() ); } ); That error-handler detail is the first thing to get right. Without it, a fatal produces an empty terminal and no clue.
What does not work, and why
ThingWhat happens ACP or front controllersRuntimeException: Only subclasses of Dispatcher can be instantiated Widget::configuration()Reaches for a Dispatcher, same error Anything building a URL from the requestUndefined array key "SERVER_NAME", then QUERY_STRING, then more S3 / R2 file storageBAD_XML unless HTTPS=on is in the environment Anything reading the logged-in memberWorks, but the member is a guest unless you load one explicitly Do not chase the $_SERVER errors
Filling in SERVER_NAME gets you as far as QUERY_STRING, then SCRIPT_NAME, and eventually to the Dispatcher error anyway. If a test needs a controller, that test needs a browser. Test the piece the controller calls instead:
// Instead of building the whole settings form, check the element it adds $field = new \IPS\Helpers\Form\Sort( 'myapp_order', $order, FALSE, array( 'checkboxes' => 'myapp_' ) ); Test widgets through __toString(), not render()
__toString() runs init(), _render() and the caching layer — the whole path the theme actually uses. Calling render() directly bypasses all of it and proves considerably less.
Reproduce the consumer, not your own method
The most valuable CLI tests are the ones that imitate what core does with your code, because that is where extension-point mistakes surface. Calling your own method and getting a sensible answer back proves very little.
/* This is the test that catches a broken core/Loader extension: it reproduces the dispatcher's merge loop across every installed application. */ foreach ( Application::allExtensions( 'core', 'Loader' ) as $key => $loader ) { foreach ( $loader->js() as $js ) { if ( !is_array( $js ) ) { say( 'BREAK ' . $key . ' returns a flat list and will 500 every page' ); } } } The same idea covers language keys an ACP screen will demand, extension classes that must exist, and any contract core enforces at runtime rather than at install.
Static checks catch what CLI cannot run
Where the failure is inside a controller, read the source instead of executing it. Scanning for 'app' => ..., 'key' => ... pairs on every Form\Editor and confirming the named EditorLocations file exists takes a second and catches a bug that otherwise only appears when a human opens the page.
Verified against
Invision Community 5.0.19, by building and testing applications against it.
Related: S3 and R2 storage break in any CLI script without HTTPS=on, Your background task is running inside a visitor's page load, Application versions: why re-uploading the same version does nothing

0 comments
28 views

Ernest Defoe

July 26Jul 26

addToStack() often returns a placeholder, not translated text

Member::loggedIn()->language()->addToStack( 'my_key' ) looks like a function that returns a translated string. Sometimes it is. Often it returns a deferred placeholder that is substituted later, when the output is rendered.
What actually happens
If the language words are already loaded, you get real text. If they are not, the key is added to an output stack and a placeholder comes back:
public function addToStack( string $key, ?bool $vle = TRUE, array $options = array() ): string Rendering later calls parseOutputForDisplay(), which walks the output and swaps placeholders for text. Note its signature — it takes the output by reference:
public function parseOutputForDisplay( mixed &$output ): void The trap: storing the result
Anything that takes the return value and puts it somewhere that is not rendered through the normal output path stores the placeholder. Written to the database, it stays a placeholder for ever:
// WRONG - may store a placeholder, not a title $title = $lang->addToStack( 'myapp_generated_title', FALSE, array( 'sprintf' => array( $name ) ) ); Post::create( $item, $title ); // RIGHT - force resolution first $title = $lang->addToStack( 'myapp_generated_title', FALSE, array( 'sprintf' => array( $name ) ) ); $lang->parseOutputForDisplay( $title ); Post::create( $item, $title ); This bites hardest in background tasks and queue jobs, where no page is being rendered and nothing calls the parser for you.
The second trap: escaping the placeholder is not escaping the text
If a value is interpolated into a language string, escaping the result of addToStack() achieves nothing — at that moment the result may be a placeholder, and the real value is substituted raw afterwards.
// WRONG - escapes a placeholder; the title is substituted raw later $out = htmlspecialchars( $lang->addToStack( 'x_posted_y', FALSE, array( 'sprintf' => array( $userTitle ) ) ) ); // RIGHT - escape the ARGUMENT going in $out = $lang->addToStack( 'x_posted_y', FALSE, array( 'sprintf' => array( htmlspecialchars( $userTitle, ENT_QUOTES, 'UTF-8' ) ) ) ); Escape what goes into the string, never what comes out.
Verified against
Invision Community 5.0.19, by reading system/Lang/Lang.php.
Related: Creating a language in code: set_short() does not store the locale, Why your app shows menutab__content or module__myapp_thing to customers, A module shows as module__myapp_thing on the administrator restrictions screen

0 comments
27 views

Ernest Defoe

July 26Jul 26

Db::insert() third and fourth arguments do very different things

Two boolean flags in a row, easy to transpose, and they mean opposite things. Getting them the wrong way round produces data corruption rather than an error.
The signature
public function insert( string $table, Select|array $set, bool $odkUpdate = FALSE, // 3rd: ON DUPLICATE KEY UPDATE bool $ignoreErrors = FALSE // 4th: INSERT IGNORE ): int|string|mysqli_stmt What each does
CallOn a duplicate key insert( $t, $v )Throws insert( $t, $v, TRUE )Overwrites the existing row with your values insert( $t, $v, FALSE, TRUE )Keeps the existing row and does nothing Why transposing them is worse than an error
Say you keep a counter and want "insert if new, otherwise leave alone":
// WRONG - resets the counter every time Db::i()->insert( 'myapp_counts', array( 'key' => $k, 'uses' => 1, 'ignored' => 0 ), TRUE ); // RIGHT - insert if new, otherwise leave the existing row alone Db::i()->insert( 'myapp_counts', array( 'key' => $k, 'uses' => 1, 'ignored' => 0 ), FALSE, TRUE ); The wrong version runs without complaint and silently resets uses to 1 and ignored to 0 on every call. Nothing is logged, nothing throws, and the bug only shows up as counters that never climb.
Which one you actually want
Caching a computed value that should refresh — ON DUPLICATE KEY UPDATE (3rd argument). Recording that something happened, where the first record is the true one — INSERT IGNORE (4th argument). Genuinely new rows only — neither, and let it throw. A duplicate key is information. If you want neither semantic, use Db::i()->replace(), which is a real REPLACE INTO — note that deletes and re-inserts, so any column you omit reverts to its default.
Verified against
Invision Community 5.0.19, by reading system/Db/Db.php.
Related: The ?: fallback silently breaks any setting an admin can set to zero, Dumping an Invision Community database from PHP: ANSI_QUOTES, multi-member gzip, and the 0x trap

0 comments
26 views

Ernest Defoe

July 26Jul 26

Event listeners: all 58 hooks, and the argument-order trap

Invision Community lets an application react to things that happen elsewhere in the suite — a member registering, a topic being posted, an invoice being paid — by registering a listener. There are 58 hooks across eight listener types.
They are not documented anywhere. The only record that a hook exists is an @method annotation in the abstract class, so this page is a transcription of those, checked against the code that fires them.
Registering a listener
data/listeners.json in your application, keyed by class name:
{ "MemberEvents": { "type": "MemberListenerType", "classname": "IPS\\myapp\\listeners\\MemberEvents", "extends": "IPS\\Member" } } Then the class itself. Implement only the hooks you want — the rest are inherited and do nothing.
namespace IPS\myapp\listeners; use IPS\Events\ListenerType\MemberListenerType; use IPS\Member; class MemberEvents extends MemberListenerType { public static string $class = 'IPS\Member'; public function onCreateAccount( Member $member ): void { // ... } } Every page on the site breaks if you omit $class
Event::loadListeners() calls getExtendedClass() on every registered listener from every application. A listener without public static string $class therefore does not break your application — it breaks the entire site, on every request.
The error names the dispatcher, not the listener that caused it, so there is nothing in the trace pointing at the responsible app.
The first argument is not what you expect
This is the trap that matters most, because it fails silently.
Hooks are fired like this:
Event::fire( 'onPaid', $item, array( $this ) ); The subject is passed first and everything else follows. For member and content hooks that reads naturally — the member or the content object comes first. For Commerce package hooks it does not: the first argument is the InvoiceItem, and the object you actually want is the second.
// WRONG - and it will not error public function onPaid( $purchase ): void { // $purchase is really the INVOICE ITEM } // RIGHT public function onPaid( $item, $invoice = null ): void { $member = $invoice->member; // Invoice::get_member() returns a Customer } PHP discards extra arguments to a non-variadic method, so the wrong signature raises nothing at all. You get an object of the wrong type, your property lookups return null, and the feature simply never runs. No log entry, no exception, no clue.
Note which object arrives second, because it differs by hook:
onPaid, onUnpaid, onAddToInvoice, onInvoiceCancel — the Invoice onRenew, onExpire, onExpireWarning, onCancel, onReactivate, onChange, onTransfer, onDelete — the Purchase onPurchaseGenerated — the Purchase, then the Invoice Both Invoice and Purchase expose member, so one accessor covers either once you are holding the right object.
Listeners run inside the operation, so never throw
A listener runs in the middle of somebody registering an account or paying an invoice. An uncaught exception in your hook becomes their failed registration. Wrap the body and swallow:
public function onCreateAccount( Member $member ): void { try { // your work } catch ( \Throwable $e ) { // never let an extension be the reason a registration fails } } For the same reason, do not do slow work here. Queue it.
The catalogue
Signatures below are transcribed from the @method annotations on each listener type. Class names are shortened for readability.
MemberListenerType — 22 hooks
Extends IPS\Member.
onCreateAccount( Member $member ) onValidate( Member $member ) onLogin( Member $member ) onLogout( Member $member, Url $returnUrl ) onProfileUpdate( Member $member, array $changes ) onSetAsSpammer( Member $member ) onUnSetAsSpammer( Member $member ) onMerge( Member $member, Member $member2 ) onDelete( Member $member ) onEmailChange( Member $member, string $new, string $old ) onPassChange( Member $member, string $new ) onJoinClub( Member $member, Club $club ) onLeaveClub( Member $member, Club $club ) onEventRsvp( Member $member, Event $event, int $response ) onReact( Member $member, Content $content, Reaction $reaction ) onUnreact( Member $member, Content $content ) onFollow( Member $member, object $object, bool $isAnonymous ) onUnfollow( Member $member, object $object ) onCourseComplete( Member $member, Course $course ) onModuleComplete( Member $member, Module $module ) onLessonComplete( Member $member, Lesson $lesson ) onQuizComplete( Member $member, Quiz $quiz ) ContentListenerType — 10 hooks
Extends any content class, e.g. IPS\forums\Topic.
onBeforeCreateOrEdit( Content $object, array $values, bool $new = FALSE ) onCreateOrEdit( Content $object, array $values, bool $new = FALSE ) onDelete( Content $object ) onStatusChange( Content $object, string $action ) onMerge( Content $object, array $items ) onItemMove( Item $item, Model $oldContainer, bool $keepLink = FALSE ) onCommentMove( Comment $comment, Item $oldItem, bool $skip = FALSE ) onItemView( Item $item ) onItemSplit( Item $item, Item $oldItem ) onReport( Content $object, Report $report ) PackageListenerType — 13 hooks
Extends IPS\nexus\Invoice\Item.
onAddToInvoice( InvoiceItem $item, Invoice $invoice ) onPaid( InvoiceItem $item, Invoice $invoice ) onUnpaid( InvoiceItem $item, Invoice $invoice, string $status ) onInvoiceCancel( InvoiceItem $item, Invoice $invoice ) onPurchaseGenerated( InvoiceItem $item, Purchase $purchase, Invoice $invoice ) onRenew( InvoiceItem $item, Purchase $purchase, int $cycles ) onExpireWarning( InvoiceItem $item, Purchase $purchase ) onExpire( InvoiceItem $item, Purchase $purchase ) onCancel( InvoiceItem $item, Purchase $purchase ) onReactivate( InvoiceItem $item, Purchase $purchase ) onChange( InvoiceItem $item, Purchase $purchase, Package $newPackage, int|RenewalTerm $chosenRenewalOption = NULL ) onTransfer( InvoiceItem $item, Purchase $purchase, Member $newCustomer ) onDelete( InvoiceItem $item, Purchase $purchase ) PollListenerType — 5 hooks
Extends IPS\Poll.
onCreateOrEdit( Poll $poll ) onDelete( Poll $poll ) onVote( Poll $poll, Vote $vote ) onVoteRecount( Poll $poll ) onStateChange( Poll $poll, string $state ) InvoiceListenerType — 3 hooks
Extends IPS\nexus\Invoice.
onStatusChange( Invoice $invoice, string $status ) onCreateAccountForGuest( Invoice $invoice, Member $member, array $guestData ) onCheckout( Invoice $invoice, string $step ) ClubListenerType — 3 hooks
Extends IPS\Member\Club.
onCreate( Club $club ) onEdit( ?Club $club ) onDelete( Club $club ) FileListenerType — 1 hooks
Extends IPS\File.
onDownload( File $file, object $content, array $extra ) NodeListenerType — 1 hooks
Extends IPS\Node\Model.
onDelete( Model $node ) Finding hooks yourself
grep -h '@method' system/Events/ListenerType/*.php That is the whole index. If a hook is not in that output, it does not exist.
Verified against
Invision Community 5.0.19, by reading system/Events/ListenerType/*.php and the code that fires each hook. Argument order for the Commerce hooks was confirmed against applications/nexus/sources/Invoice/Invoice.php.
Related: One content listener for every content type, including Pages databases, A content listener without $class takes down your whole community, Extension method signatures are fixed, and a mismatch is a fatal on install

0 comments
28 views

Ernest Defoe

July 26Jul 26

Extensions, themes & support for Flarum and Invision Community

Things that cost me a day, so they cost you none

Invision Community 5

Extensions and contracts

Languages and text

Theming, templates and forms

Background work and scheduled tasks

Data, settings and storage

AI features and expectations

Application structure and releases

Realtime, chat and calls

Flarum 2

Configure browser push notifications

Chrome (Android)

Chrome (Desktop)

Safari (iOS 16.4+)

Safari (macOS)

Edge (Android)

Edge (Desktop)

Firefox (Android)

Firefox (Desktop)