Skip to content
View in the app

A better way to browse. Learn more.

ernestdefoe.online

A full-screen app on your home screen with push notifications, badges and more.

To install this app on iOS and iPadOS
  1. Tap the Share icon in Safari
  2. Scroll the menu and tap Add to Home Screen.
  3. Tap Add in the top-right corner.
To install this app on Android
  1. Tap the 3-dot menu (⋮) in the top-right corner of the browser.
  2. Tap Add to Home screen or Install app.
  3. Confirm by tapping Install.
ernestdefoe.online

Extensions, themes & support for Flarum and Invision Community

Vibe coding for the community web. Report a bug, request a feature, or dig into the source — this is where the tools you use get built, in the open.

We do custom Bespoke Invision Community apps. If you have an idea for something you want then use the contact form to get in touch with us.
Knowledge base

Things that cost me a day, so they cost you none

Working notes from building Invision Community and Flarum applications. Mostly the failures that give no error at all — the ones where everything installs cleanly and quietly does the wrong thing.

92 articles

Invision Community 5

86 articles

Extensions and contracts

39

What each extension point is for, what it must declare, and what happens when it is wrong — which is usually nothing visible.

Languages and text

5

The string table, translation, and the places where text does not appear where you expected it to.

Theming, templates and forms

9

Theme hooks, CSS that survives both colour schemes, and building forms that do not throw on render.

Background work and scheduled tasks

5

The queue system, work that has to happen after the response, and jobs that finish without doing anything.

Data, settings and storage

11

The database layer, settings, tags, file storage, and backing up a live site.

AI features and expectations

5

What these features do, what they cost, and what buyers reasonably but wrongly assume they do.

Application structure and releases

11

The JSON files an application is made of, versioning and upgrade steps, and testing from the command line.

Realtime, chat and calls

1

WebSocket gateways, relays and the server-side pieces live features depend on — where "it works when I test it" and "it works for your members" are different claims.

Nothing matches that.

The core/Sitemap extension in Invision Community 5

A core/Sitemap extension contributes URLs to the XML sitemap that sitemap.php serves, and adds the per-app fields to ACP → Promotion → Search Engine Optimisation → Sitemap. Core calls it from three places: the hourly core/sitemapgenerator task (via IPS\Sitemap::buildNextSitemap()), the core/RebuildSitemap background queue task fired by the ACP "Rebuild Sitemap" button, and the ACP SEO controller itself when it builds and saves the settings form.

The contract

All four methods in IPS\Extensions\SitemapAbstract are abstract. There are no optional methods and the abstract declares no @throws at all — so there is no exception core is documented to expect from you, and (as the traps below show) almost none that it catches.

namespace IPS\Extensions;

abstract class SitemapAbstract
{
    /* Merged into the ACP "recommended settings" pool and compared field-by-field */
    public array $recommendedSettings = array();

    /* Form fields for the ACP Sitemap tab. Keys MUST match keys in $recommendedSettings. */
    abstract public function settings(): array;

    /* Called once per extension with the WHOLE form's values, including every other
       app's fields and the always-present 'sitemap_configuration_info' YesNo. */
    abstract public function saveSettings( array $values ) : void;

    /* Every sitemap file this extension owns, as a flat array of strings.
       These become the primary key values in core_sitemap.sitemap (VARCHAR 255). */
    abstract public function getFilenames(): array;

    /* Build one file. $filename is one of the strings getFilenames() returned.
       The return value is DISCARDED by both callers - see the trap below. */
    abstract public function generateSitemap( string $filename, Sitemap $sitemap ) : ?int;
}

The only way to emit anything is IPS\Sitemap::buildSitemapFile(), which writes the XML into the core_sitemap table:

public function buildSitemapFile(
    string $filename,
    array  $entries,
    int    $lastId = 0,
    array  $namespaces = array()   /* e.g. array( 'image' => 'https://www.google.com/schemas/sitemap-image/1.1' ) */
) : void

Each entry in $entries is an array. $entry['url'] is read with no isset() guard — it is mandatory. lastmod, priority and changefreq are optional and are each only written when the value is truthy (isset( $entry[...] ) AND $entry[...]). lastmod may be an IPS\DateTime or a raw unix timestamp; anything else is fed to DateTime::ts() and will produce a nonsense date rather than an error. core_sitemap has four columns — sitemap (VARCHAR 255, PRIMARY KEY on the first 191 characters), data (MEDIUMTEXT, nullable), updated (INT), last_id (BIGINT UNSIGNED).

Batching is entirely your responsibility. Core has one constant, \IPS\SITEMAP_MAX_PER_FILE (default 500, defined in init.php), and one storage slot, core_sitemap.last_id. Every core implementation follows the same unwritten convention: name your files <prefix>_1, <prefix>_2, … , explode('_')/array_pop() the block number back off $filename in generateSitemap(), and read the previous block's last_id out of core_sitemap to use as a cursor. Nothing in core enforces this; nothing in core does it for you.

A minimal example

First, register the extension. In IC5 an extension file that is not listed in data/extensions.json is invisible — IPS\Application::extensions() reads only that JSON file and never scans the directory.

/* applications/myapp/data/extensions.json */
{
    "core": {
        "Sitemap": {
            "Listings": "IPS\\myapp\\extensions\\core\\Sitemap\\Listings"
        }
    }
}
/* applications/myapp/data/settings.json */
[
    { "key": "sitemap_myapp_listings", "default": "", "report": "none" }
]
/* applications/myapp/dev/lang.php - the __admin__ pack */
'sitemap_myapp_Listings'            => "Listings",   /* {appdir}_{extension key} */
'sitemap_myapp_listings_include'    => "Include in sitemap",
'sitemap_myapp_listings_priority'   => "Priority",
<?php
/* applications/myapp/extensions/core/Sitemap/Listings.php */

namespace IPS\myapp\extensions\core\Sitemap;

use Exception;
use IPS\Db;
use IPS\Extensions\SitemapAbstract;
use IPS\Helpers\Form\Select;
use IPS\Helpers\Form\YesNo;
use IPS\Member;
use IPS\myapp\Listing;
use IPS\Settings;
use IPS\Sitemap;
use UnderflowException;
use function defined;
use const IPS\SITEMAP_MAX_PER_FILE;

if ( !defined( '\IPS\SUITE_UNIQUE_KEY' ) )
{
    header( ( $_SERVER['SERVER_PROTOCOL'] ?? 'HTTP/1.0' ) . ' 403 Forbidden' );
    exit;
}

class Listings extends SitemapAbstract
{
    /* Declared as a literal. Do NOT populate this inside settings() - see the trap below. */
    public array $recommendedSettings = array(
        'sitemap_myapp_listings_include'  => true,
        'sitemap_myapp_listings_priority' => '0.8'
    );

    /* getFilenames() is called many times per task run - memoise it. */
    protected ?array $filenames = null;

    /* Core constructs extensions with new $class( $member ) where $member is
       IPS\Member (guest) from the task, or NULL from the ACP. Make it optional. */
    public function __construct( ?Member $member = null ) {}

    protected function config() : array
    {
        return Settings::i()->sitemap_myapp_listings
            ? json_decode( Settings::i()->sitemap_myapp_listings, true )
            : array();
    }

    public function settings(): array
    {
        $settings = $this->config();
        $return   = array();

        /* Every key here must also exist in $recommendedSettings, and the 8th
           constructor argument (htmlId) is required for the toggles to work. */
        $return['sitemap_myapp_listings_include'] = new YesNo(
            'sitemap_myapp_listings_include',
            $settings['sitemap_myapp_listings_include'] ?? $this->recommendedSettings['sitemap_myapp_listings_include'],
            false,
            array( 'togglesOn' => array( 'sitemap_myapp_listings_priority' ) ),
            null, null, null,
            'sitemap_myapp_listings_include'
        );

        /* Deliberately NOT using 'unlimited' => '-1' here - see the priority trap. */
        $return['sitemap_myapp_listings_priority'] = new Select(
            'sitemap_myapp_listings_priority',
            $settings['sitemap_myapp_listings_priority'] ?? $this->recommendedSettings['sitemap_myapp_listings_priority'],
            false,
            array( 'options' => Sitemap::$priorities ),
            null, null, null,
            'sitemap_myapp_listings_priority'
        );

        return $return;
    }

    public function saveSettings( array $values ) : void
    {
        /* $values is the ENTIRE form - sitemap_url, every other app's fields, and
           sitemap_configuration_info, which is always present. Never throw here. */
        if ( $values['sitemap_configuration_info'] )
        {
            $save = $this->recommendedSettings;
        }
        else
        {
            $save = array(
                'sitemap_myapp_listings_include'  => (bool) $values['sitemap_myapp_listings_include'],
                'sitemap_myapp_listings_priority' => (string) $values['sitemap_myapp_listings_priority']
            );
        }

        Settings::i()->changeValues( array( 'sitemap_myapp_listings' => json_encode( $save ) ) );
    }

    public function getFilenames(): array
    {
        if ( $this->filenames !== null )
        {
            return $this->filenames;
        }

        $settings = $this->config();
        if ( isset( $settings['sitemap_myapp_listings_include'] ) and !$settings['sitemap_myapp_listings_include'] )
        {
            return $this->filenames = array();
        }

        try
        {
            $count = (int) Db::i()->select( 'COUNT(*)', 'myapp_listings', array( 'listing_open=?', 1 ) )->first();
        }
        catch ( Exception $e )
        {
            /* Never throw, and never return array() on a transient failure - core deletes
               every core_sitemap row that is not in the returned list. Fall back to
               whatever we have already built. */
            return $this->filenames = iterator_to_array(
                Db::i()->select( 'sitemap', 'core_sitemap', array( "sitemap LIKE 'sitemap_myapp_listings_%'" ) )
            );
        }

        $files = array();
        for ( $i = 1; $i <= ceil( $count / SITEMAP_MAX_PER_FILE ); $i++ )
        {
            $files[] = 'sitemap_myapp_listings_' . $i;
        }

        return $this->filenames = $files;
    }

    public function generateSitemap( string $filename, Sitemap $sitemap ) : ?int
    {
        $settings = $this->config();
        $priority = (string) ( $settings['sitemap_myapp_listings_priority']
                               ?? $this->recommendedSettings['sitemap_myapp_listings_priority'] );

        $exploded = explode( '_', $filename );
        $block    = (int) array_pop( $exploded );

        $entries     = array();
        $lastId      = 0;
        $where       = array( array( 'listing_open=?', 1 ) );
        $limitClause = array( ( $block - 1 ) * SITEMAP_MAX_PER_FILE, SITEMAP_MAX_PER_FILE );

        /* Cursor: the previous block stored its highest ID in core_sitemap.last_id */
        try
        {
            $previous = (int) Db::i()->select(
                'last_id',
                'core_sitemap',
                array( 'sitemap=?', implode( '_', $exploded ) . '_' . ( $block - 1 ) )
            )->first();

            if ( $previous > 0 )
            {
                $where[]     = array( 'listing_id > ?', $previous );
                $limitClause = SITEMAP_MAX_PER_FILE;
            }
        }
        catch ( UnderflowException $e ) {}

        $guest = new Member;

        foreach ( Db::i()->select( '*', 'myapp_listings', $where, 'listing_id ASC', $limitClause ) as $row )
        {
            $lastId = (int) $row['listing_id'];   /* advance even when skipping */

            $item = Listing::constructFromData( $row );
            if ( !$item->canView( $guest ) )
            {
                continue;
            }

            $entries[] = array(
                'url'      => (string) $item->url(),
                'lastmod'  => (int) $row['listing_updated'],
                'priority' => $priority          /* string, never intval()'d */
            );
        }

        /* ALWAYS call this, even with zero entries, or the file is re-queued forever. */
        $sitemap->buildSitemapFile( $filename, $entries, $lastId );

        return $lastId;
    }
}

My extension does not appear on the ACP sitemap page at all

No error, no log entry, no header on the Sitemap tab. Two independent causes, both silent.

First, Application::extensions() in IC5 reads only applications/<app>/data/extensions.json. If your class is not listed under {"core": {"Sitemap": {...}}}, or if class_exists() on the listed name returns false, the loop does continue with no message — the comment in core literally reads "Switching between branches confuses extensions".

Second, Application::allExtensions() caches the resolved class list in Store::i()->extensions and only rebuilds it when the Sitemap key is missing from that store. Adding a new extension to a live site does nothing until the datastore is cleared (ACP → System → Support → Clear cache, or unset( Store::i()->extensions )).

Also note that allExtensions() skips every non-IPS application when \IPS\RECOVERY_MODE is on. If a site is in recovery mode, no third-party URLs are contributed and the sitemap silently shrinks.

My extension is listed but silently disappears from the generated sitemap

Extensions are instantiated by Application::constructExtensionClass():

$obj = new $classToUse( $checkAccess === TRUE ? Member::loggedIn()
                        : ( $checkAccess === FALSE ? NULL : $checkAccess ) );
/* ... */
catch( RuntimeException | OutOfRangeException $e ){}
return null;

Two consequences. Your constructor is always called with one argument, and it is not the same argument in both call paths: IPS\Sitemap::buildNextSitemap() and RebuildSitemap pass allExtensions( 'core', 'Sitemap', new Member, 'core' ), so you get a guest IPS\Member; the ACP SEO controller passes FALSE, so you get NULL. A constructor declared __construct( Member $member ) works on the task and throws a TypeError in the ACP. Declare it __construct( ?Member $member = null ), or do not declare one at all — SitemapAbstract has no constructor, and PHP tolerates extra arguments to user-defined functions.

And if your constructor throws RuntimeException or OutOfRangeException, the object is dropped and null is returned with no log line anywhere. Your extension simply stops existing. This is the failure mode where nothing at all is written to core_sitemap and there is no evidence to search for.

The ACP shows a header reading "sitemap_myapp_Listings"

And the rebuild progress bar reads "Rebuilding sitemap_myapp_Listings sitemap". The extension key core builds is {application directory}_{extension key from extensions.json}. In applications/core/modules/admin/promotion/seo.php the header is added as $form->addHeader( 'sitemap_' . $extKey ), and in RebuildSitemap::getProgress() the same key is used: $key = 'sitemap_' . $data['extensionKey']. Core ships sitemap_core_Clubs, sitemap_cms_Pages, sitemap_cms_Databases, sitemap_gallery_Images, sitemap_gallery_Videos and sitemap_nexus_Subscriptions in its language packs for exactly this reason. addToStack() on a missing key returns the key itself, so you get the raw string rather than an error.

Define sitemap_{appdir}_{ExtensionKey} in your admin language pack. The header is only rendered when settings() returned at least one field, but the progress-bar key is used regardless.

"Undefined array key" warning on ACP → Promotion → Search Engine Optimisation

The Sitemap tab errors and you cannot reach the settings at all. The form builder does this, with no isset():

foreach ( $extension->settings() as $k => $setting )
{
    if ( $setting->value != $extension->recommendedSettings[ $k ] )
    {
        $useRecommendedSettings = FALSE;
    }
    /* ... */
}

Every key returned by settings() must also exist in $recommendedSettings. Returning a numerically-indexed array of form fields — which is the natural thing to do, since the array is only ever iterated for its values afterwards — will produce Undefined array key 0. Every core implementation returns a map keyed identically to its recommendedSettings; gallery's Images and Videos sidestep the issue by returning an empty array from both.

Note where this surfaces: it is a shared screen. A bad key in your app breaks the SEO settings page for every app, and the stack trace names seo.php, not your extension.

"Use recommended settings" saves the wrong values for my app

The admin flips sitemap_configuration_info to Yes, saves, and your extension's recommended defaults are not applied. The cause is statement order in _manageSitemap():

$recommendedSettings = array_merge( $recommendedSettings, $extension->recommendedSettings );
foreach ( $extension->settings() as $k => $setting )  /* <- settings() runs AFTER the merge */

and later, on save, $values = array_merge( $values, $recommendedSettings ). If you populate $this->recommendedSettings inside settings(), the merge has already happened and your defaults never enter the pool. IPS\core\extensions\core\Sitemap\Content does exactly this — it fills recommendedSettings in the body of settings() — and gets away with it only because its own saveSettings() ignores $recommendedSettings entirely and writes json_encode( array() ) in the recommended branch.

Declare public array $recommendedSettings as a literal property, the way Clubs, Pages, Databases and Subscriptions do. If your defaults genuinely have to be computed, do it in the constructor, not in settings().

My settings fields stay visible when "use recommended settings" is switched on

The master YesNo is built as new YesNo( 'sitemap_configuration_info', $useRecommendedSettings, FALSE, array( 'togglesOff' => $toggles ) ), and $toggles is filled with $setting->htmlId. In IPS\Helpers\Form\FormAbstract, public ?string $htmlId = NULL; and it is only set from the eighth constructor argument:

$this->htmlId = $id ? preg_replace( "/[^a-zA-Z0-9\-_]/", "_", $id ) : NULL;

Omit it and $toggles collects a NULL, the toggle targets nothing, and your rows stay on screen. Every core Sitemap extension passes the field name again as the eighth argument for this reason. The argument list is ( $name, $default, $required, $options, $validation, $prefix, $suffix, $id ) — you must pass four NULLs to reach it.

Saving the sitemap settings page applies some apps' settings but not others

The save loop has no protection whatsoever:

foreach( Application::allExtensions( 'core', 'Sitemap', FALSE, 'core' ) as $extKey => $extension )
{
    $extension->saveSettings( $values );
}

An exception from your saveSettings() aborts the loop. Extensions iterated before yours have already written their settings; extensions after yours never run; and $form->saveAsSettings() for sitemap_url and the ACP audit-log entry (acplogs__seo_sitemap_settings) are both skipped. The admin sees a suite-level error and a half-saved form.

Remember also that $values is the whole form, not your slice of it: it contains sitemap_url, sitemap_configuration_info and every other app's fields. Only read keys you defined, and always handle the $values['sitemap_configuration_info'] branch — it is present on every save.

Nobody's sitemap updates any more, and nothing in my app logs an error

This is the worst one. IPS\Sitemap::buildNextSitemap() calls your getFilenames() with no try/catch, twice over:

foreach ( $extensions as $extension )
{
    $files = array_merge( $files, $extension->getFilenames() );   /* no try/catch */
}
/* ... */
foreach( $extensions as $extension )
{
    if( in_array( $toBuild, $extension->getFilenames() ) )        /* no try/catch */
    {
        $extension->generateSitemap( $toBuild, $this );           /* no try/catch */
    }
}

The only handler is one level up, in applications/core/tasks/sitemapgenerator.php:

catch( Exception $e )
{
    Log::log( $e, 'sitemap_generator' );
    $generator->log[] = $e->getMessage();
    return FALSE;   /* stops runUntilTimeout for this whole run */
}

Because the filename union is built before any file is generated, a throw from your getFilenames() means zero sitemap files are built that hour, for every application on the site. And it does not resolve itself: the task runs on a 60-minute schedule (P0Y0M0DT0H60M0S), constructs a fresh IPS\Sitemap each time, and re-derives the list from scratch — so it hits your exception again immediately, forever.

The error surfaces nowhere near its cause. The only trace is a row in ACP → System → Logs under the key sitemap_generator, plus the message attached to the sitemapgenerator task's log. Nothing points at your extension, and the visible symptom is that some other app's sitemap has gone stale. Worse, catch( Exception $e ) does not catch \Error — a TypeError or a call to an undefined method inside getFilenames() is not logged at all and takes the task down.

Wrap the whole body of getFilenames() in your own try/catch ( \Throwable ) and return something sane.

sitemap.php?file=... returns 404 and my previously built files have vanished

The exact response is sitemap_not_found with error code 2C152/1, raised in applications/core/modules/front/sitemap/sitemap.php. The cause is upstream, in the first block of buildNextSitemap():

Db::i()->delete( 'core_sitemap', Db::i()->in( 'sitemap', $files, TRUE ) );

Once per task run, core deletes every row in core_sitemap whose name is not in the union of all extensions' current getFilenames() output. If your implementation returns a shorter list than last hour — because a count query failed, because a cache was cold, because a permission check flipped — those rows are gone, along with their last_id cursors, and the URLs 404 until the file is rebuilt from block 1.

There is a sharper edge here. Db::in() returns the literal string '1=1' when the value array is empty and $reverse is true:

return $reverse ? '1=1' : '1=0';

So if every Sitemap extension returns an empty array on a given run, the delete becomes DELETE FROM core_sitemap WHERE 1=1 and truncates the table.

Make getFilenames() deterministic and total. Return the same list for the same data, never let it throw, and on a transient failure return the filenames already recorded in core_sitemap for your prefix rather than an empty array.

Two apps' entries overwrite each other in the same file

core_sitemap.sitemap is a single flat namespace with a PRIMARY KEY over the first 191 characters — there is no application column. When core picks a file to build it calls every extension that claims it, with no break:

foreach( $extensions as $extension )
{
    if( in_array( $toBuild, $extension->getFilenames() ) )
    {
        $extension->generateSitemap( $toBuild, $this );
    }
}

and buildSitemapFile() finishes with Db::i()->replace( 'core_sitemap', ... ). Two extensions returning the same string both run, both replace the same row, and the one that runs later wins. No error, no warning — one app's URLs simply never appear. Note that iteration order is not stable in a useful way either: allExtensions() is called with $firstApp = 'core', which only guarantees that core sorts first.

Prefix every filename with your application directory, and keep the whole string under 191 characters. Watch out for prefixes that are substrings of each other, and for per-record prefixes: core's own Clubs extension emits {clubId}_sitemap_clubs_pages_{n}, and Databases emits {databaseId}_sitemap_database_records_{n}.

My file is generated every hour but never appears in the sitemap index

Two separate causes, and they look identical from outside.

If $entries is empty, buildSitemapFile() stores data => NULL, and the index builder selects with array( 'data IS NOT NULL' ). The row exists, its updated timestamp keeps refreshing, and the file never appears in <sitemapindex>. That is intended — an empty file is skipped, not published.

If generateSitemap() returns without calling buildSitemapFile() at all — the early-return pattern that Content, Images, Videos, Pages and Databases all use when their count setting is zero — then no row is ever written. That filename stays in array_diff( $files, ...existing rows... ) on every single run, so core picks it as an unbuilt file first, every hour, forever, and no other file ever gets a turn. The build makes no progress and the log is empty.

Always call buildSitemapFile() before returning, even with array(). If you want a file not to exist, do not return its name from getFilenames().

The <priority> element is missing, or contains -1

Sitemap::$priorities holds strings: '1.0' => '1.0' through '0.1' => '0.1'. The Select field therefore returns a string. Core's content extensions then do:

$priority = ( $item->sitemapPriority() ?: ( intval( $settings["sitemap_{$class::$title}_priority"] ?? 1 ) ) );
if ( $priority !== -1 )
{
    $data['priority'] = $priority;
}

intval( '0.8' ) is 0. And buildSitemapFile() only writes the element when the value is truthy (isset( $entry['priority'] ) AND $entry['priority']), so 0 is dropped. The net effect in core is that every priority between 0.1 and 0.9 produces no <priority> element at all — including Content::RECOMMENDED_NODE_PRIORIY, which is 0.6. Only 1.0 survives, as <priority>1</priority>.

The opposite mistake is in Clubs and Subscriptions, which skip intval() but compare strictly:

$priority = $settings['sitemap_club_priority'] ?? 1;
if ( $priority !== -1 ) { $data['priority'] = $priority; }

Those fields are declared with 'unlimited' => '-1', 'unlimitedLang' => 'sitemap_dont_include', and Select::getValue() returns $this->options['unlimited'] verbatim — the string '-1'. '-1' !== -1 is true, so choosing "Don't include" writes <priority>-1</priority> into the served XML, which is outside the 0.0–1.0 range the sitemap schema allows.

Do neither. Keep the value as a string, and if you offer a "don't include" option compare it loosely or against the string: if ( (string) $priority !== '-1' ).

Returning the last ID from generateSitemap() has no effect

The abstract declares : ?int and the docblock calls it the last built ID, but both callers discard it:

/* IPS\Sitemap::buildNextSitemap() */
$extension->generateSitemap( $toBuild, $this );

/* IPS\core\extensions\core\Queue\RebuildSitemap::run() */
$extension->generateSitemap( $name, $sitemap );
return $offset + 1;

The only thing that persists a cursor is the third argument to buildSitemapFile(), which writes core_sitemap.last_id. If you return the ID but pass 0 (or omit the argument, as cms\Pages and cms\Databases do), last_id stays 0 and every subsequent block falls back to LIMIT offset, 500 paging.

Two consequences for batching. First, block n reads block n-1's row, so block 1 always throws UnderflowException and must fall back to offset paging — catch it, as every core implementation does. Second, advance your cursor even for rows you skip: Content updates $lastId before continue-ing on an unviewable item, precisely so that a block of 500 invisible items still moves the cursor forward instead of stalling.

Note too that buildNextSitemap() chooses files by "never built" first, then by updated ASC — it does not guarantee blocks are rebuilt in numerical order. A cursor read from block n-1 may be an hour old.

My image:, video: or news: elements never appear in the XML

Namespaced child elements are only processed inside if( count( $namespaces ) ). If you do not pass the fourth argument to buildSitemapFile(), every extra key in your entry array is skipped in complete silence — no warning, no malformed XML, just missing elements.

Three further undocumented requirements, all readable in the writer loop. The entry key must literally begin with the prefix plus a colon (mb_strpos( $key, $prefix . ':' ) === 0). The value must be an array — a scalar reaches foreach( $value as $k => $v ) and raises "foreach() argument must be of type array|object". And a nested value that is itself an array uses a positional convention: index 0 becomes the element text and every other key becomes an attribute. gallery's Videos extension is the only example of that last form in core:

$data['video:video'] = array(
    'video:content_loc' => (string) $url,
    'video:title'       => $item->mapped('title'),
    /* array( 0 => element text, 'attribute' => attribute value ) */
    'video:uploader'    => array( 0 => $item->author()->name, 'info' => (string) $item->author()->url() ),
);

$sitemap->buildSitemapFile(
    $filename, $entries, $lastId,
    array( 'video' => 'https://www.google.com/schemas/sitemap-video/1.1' )
);

One more thing about URLs: buildSitemapFile() rewrites $entry['url'] with an anchored preg_replace( '/^' . preg_quote( Settings::i()->base_url, '/' ) . '/', '{base_url}', ... ), and the front controller substitutes it back with str_replace( '<loc>{base_url}', ... ). Only the url key gets this treatment — namespaced values like image:loc are stored absolute. If your URL does not start with base_url character-for-character (different scheme, www mismatch, CDN host) the token is not applied and the absolute URL is baked into the stored XML, surviving any later change of base_url until the file is rebuilt.

Only the first few of my sitemap files ever get rebuilt

Two compounding budget problems.

The sitemapgenerator task runs hourly and calls runUntilTimeout(), which allows roughly 90 seconds (less if max_execution_time or MySQL's wait_timeout is lower). Every sitemap file on the entire site — yours, forums', gallery's, Pages' — competes for that one window. Files are eligible for rebuild once their updated is more than an hour old, ordered updated ASC, so they do rotate, but returning thousands of filenames from getFilenames() starves everybody.

The second problem is that getFilenames() is not called once — it is called for every extension, on every call to buildNextSitemap(), i.e. once per file built per extension. A default install has a Sitemap extension per routed content class (each generated by IPS\Content\ExtensionGenerator), so a run that builds twenty files can make several hundred getFilenames() calls. Core's own Content::getFilenames() runs permission-index joins and per-node $node->_items lookups on every one of them.

Memoise the result on the instance, as the example above does. The same object is reused for the whole run, so a single protected ?array $filenames cache removes the repeat cost entirely.

Clicking "Rebuild Sitemap" does nothing for my app

The ACP action deletes the whole of core_sitemap and then queues one core/RebuildSitemap background task per extension at priority 5. RebuildSitemap::preQueueData() then does:

$files = $extension->getFilenames();
$data['count'] = count( $files );

if( $data['count'] == 0 )
{
    return NULL;
}

and Task::queue() treats a NULL return as "nothing to do" — it calls postComplete( [], FALSE ) and inserts no queue row. So if getFilenames() is empty at the moment the button is pressed, your app is silently excluded from the rebuild, with no progress bar and no log line.

Note also that $data['files'] is snapshotted at queue time and run() reads $data['files'][$offset] from that snapshot. If your file list changes while the queue drains, the queue keeps building the old names. And run() calls generateSitemap() with no try/catch: in Task::runQueue(), only IPS\Task\Queue\OutOfRangeException (completion) and OutOfRangeException (rethrown as RuntimeException) are handled, so any other exception escapes before core_queue.offset is updated — the same row is retried on the next queue tick indefinitely.

Verified against

Read from Invision Community 5.0.19 source: system/Extensions/SitemapAbstract.php, system/Sitemap/Sitemap.php, system/Application/Application.php (allExtensions(), extensions(), constructExtensionClass()), system/Db/Db.php (in()), system/Helpers/Form/FormAbstract.php and Select.php, applications/core/tasks/sitemapgenerator.php, applications/core/extensions/core/Queue/RebuildSitemap.php, applications/core/modules/admin/promotion/seo.php, applications/core/modules/front/sitemap/sitemap.php, and all seven shipped implementations under applications/{core,cms,gallery,nexus}/extensions/core/Sitemap/. Schema details come from applications/core/data/schema.json and the SITEMAP_MAX_PER_FILE default from init.php.



User Feedback

Recommended Comments

There are no comments to display.

Account

Navigation

Search

Search

Configure browser push notifications

Chrome (Android)
  1. Tap the lock icon next to the address bar.
  2. Tap Permissions → Notifications.
  3. Adjust your preference.
Chrome (Desktop)
  1. Click the padlock icon in the address bar.
  2. Select Site settings.
  3. Find Notifications and adjust your preference.