Skip to content
View in the app

A better way to browse. Learn more.

ernestdefoe.online

A full-screen app on your home screen with push notifications, badges and more.

To install this app on iOS and iPadOS
  1. Tap the Share icon in Safari
  2. Scroll the menu and tap Add to Home Screen.
  3. Tap Add in the top-right corner.
To install this app on Android
  1. Tap the 3-dot menu (⋮) in the top-right corner of the browser.
  2. Tap Add to Home screen or Install app.
  3. Confirm by tapping Install.
ernestdefoe.online

Extensions, themes & support for Flarum and Invision Community

Vibe coding for the community web. Report a bug, request a feature, or dig into the source — this is where the tools you use get built, in the open.

We do custom Bespoke Invision Community apps. If you have an idea for something you want then use the contact form to get in touch with us.
Knowledge base

Things that cost me a day, so they cost you none

Working notes from building Invision Community and Flarum applications. Mostly the failures that give no error at all — the ones where everything installs cleanly and quietly does the wrong thing.

92 articles

Invision Community 5

86 articles

Extensions and contracts

39

What each extension point is for, what it must declare, and what happens when it is wrong — which is usually nothing visible.

Languages and text

5

The string table, translation, and the places where text does not appear where you expected it to.

Theming, templates and forms

9

Theme hooks, CSS that survives both colour schemes, and building forms that do not throw on render.

Background work and scheduled tasks

5

The queue system, work that has to happen after the response, and jobs that finish without doing anything.

Data, settings and storage

11

The database layer, settings, tags, file storage, and backing up a live site.

AI features and expectations

5

What these features do, what they cost, and what buyers reasonably but wrongly assume they do.

Application structure and releases

11

The JSON files an application is made of, versioning and upgrade steps, and testing from the command line.

Realtime, chat and calls

1

WebSocket gateways, relays and the server-side pieces live features depend on — where "it works when I test it" and "it works for your members" are different claims.

Nothing matches that.

The core/MemberExportPersonalInformation extension in Invision Community 5

The core/MemberExportPersonalInformation extension decides what goes into the XML file a community produces when someone asks for a copy of their personal data. There are two places on a site where that file is produced: a member can request their own copy from Account Settings → Privacy, an administrator approves the request, and the member then downloads it; or an administrator with the member_export_pi restriction can go to a member in the ACP and export the file directly. In both cases the browser is handed a document called <membername>_personal_information.xml. Every application that stores anything identifying about a member is expected to contribute a section to that file. Core's own extension contributes name, email, join date, IP addresses, known devices, terms acceptances, bulk-email opt-in history and profile fields flagged for PII export; Commerce contributes billing names, addresses, customer fields and the last four digits of stored cards. If your application holds member data and has no extension of this type, the exported file simply does not mention it, and nothing anywhere says so.

The Developer Center describes it in one line — applications/core/dev/lang.php:1297, key ext__MemberExportPersonalInformation: "Add additional data to the personal information XML file."

The contract

IPS\Extensions\MemberExportPiiAbstract is one of the smallest abstracts in the suite — GroupLimitsAbstract and MFAAreaAbstract are shorter still. It declares no properties, no constructor and exactly one abstract method (system/Extensions/MemberExportPiiAbstract.php:25-34):

abstract class MemberExportPiiAbstract
{
    /**
     * Return data
     * @param	Member		$member		The member
     *
     * @return	array
     */
    abstract public function getData( Member $member ): array;
}

$member is the member the export is about. On the ACP route that is not the person clicking the button. Read nothing from Member::loggedIn() inside getData() or an administrator exporting someone else's file will get their own data instead.

The return value is an array that becomes a subtree of the XML document. Array keys become element names and values become element text. The conversion is done by IPS\Xml\SimpleXML::addChild() (system/Xml/SimpleXML.php:158-224), which is worth understanding before you design your return shape:

  • Nested arrays are handled recursively, to any depth. array( 'customer' => array( 'first_name' => 'Ada' ) ) becomes <customer><first_name>Ada</first_name></customer>.
  • An integer key cannot be an element name, so it is replaced by the name of the parent element, with a trailing es or s stripped (lines 160-174). A list under the key addresses produces repeated <address> elements; a list under data produces <data> nested inside <data>, because data does not end in s.
  • Booleans are cast to integers, so TRUE is written as 1 (lines 180-183).
  • A value containing <, > or & is wrapped in a CDATA section (lines 186-195).
  • Anything else is passed straight to SimpleXMLElement::addChild() as a string.

There is no __construct() on the abstract, but core does pass an argument when it builds your object — see the note on the constructor further down. In practice you should not declare one.

Who calls it

One place, and only one: IPS\Member::getPiiData() at system/Member/Member.php:6925-6944.

public function getPiiData(): SimpleXML
{
    /* Init */
    $xml = SimpleXML::create('member_export');
    $xml->addAttribute( 'created', DateTime::ts( time() )->rfc3339() );

    /* Get the data */
    foreach( Application::allExtensions( 'core', 'MemberExportPersonalInformation', TRUE, 'core' ) AS $key => $ext )
    {
        if ( $data = $ext->getData( $this ) and is_array( $data ) and count( $data ) )
        {
            $child = $xml->addChild( $key );
            foreach( $data as $k => $v )
            {
                $child->addChild( $k, $v );
            }
        }
    }
    return $xml;
}

$key is the key Application::allExtensions() uses for the whole suite: <appDirectory>_<extensionKey>, built at system/Application/Application.php:419. So core's contribution lands under <core_Main>, Commerce's under <nexus_Customer>, and an extension called Data.php in an app called acme lands under <acme_Data>. You do not choose that name; it is your app directory and your file name.

getPiiData() itself has two callers:

  • applications/core/modules/admin/members/members.php:4828, in exportPersonalInfo(). Gated by Dispatcher::i()->checkAcpPermission( 'member_export_pi' ) at line 4811, logged to the ACP log as acplog__member_pii_exported, then sent as application/xml with a download disposition.
  • applications/core/modules/front/system/settings.php:943, in downloadPiiData(). Gated by a CSRF check, the MFA session flags, PrivacyAction::canDownloadPiiData() and the pii_type setting being exactly 'on' (line 938). That setting is the radio at applications/core/modules/admin/members/privacy.php:280-290 with options off, on and redirect; only on uses this extension at all. canDownloadPiiData() (system/Member/PrivacyAction.php:110-124) requires an approved row in core_member_privacy_actions with action = 'pii_download', which an administrator creates by approving the member's request.

Nothing else in the suite reads this extension type. There is no background task, no REST endpoint and no GraphQL query that consumes it, so your getData() only ever runs on one of those two page loads.

The shipped implementations

There are exactly two in the whole product: applications/core/extensions/core/MemberExportPersonalInformation/Main.php and applications/nexus/extensions/core/MemberExportPersonalInformation/Customer.php. Forums, Pages, Downloads, Gallery, Blog and Calendar ship none. If you were hoping to copy a pattern from a content application, there is nothing to copy.

A minimal example

Commerce's extension is the better model to start from, because it is the one that guards against the member having no record in the application at all. In full apart from the copyright docblocks, from applications/nexus/extensions/core/MemberExportPersonalInformation/Customer.php:

<?php

namespace IPS\nexus\extensions\core\MemberExportPersonalInformation;

use Exception;
use IPS\Db;
use IPS\Extensions\MemberExportPiiAbstract;
use IPS\GeoLocation;
use IPS\Member;
use IPS\nexus\Customer as NexusCustomer;
use IPS\nexus\Customer\CustomField;
use IPS\Patterns\ActiveRecordIterator;
use OutOfRangeException;
use function defined;
use function is_null;

if ( !defined( '\IPS\SUITE_UNIQUE_KEY' ) )
{
    header( ( $_SERVER['SERVER_PROTOCOL'] ?? 'HTTP/1.0' ) . ' 403 Forbidden' );
    exit;
}

class Customer extends MemberExportPiiAbstract
{
    /**
     * Return data
     * @param	Member		$member		The member
     *
     * @return	array
     */
    public function getData( Member $member ): array
    {
        $return = array();

        try
        {
            $customer = NexusCustomer::load( $member->member_id );

            /* Name and addresses */
            $return['customer'] = array(
                'first_name' => $customer->cm_first_name,
                'last_name'  => $customer->cm_last_name
            );

            foreach( Db::i()->select( '*', 'nexus_customer_addresses', array( '`member`=?', $member->member_id ) ) as $address )
            {
                $return['addresses'][] = GeoLocation::buildFromJson( $address['address'] )->toString( "," );
            }

            /* Customer custom fields */
            foreach ( CustomField::roots() as $field )
            {
                $column = $field->column;
                if ( $column )
                {
                    $return['fields'][ $column ] = $field->displayValue( $customer->$column, TRUE );
                }
            }

            /* Credit cards */
            foreach ( new ActiveRecordIterator( Db::i()->select( '*', 'nexus_customer_cards', array( 'card_member=?', $member->member_id ) ), 'IPS\nexus\Customer\CreditCard' ) as $card )
            {
                try
                {
                    $cardData = $card->card;
                    $return['credit_cards'][ $card->id ] = array(
                        'card_type'   => $cardData->type,
                        'card_number' => $cardData->lastFour,
                        'card_expire' => ( !is_null( $cardData->expMonth ) AND !is_null( $cardData->expYear ) ) ? str_pad( $cardData->expMonth , 2, '0', STR_PAD_LEFT ). '/' . $cardData->expYear : NULL
                    );
                }
                catch ( Exception ) {}
            }
        }
        catch( OutOfRangeException )
        {

        }

        return $return;
    }
}

Note the things it does that matter: it wraps the whole body so a member with no Commerce record produces an empty array rather than an exception; it wraps each card separately so one unreadable card does not lose the rest; it names its list keys addresses and credit_cards so the repeated children come out as <address> and <credit_card> (the card rows are keyed by integer card ID, which is exactly the case the integer-key rule above exists for); and every value it puts in the array is a string or NULL — never an object.

The matching entry in applications/acme/data/extensions.json for your own app:

{
    "core": {
        "MemberExportPersonalInformation": {
            "Data": "IPS\\acme\\extensions\\core\\MemberExportPersonalInformation\\Data"
        }
    }
}

The XML downloads but no XML reader will open it

This is the failure mode to worry about, because there is no error, no log entry and no warning. The file is served with a 200 and Content-Type: application/xml, and it is malformed.

Your array keys become element names verbatim. Nothing validates them. SimpleXMLElement::addChild() does not reject an invalid name — tested on PHP 8.5, addChild( 'my field', 'x' ) and addChild( '123bad', 'y' ) both return an element object and emit no warning, and the resulting document produces "PARSE FAILED" when fed back to simplexml_load_string(). A key containing a space, a slash, a colon or a leading digit will silently poison the whole file, including core's and Commerce's sections.

Keys that come from data rather than from your code are where this bites: a custom field label, a product name, a user-supplied tag. Core avoids it for profile fields by prefixing ($return['pfield_' . $fieldID] at Main.php:130) and Commerce by using a database column name (Customer.php:69). Do the same. Core is not entirely consistent about this — Main.php:109 uses $return['terms_accepted'][ $data['type'] ][], where the sub-key comes out of the JSON in a core_member_history row rather than out of the source — so treat the prefixing rule as the advice, not as something the product enforces. If a key is not a literal string you typed, prefix it with a constant and restrict the rest to [A-Za-z0-9_].

The developer only notices when someone tries to actually read the export — which, for a data-subject request, may be a regulator or a lawyer rather than you.

Fatal: the class must be declared abstract

Fatal error: Class IPS\acme\extensions\core\MemberExportPersonalInformation\Data
contains 1 abstract method and must therefore be declared abstract or implement
the remaining methods (IPS\Extensions\MemberExportPiiAbstract::getData)

getData() is abstract; you must declare it. The compile error is triggered by a class_exists() call, which autoloads and compiles your file. There are two such call sites and which one you hit depends on the cache: on the first request after the datastore is cleared it is class_exists() inside Application::extensions() at system/Application/Application.php:926 (reached from allExtensions() at line 417 while the class map is being built); on every request thereafter the map is already cached and the class is compiled by class_exists() inside constructExtensionClass() at line 472. These are separate call sites, not one nested inside the other.

Unlike most extension types, the blast radius here is small. This extension type is loaded only when getPiiData() asks for it, so the fatal appears on the ACP export screen and the member's download link and nowhere else. It will not take down the front end. That also means a broken extension of this type can sit undetected for months.

The same applies to a signature that does not match. Dropping the : array return type gives "Declaration of ... must be compatible with ..." at the same point, because a return type may only be narrowed, never widened. The parameter is the opposite: PHP's contravariance rules mean you may widen it, so getData( mixed $member ): array and getData( $member ): array are both legal overrides and compile fine (verified on PHP 8.5.5). Only a type that is neither IPS\Member nor a supertype of it — string, array, some unrelated class — produces the fatal.

TypeError: Return value must be of type array

The return type is declared on the abstract, so returning NULL or a string throws a TypeError from inside your own method. Nothing catches it. getPiiData() has no try/catch, and neither controller wraps the call. The catch( RuntimeException | OutOfRangeException ) at system/Application/Application.php:497 only covers construction of the extension object, not the call to getData().

The result is an uncaught exception: logged by IPS\IPS::exceptionHandler() under uncaught_exception, and the user gets a generic error page instead of a download. The stack trace names your file, so this one is at least easy to diagnose. Check ACP → Support → System Logs.

One extension throws and the whole export dies, including core's data

The loop at system/Member/Member.php:6932-6942 is unguarded. Any exception escaping any application's getData() aborts the entire document. The member gets no file at all, not a file missing one section. There is no per-extension isolation here — and, for what it is worth, no other extension loop in core provides any either: the Uninstall extension is dispatched by an equally unguarded loop in Application::delete() (system/Application/Application.php:5455-5469). Nobody is going to catch this for you.

This is why Commerce wraps its whole body in try { } catch( OutOfRangeException ) { }. Assume the member you are handed may have no row in any of your tables, may have been merged, or may be a guest-turned-member with a partial record.

Core's own extension is a demonstration of the risk rather than of good practice. Main.php:124 reads:

$fieldValues = Db::i()->select( '*', 'core_pfields_content', array( 'member_id=?', $member->member_id ) )->first();

IPS\Db\Select::first() throws UnderflowException when the query matches nothing (system/Db/Select.php:381-392). If a member has no row in core_pfields_content, that exception propagates all the way out and no export is produced for anyone in that situation. Whether such a member can exist on a healthy install is unverified — IPS normally inserts the row at registration — but the line is unguarded, and it is the first thing to check if exports fail for some members and not others.

Returning an object value takes the export down

Every value you put in the array eventually reaches preg_match( '/[<>&]/', $value ) at system/Xml/SimpleXML.php:186. Arrays and IPS\Xml\SimpleXML instances are routed away before that line; nothing else is. An object without __toString() — an IPS\DateTime, an IPS\Member, an ActiveRecord — produces:

TypeError: preg_match(): Argument #2 ($subject) must be of type string, IPS\DateTime given

thrown from a core file, with your extension a couple of frames up. This is the most common way a first attempt at this extension fails, because dates are the obvious thing to export. Core converts them explicitly (Main.php:67, Main.php:70):

$val = ( ! empty( $member->$col ) ) ? DateTime::ts( $member->$col )->rfc3339() : NULL;

An object that does implement __toString() is coerced silently and works. That inconsistency makes the failure look intermittent when your values are mixed types.

NULL values are safe in practice. They produce an empty element and, on PHP 8.1 and later, a deprecation notice from that preg_match() call — but IPS\IPS::errorHandler() returns early for E_DEPRECATED, E_WARNING, E_NOTICE and E_STRICT (init.php:796-799), so nothing is thrown or logged. Core relies on this: Main.php:67 returns NULL for a member who has never posted.

Nothing from my app appears in the file, and no error is raised

Four separate causes, all silent. Work through them in order.

An empty array. if ( $data = $ext->getData( $this ) and is_array( $data ) and count( $data ) ) skips the extension entirely when the array is empty — not even an empty <acme_Data> element is written. This is correct behaviour and it is what Commerce's catch block produces, but it means "my section is missing" and "my extension threw and was swallowed somewhere" look identical. They are not: nothing is swallowed here, so if the file is produced at all, an empty array is the explanation.

Not registered in extensions.json. Application::extensions() reads applications/<app>/data/extensions.json and nothing else (system/Application/Application.php:917-950). It never scans the extensions/ directory. Creating the PHP file by hand is not enough. Add it through the Developer Center, or edit the JSON yourself. A class name that does not match the JSON value fails class_exists() at line 926 and is skipped by continue, with no message.

The datastore cache. allExtensions() caches the resolved class map in Store::i()->extensions, keyed by extension type only (system/Application/Application.php:357-443). Once MemberExportPersonalInformation is in that store the extensions.json read described above never runs again. Dropping a file in over FTP clears nothing, and neither does editing the JSON. Use ACP → Support → Clear Caches, or save the application record.

A method called generate(). Application::extensions() checks method_exists( $classname, 'generate' ) at line 932 and, if present, treats your class as an extension generator — it calls $classname::generate() and registers whatever that returns instead of your class. Adding a helper method with that name makes your extension disappear. Name it anything else.

My extension is skipped for some people and not others

The call passes TRUE as the third argument, which is $checkAccess (system/Application/Application.php:355):

public static function allExtensions( Application|string $app, string $extension, bool|Group|Member|null $checkAccess=TRUE, string $firstApp=NULL, string $firstExtensionKey=NULL, bool $construct=TRUE ): array

During the cache build that becomes $application->canAccess( NULL ) at line 408, which falls back to Member::loggedIn() and tests the application's disabled_groups. Two consequences follow, and both are properties of the cache rather than of your code:

  • If your application is restricted by group, whether your section appears depends on who first triggered the cache build for this extension type — not on who is downloading the file. Whoever ran it decides for everyone until the cache is cleared.
  • If the cache is first built while RECOVERY_MODE is on, every third-party application is skipped (line 396) and that list is written to the datastore. Turning recovery mode off does not clear it.

The fourth argument, 'core', is $firstApp. It sorts the core application to the front of the list (lines 374-390) so <core_Main> is the first section of the document. That ordering is also baked into the cache. You have no way to influence where your section lands beyond your app directory name.

Do not declare a constructor

The abstract declares none, but constructExtensionClass() still passes one argument (system/Application/Application.php:479):

$obj = new $classToUse( $checkAccess === TRUE ? Member::loggedIn() : ( $checkAccess === FALSE ? NULL : $checkAccess ) );

With $checkAccess of TRUE, your class is constructed with Member::loggedIn(). PHP tolerates extra arguments to a class that declares no constructor, which is why neither shipped implementation has one. If you add a constructor with a parameter typed as something other than Member, you get a TypeError — and because TypeError extends Error, not Exception, the catch( RuntimeException | OutOfRangeException ) on line 497 does not catch it and the export page dies.

If you do declare one, note again that the member it receives is the logged-in user, not the subject of the export. On the ACP route those are different people. Everything you export must be derived from the $member argument to getData().

What this extension is not

It is export only. Registering here does not delete anything, does not participate in the right-to-be-forgotten flow (right_to_be_forgotten_type and PrivacyAction::TYPE_REQUEST_DELETE are a separate mechanism), and does not anonymise. It also does not gate access to the data — the permission checks live entirely in the two controllers listed above, and by the time getData() runs the decision to hand this member's data over has already been made.

There is no partial or streamed mode. The whole document is built in memory in a single request. If your application holds a large volume of per-member rows, be selective about what you return; there is no background-queue path for this export in 5.0.19.

Verified against

Read from Invision Community 5.0.19 source: system/Extensions/MemberExportPiiAbstract.php, system/Member/Member.php (getPiiData(), lines 6925-6944), system/Xml/SimpleXML.php (create(), addChild()), system/Application/Application.php (allExtensions(), extensions(), constructExtensionClass(), canAccess(), delete()), system/Db/Select.php (first()), system/Member/PrivacyAction.php, init.php (errorHandler()), applications/core/modules/admin/members/members.php (exportPersonalInfo()), applications/core/modules/front/system/settings.php (downloadPiiData()), applications/core/modules/admin/members/privacy.php (settings()), and both shipped implementations, applications/core/extensions/core/MemberExportPersonalInformation/Main.php and applications/nexus/extensions/core/MemberExportPersonalInformation/Customer.php. The claims about SimpleXMLElement::addChild() not validating element names, about preg_match() rejecting non-stringable objects while accepting NULL with only a deprecation, and about which getData() signatures are legal overrides, were confirmed by running them on PHP 8.5.5.


User Feedback

Recommended Comments

There are no comments to display.

Account

Navigation

Search

Search

Configure browser push notifications

Chrome (Android)
  1. Tap the lock icon next to the address bar.
  2. Tap Permissions → Notifications.
  3. Adjust your preference.
Chrome (Desktop)
  1. Click the padlock icon in the address bar.
  2. Select Site settings.
  3. Find Notifications and adjust your preference.