Loading init.php from a CLI script gives you most of the framework, and it is by far the fastest way to test an application. Some things genuinely cannot run there, and knowing which saves a lot of time spent debugging the harness instead of the code.

The basic harness

<?php
require_once '/var/www/html/init.php';

use IPS\Db;

/* IPS installs its own error handlers. Under CLI a fatal can end the process
   with NO OUTPUT AT ALL, which reads as the script silently stopping. Write to
   a file so failures are visible. */
const LOG = '/tmp/mytest.log';
@unlink( LOG );

function say( string $line = '' ): void
{
    file_put_contents( LOG, $line . PHP_EOL, FILE_APPEND );
}

set_exception_handler( function ( $e ) {
    say( 'FATAL ' . get_class( $e ) . ': ' . $e->getMessage() );
    say( $e->getTraceAsString() );
} );

That error-handler detail is the first thing to get right. Without it, a fatal produces an empty terminal and no clue.

What does not work, and why

Do not chase the $_SERVER errors

Filling in SERVER_NAME gets you as far as QUERY_STRING, then SCRIPT_NAME, and eventually to the Dispatcher error anyway. If a test needs a controller, that test needs a browser. Test the piece the controller calls instead:

// Instead of building the whole settings form, check the element it adds
$field = new \IPS\Helpers\Form\Sort( 'myapp_order', $order, FALSE, array( 'checkboxes' => 'myapp_' ) );

Test widgets through __toString(), not render()

__toString() runs init(), _render() and the caching layer — the whole path the theme actually uses. Calling render() directly bypasses all of it and proves considerably less.

Reproduce the consumer, not your own method

The most valuable CLI tests are the ones that imitate what core does with your code, because that is where extension-point mistakes surface. Calling your own method and getting a sensible answer back proves very little.

/* This is the test that catches a broken core/Loader extension: it reproduces
   the dispatcher's merge loop across every installed application. */
foreach ( Application::allExtensions( 'core', 'Loader' ) as $key => $loader )
{
    foreach ( $loader->js() as $js )
    {
        if ( !is_array( $js ) )
        {
            say( 'BREAK ' . $key . ' returns a flat list and will 500 every page' );
        }
    }
}

The same idea covers language keys an ACP screen will demand, extension classes that must exist, and any contract core enforces at runtime rather than at install.

Static checks catch what CLI cannot run

Where the failure is inside a controller, read the source instead of executing it. Scanning for 'app' => ..., 'key' => ... pairs on every Form\Editor and confirming the named EditorLocations file exists takes a second and catches a bug that otherwise only appears when a human opens the page.

Verified against

Invision Community 5.0.19, by building and testing applications against it.

Related: S3 and R2 storage break in any CLI script without HTTPS=on, Your background task is running inside a visitor's page load, Application versions: why re-uploading the same version does nothing