Testing Magic Smoke

This page describes how to run the self-tests of MagicSmoke. You do not need those tests if you merely want to compile, install and use a release version of the software. This should come as a relief to you, since due to the complex nature of MagicSmoke these tests are quite complex to set up.

It is however recommended that you set up and run them if you make any modifications to MagicSmoke.

Important Note: the MagicSmoke tests are designed to be run on a Linux system (Debian if you want to make sure). Windows is not supported because it is not supported for the server component. MacOS is also not supported (at the moment for any component - I do not have the kind of spare money to keep up with Apple's rather short product life-cycles).

There are two sets of tests for MagicSmoke: Unit tests and GUI tests. The Unit tests check interfaces and internal functions of MagicSmoke, while the GUI tests check the integration from the client perspective. Both sets can be run separately.

The MagicSmoke test suites create their own test environment with their own database, web server and configuration location - so normally it should not disturb a normal installation of MagicSmoke. It is still recommended that you create a separate user for those tests.

The tests described here assume that components used by MagicSmoke already work correctly - so they only test actual MagicSmoke functionality. External code (like Elam, TZone, ...) comes with its own tests which are not triggered from the MagicSmoke tests.


You have to build MagicSmoke first. All components mentioned as prerequisites on the build page are also needed for tests.

In addition you'll need:

Debian Linux - the test setup was originally created for Debian Wheezy. Newer (or slightly older) versions of Debian and Ubuntu should also work. Other distributions may be similar enough to work or too different to find important components - good luck - patches are welcome.

PostgreSQL 9.x - the test server backend expects PostgreSQL as a database, since it is the more strict of all available options, it is not possible to run the tests with MySQL without a lot of patching in the test environment.

Apache 2.2 with the php5 module and mod_ssl - the test server is set up for Apache 2.2 (other 2.x versions might work, but I did not test them), I also only tested this setup on Debian - if Apache components are found in a different directory for your distribution you may need to change the test setup.

GNU Wget is used for creating the database structure (calling admin.php).

GNU Bash is used for all helper scripts, it is expected to reside in /bin/bash. The standard Dash shell of many modern Linux systems does not understand a few of the parameters used in those scripts.

GUI tests only:
Froglogic Squish 4.3 (or newer) for Qt5 - although this is a commercial toolkit, MagicSmoke GUI tests rely on it since there is no Open Source alternative that is up to this job yet.

Test Setup and Scripts

Tests are distributed over the following directories:

Make sure ports 20987-20989 are free on your system - the test backend will use them:
PortUsed by...
20987used by the PostgreSQL server
20988used for the Apache Web Server for unencrypted HTTP
20989used for the Apache Web Server for encrypted HTTPS

Those ports can be changed in the configuration. But it is recommended to leave them as is. The servers will always bind to localhost to prevent connections from outside the test box.

Copy the config.tmpl file to config, otherwise the helper scripts will not work. If you run Debian/Wheezy you are done, otherwise you may have to modify the file. See the comments inside the file for help on options that you can set.

You normally should not need to change the configuration templates for Postgres and Apache. However, if you do: configurations are in apache/etc/* (Apache) and db/etc/* (Postgres). Files with the extension .tmpl are templates that can contain patterns that are exchanged when the real configuration files are created from them before the servers are started:
PatternUsed by...
@@Postgres: absolute path to Postgres (the main tests/db directory)
Apache: absolute path to the Apache directory (main tests/apache directory)
@PGPORT@Postgres TCP port
@HTTPPORT@Apache HTTP port

There are a few helper scripts that are normally called from the test programs to initialize the test environment. To ensure the environment works you may also call them manually. The following helper scripts exist:

setup-start.sh starts the Postgres and Apache servers. It always regenerates configuration files. If it has not happened yet it will also generate the database data directory and the local MagicSmoke server installation. It will not overwrite either of those.

setup-stop.sh shuts down the Postgres and Apache servers. Leaves the data intact. Call this if one of the tests crashes and you have to clean up.

setup-create.sh creates the MagicSmoke database. It relies on both servers already running and then makes the necessary calls to completely re-initialize the test database (all data currently stored is lost).

setup-clean.sh deletes the test environment. You must shutdown the servers first. It deletes the test database and test server installation. Call this to completely re-initialize the environment or after you made changes to the server installation process.

Running Unit Tests

Running GUI Tests

Integration with Jenkins