Testing Magic Smoke

It is however recommended that you set up and run them if you make any modifications to MagicSmoke.

Important Note: the MagicSmoke tests are designed to be run on a Linux system (Debian if you want to make sure). Windows is not supported because it is not supported for the server component. MacOS is also not supported (at the moment for any component - I do not have the kind of spare money to keep up with Apple's rather short product life-cycles).

There are two sets of tests for MagicSmoke: Unit tests and GUI tests. The Unit tests check interfaces and internal functions of MagicSmoke, while the GUI tests check the integration from the client perspective. Both sets can be run separately.

The MagicSmoke test suites create their own test environment with their own database, web server and configuration location - so normally it should not disturb a normal installation of MagicSmoke. It is still recommended that you create a separate user for those tests.

The tests described here assume that components used by MagicSmoke already work correctly - so they only test actual MagicSmoke functionality. External code (like Elam, TZone, ...) comes with its own tests which are not triggered from the MagicSmoke tests.

Prerequisites

In addition you'll need:

Debian Linux - the test setup was originally created for Debian Wheezy. Newer (or slightly older) versions of Debian and Ubuntu should also work. Other distributions may be similar enough to work or too different to find important components - good luck - patches are welcome.

PostgreSQL 9.x - the test server backend expects PostgreSQL as a database, since it is the more strict of all available options, it is not possible to run the tests with MySQL without a lot of patching in the test environment.

Apache 2.2 with the php5 module and mod_ssl - the test server is set up for Apache 2.2 (other 2.x versions might work, but I did not test them), I also only tested this setup on Debian - if Apache components are found in a different directory for your distribution you may need to change the test setup.

GNU Wget is used for creating the database structure (calling admin.php).

GNU Bash is used for all helper scripts, it is expected to reside in /bin/bash. The standard Dash shell of many modern Linux systems does not understand a few of the parameters used in those scripts.

GUI tests only:
Froglogic Squish 4.3 (or newer) for Qt5 - although this is a commercial toolkit, MagicSmoke GUI tests rely on it since there is no Open Source alternative that is up to this job yet.

Test Setup and Scripts

• tests - the main test directory, contains the test environment and Unit tests
  • apache - the Apache Web server setup for the test environment
    • etc - Apache and MagicSmoke server configuration files, change config.php here if you need to alter parameters for the test instance of the MagicSmoke server
    • log - Apache log files, in case something fails, you can inspect them there, the cleanup script deletes this directory!
    • www - dynamically generated MagicSmoke server installation
  • db - Postgres database setup and storage directory
    • etc - DB server configuration
    • data - dynamically created Database storage directory, this holds the actual data
    • run - dynamically created directory that hold server logs, sockets and other housekeeping
  • ssl - contains the SSL certificates used for the tests, normally you should not need to alter them
  • appdata - dynamically created: contains the profile data for the client under test
  • appconf - dynamically created: contains the configuration data for the client under test
• guitest - GUI test scripts for Squish

Those ports can be changed in the configuration. But it is recommended to leave them as is. The servers will always bind to localhost to prevent connections from outside the test box.

Copy the config.tmpl file to config, otherwise the helper scripts will not work. If you run Debian/Wheezy you are done, otherwise you may have to modify the file. See the comments inside the file for help on options that you can set.

You normally should not need to change the configuration templates for Postgres and Apache. However, if you do: configurations are in apache/etc/* (Apache) and db/etc/* (Postgres). Files with the extension .tmpl are templates that can contain patterns that are exchanged when the real configuration files are created from them before the servers are started:

Pattern	Used by...
@@	Postgres: absolute path to Postgres (the main tests/db directory) Apache: absolute path to the Apache directory (main tests/apache directory)
@PGPORT@	Postgres TCP port
@HTTPPORT@	Apache HTTP port
@HTTPSPORT@	Apache HTTPS port

There are a few helper scripts that are normally called from the test programs to initialize the test environment. To ensure the environment works you may also call them manually. The following helper scripts exist:

setup-start.sh starts the Postgres and Apache servers. It always regenerates configuration files. If it has not happened yet it will also generate the database data directory and the local MagicSmoke server installation. It will not overwrite either of those.

setup-stop.sh shuts down the Postgres and Apache servers. Leaves the data intact. Call this if one of the tests crashes and you have to clean up.

setup-create.sh creates the MagicSmoke database. It relies on both servers already running and then makes the necessary calls to completely re-initialize the test database (all data currently stored is lost).

setup-clean.sh deletes the test environment. You must shutdown the servers first. It deletes the test database and test server installation. Call this to completely re-initialize the environment or after you made changes to the server installation process.

Running Unit Tests

Running GUI Tests

Integration with Jenkins