## Writing new tests

The test suite is intended to run a binary and compare its standard output,
standard error and return value against provided values. This is implemented
using Python's `unittest` module and thus all test files are Python files.

The simplest test has the following structure:
``` python
# -*- coding: utf-8 -*-

import system_tests


class GoodTestName(system_tests.Case):

    filename = "{data_path}/test_file"
    commands = ["{exiv2} " + filename, "{exiv2} " + filename + '_2']
    stdout = [""] * 2
    stderr = ["""{exiv2_exception_msg} """ + filename + """:
{error_58_message}
"""] * 2
    retval = [1] * 2
```

The test suite will run the provided commands in `commands` and compare them to
the output in `stdout` and `stderr` and it will compare the return values.

The strings in curly braces are variables either defined in this test's class or
are taken from the suite's configuration file (see `doc.md` for a complete
explanation).

When creating new tests, follow roughly these steps:

1. Choose an appropriate subdirectory where the test belongs. If none fits
   create a new one and put an empty `__init__.py` file there.

2. Create a new file with a name matching `test_*.py`. Copy the class definition
   from the above example and choose an appropriate class name.

3. Run the test suite via `python3 runner.py` and ensure that your test case is
   actually run! Either run the suite with the `-v` option which will output all
   test cases that were run or simply add an error and check if errors occur.