module. Such a configuration file looks roughly like this:
@ -55,21 +164,7 @@ is equivalent to this:
some_var:some value with whitespaces before and after
```
The test suite itself uses the builtin `unittest` module of Python to discover
and run the individual test cases. The test cases themselves are implemented in
Python source files, but the required Python knowledge is minimal.
## Test suite
The test suite is configured via one configuration file whose location
automatically sets the root directory of the test suite. The `unittest` module
then recursively searches all sub-directories with a `__init__.py` file for
files of the form `test_*.py`, which it automatically interprets as test cases
(more about these in the next section). Python will automatically interpret each
directory as a module and use this to format the output, e.g. the test case
`regression/crashes/test_bug_15.py` will be interpreted as the module
`regression.crashes.test_bug_15`. Thus one can use the directory structure to
group test cases.
#### Parameters
The test suite's configuration file should have the following form:
@ -161,7 +256,7 @@ Please note that while the `INI` file allows for variables with whitespaces or
variable names in Python.
## Test cases
### Test cases
The test cases are defined in Python source files utilizing the unittest module,
thus every file must also be a valid Python file. Each file defining a test case
@ -692,28 +787,7 @@ class AnInformativeName(metaclass=system_tests.CaseMeta):
for expansion.
## Running the test suite
## Bash test cases
The test suite is written for Python 3 and is not compatible with Python 2, thus
it must be run with `python3` and not with `python` (which is usually an alias
for Python 2).
Then navigate to the `tests/` subdirectory and run:
``` shell
python3 runner.py
```
One can supply the script with a directory where the suite should look for the
tests (it will search the directory recursively). If omitted, the runner will
look in the directory where the configuration file is located. It is also
possible to instead pass a file as the parameter, the test suite will then only
run the tests from this file.
The runner script also supports the optional arguments `--config_file` which
allows to provide a different test suite configuration file than the default
`suite.conf`. It also forwards the verbosity setting via the `-v`/`--verbose`
flags to Python's unittest module.
Optionally one can provide the `--debug` flag which will instruct test suite to
print all command invocations and all expected and obtained outputs to the
standard output.
- Previously, Exiv2 had some bash test scripts, which were saved as the file `EXIV2_DIR/test/*.sh`. We're going to rewrite them as Python test scripts and save them to the directory `EXIV2_DIR/tests/bash_test`.
- These Python test scripts are based on [unittest](https://docs.python.org/3/library/unittest.html) and written in a common format, which is different from the format described in [Writing new tests](#writing-new-tests), but can be executed compatibly by `python3 runner.py`.
- Plan to convert the bash test scripts `../../test/*.sh` into Python scripts and save them to this directory.
- The test cases in this directory are based on [unittest](https://docs.python.org/3/library/unittest.html) and written in a common format, which is different from the format described in `../writing_tests.md`, but can be executed compatibly.
## Running the test cases
- Use runner.py to collect test cases and execute them:
```sh
cd EXIV2_DIR/tests
python3 runner.py -v unit_test
```
- Also, you can use pytest to execute the test cases for unittest, so that the test report looks better:
```sh
cd EXIV2_DIR/tests
pytest -v unit_test
```
- If no exception occurs during the execution of the test case, it passes. Otherwise it fails.
LeoHsiao
5 years ago