Updating documentation to respect ctest.

main
Robin Mills 4 years ago
parent 29cc981180
commit 122fb95eb3

@ -118,9 +118,9 @@ if( EXIV2_BUILD_SAMPLES )
WORKING_DIRECTORY ${PROJECT_SOURCE_DIR}/tests WORKING_DIRECTORY ${PROJECT_SOURCE_DIR}/tests
COMMAND cmake -E env EXIV2_BINDIR=${CMAKE_RUNTIME_OUTPUT_DIRECTORY} ${Python3_EXECUTABLE} runner.py --verbose tiff_test COMMAND cmake -E env EXIV2_BINDIR=${CMAKE_RUNTIME_OUTPUT_DIRECTORY} ${Python3_EXECUTABLE} runner.py --verbose tiff_test
) )
add_test(NAME lensTests add_test(NAME versionTest
WORKING_DIRECTORY ${PROJECT_SOURCE_DIR}/tests WORKING_DIRECTORY ${PROJECT_SOURCE_DIR}/tests
COMMAND cmake -E env EXIV2_BINDIR=${CMAKE_RUNTIME_OUTPUT_DIRECTORY} ${Python3_EXECUTABLE} runner.py --verbose lens_tests COMMAND cmake -E env EXIV2_BINDIR=${CMAKE_RUNTIME_OUTPUT_DIRECTORY} ${Python3_EXECUTABLE} runner.py --verbose bash_tests/version_test.py
) )
endif() endif()
endif() endif()

@ -110,7 +110,7 @@ _Profiles for Visual Studio are discussed in detail here: [Visual Studio Notes](
| _**1**_ | Get conan to fetch dependencies<br><br>The output can be quite<br>long as conan downloads and/or builds<br>zlib, expat, curl and other dependencies.| $ conan install ..<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;--build missing | c:\\..\\build> conan install .. --build missing<br>&nbsp;&nbsp;&nbsp;&nbsp;--profile msvc2019Release64 | | _**1**_ | Get conan to fetch dependencies<br><br>The output can be quite<br>long as conan downloads and/or builds<br>zlib, expat, curl and other dependencies.| $ conan install ..<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;--build missing | c:\\..\\build> conan install .. --build missing<br>&nbsp;&nbsp;&nbsp;&nbsp;--profile msvc2019Release64 |
| _**2**_ | Get cmake to generate<br>makefiles or sln/vcxproj | $ cmake .. | c:\\..\\build> cmake&nbsp;..&nbsp;-G&nbsp;"Visual Studio 16 2019" | _**2**_ | Get cmake to generate<br>makefiles or sln/vcxproj | $ cmake .. | c:\\..\\build> cmake&nbsp;..&nbsp;-G&nbsp;"Visual Studio 16 2019"
| _**3**_ | Build | $ cmake --build . | c:\\..\\build>&nbsp;cmake&nbsp;--build&nbsp;.&nbsp;--config&nbsp;Release<br>You may prefer to open exiv2.sln and build using the IDE. | | _**3**_ | Build | $ cmake --build . | c:\\..\\build>&nbsp;cmake&nbsp;--build&nbsp;.&nbsp;--config&nbsp;Release<br>You may prefer to open exiv2.sln and build using the IDE. |
| _**4**_ | Optionally Run Test Suite | $ make tests | c:\\..\\build>&nbsp;cmake&nbsp;--build&nbsp;.&nbsp;--config&nbsp;Release --target tests<br/>[README.md](README.md) | | _**4**_ | Optionally Run Test Suite | $ make test | c:\\..\\build>&nbsp;cmake&nbsp;--build&nbsp;.&nbsp;--config&nbsp;Release --target test<br/>[README.md](README.md) |
@ -587,4 +587,4 @@ $ cmake -DEXIV2_ENABLE_WEBREADY=ON -DEXIV2_ENABLE_CURL=ON ..
[TOC](#TOC) [TOC](#TOC)
Written by Robin Mills<br>robin@clanmills.com<br>Updated: 2021-03-18 Written by Robin Mills<br>robin@clanmills.com<br>Updated: 2021-12-17

@ -62,7 +62,7 @@ The file ReadMe.txt in a build bundle describes how to install the library on th
1. [Running tests on a UNIX-like system](#4-1) 1. [Running tests on a UNIX-like system](#4-1)
2. [Running tests on Visual Studio builds](#4-2) 2. [Running tests on Visual Studio builds](#4-2)
3. [Unit tests](#4-3) 3. [Unit tests](#4-3)
4. [Python tests](#4-4) 4. [Bugfix tests](#4-4)
5. [Test Summary](#4-5) 5. [Test Summary](#4-5)
6. [Fuzzing](#4-6) 6. [Fuzzing](#4-6)
1. [OSS-Fuzz](#4-6-1) 1. [OSS-Fuzz](#4-6-1)
@ -90,8 +90,8 @@ $ cd ~/gnu/github/exiv2 # location of the project code
$ mkdir build && cd build $ mkdir build && cd build
$ cmake .. -DCMAKE_BUILD_TYPE=Release $ cmake .. -DCMAKE_BUILD_TYPE=Release
$ cmake --build . $ cmake --build .
$ make tests $ ctest
$ sudo make install $ sudo cmake --build . --target install
``` ```
This will install the library into the "standard locations". The library will be installed in `/usr/local/lib`, executables (including the exiv2 command-line program) in `/usr/local/bin/` and header files in `/usr/local/include/exiv2` This will install the library into the "standard locations". The library will be installed in `/usr/local/lib`, executables (including the exiv2 command-line program) in `/usr/local/bin/` and header files in `/usr/local/include/exiv2`
@ -120,7 +120,7 @@ I don't know why anybody would uninstall Exiv2.
```bash ```bash
$ cd ~/gnu/github/exiv2 # location of the project code $ cd ~/gnu/github/exiv2 # location of the project code
$ cd build $ cd build
$ sudo make uninstall $ sudo cmake --build . --target uninstall
``` ```
These commands will remove the exiv2 executables, library, header files and man page from the standard locations. These commands will remove the exiv2 executables, library, header files and man page from the standard locations.
@ -395,7 +395,7 @@ Additionally, you will require an additional build step to actually build the do
```bash ```bash
$ cmake ..options.. -DEXIV2_BUILD_DOC=On $ cmake ..options.. -DEXIV2_BUILD_DOC=On
$ make doc $ cmake --build . --target doc
``` ```
To build the documentation, you must install the following products: To build the documentation, you must install the following products:
@ -429,7 +429,7 @@ $ cmake .. -G "Unix Makefiles" -DEXIV2_TEAM_PACKAGING=On
$ cmake --build . --config Release $ cmake --build . --config Release
... ...
[100%] Built target addmoddel [100%] Built target addmoddel
$ make package $ cmake --build . --target package
... ...
CPack: - package: /path/to/exiv2/build/exiv2-0.27.1-Linux.tar.gz generated. CPack: - package: /path/to/exiv2/build/exiv2-0.27.1-Linux.tar.gz generated.
``` ```
@ -437,14 +437,12 @@ CPack: - package: /path/to/exiv2/build/exiv2-0.27.1-Linux.tar.gz generated.
2) Source Package 2) Source Package
```bash ```bash
$ make package_source $ cmake --build . --target package_source
Run CPack packaging tool for source... Run CPack packaging tool for source...
... ...
CPack: - package: /path/to/exiv2/build/exiv2-0.27.1-Source.tar.gz generated. CPack: - package: /path/to/exiv2/build/exiv2-0.27.1-Source.tar.gz generated.
``` ```
You may prefer to run `$ cmake --build . --config Release --target package_source`
[TOC](#TOC) [TOC](#TOC)
<div id="2-11"> <div id="2-11">
@ -460,8 +458,7 @@ $ cd <exiv2dir>
$ mkdir build $ mkdir build
$ cd build $ cd build
$ cmake .. -G "Unix Makefiles" "-DCMAKE_BUILD_TYPE=Debug" $ cmake .. -G "Unix Makefiles" "-DCMAKE_BUILD_TYPE=Debug"
$ make $ cmake --build .
``` ```
You must install the library to ensure that your code is linked to the debug library. You must install the library to ensure that your code is linked to the debug library.
@ -532,7 +529,7 @@ Visual Studio and Xcode can build debug or release builds without using the opti
With the Unix Makefile generator, the targets can be listed: With the Unix Makefile generator, the targets can be listed:
```bash ```bash
$ make help $ cmake --build . --target help
The following are some of the valid targets for this Makefile: The following are some of the valid targets for this Makefile:
... all (the default if no target is provided) ... all (the default if no target is provided)
... clean ... clean
@ -593,10 +590,10 @@ To build with ccache, use the cmake option **-DBUILD\_WITH\_CCACHE=On**
$ cd <exiv2dir> $ cd <exiv2dir>
$ mkdir build ; cd build ; cd build $ mkdir build ; cd build ; cd build
$ cmake .. -G "Unix Makefiles" -DBUILD_WITH_CCACHE=On $ cmake .. -G "Unix Makefiles" -DBUILD_WITH_CCACHE=On
$ make $ cmake --build .
# Build again to appreciate the performance gain # Build again to appreciate the performance gain
$ make clean $ cmake --build . --target clean
$ make $ cmake --build .
``` ```
Due to the way in which ccache is installed in Fedora (and other Linux distros), ccache effectively replaces the compiler. A default build or **-DBUILD\_WITH\_CCACHE=Off** is not effective and the environment variable CCACHE_DISABLE is required to disable ccache. [https://github.com/Exiv2/exiv2/issues/361](https://github.com/Exiv2/exiv2/issues/361) Due to the way in which ccache is installed in Fedora (and other Linux distros), ccache effectively replaces the compiler. A default build or **-DBUILD\_WITH\_CCACHE=Off** is not effective and the environment variable CCACHE_DISABLE is required to disable ccache. [https://github.com/Exiv2/exiv2/issues/361](https://github.com/Exiv2/exiv2/issues/361)
@ -767,7 +764,7 @@ To build Exiv2 v0.27.X with C++11:
cd <exiv2dir> cd <exiv2dir>
mkdir build ; cd build mkdir build ; cd build
cmake .. -DCMAKE_CXX_STANDARD=11 -DCMAKE_CXX_FLAGS=-Wno-deprecated cmake .. -DCMAKE_CXX_STANDARD=11 -DCMAKE_CXX_FLAGS=-Wno-deprecated
make cmake --build .
``` ```
The option -DCMAKE\_CXX\_STANDARD=11 specifies the C++ Language Standard. Possible values are 98, 11, 14, 17 or 20. The option -DCMAKE\_CXX\_STANDARD=11 specifies the C++ Language Standard. Possible values are 98, 11, 14, 17 or 20.
@ -864,18 +861,27 @@ For new bug reports, feature requests and support: Please open an issue in Gith
[TOC](#TOC) [TOC](#TOC)
<div id="4"> <div id="4">
## 4 Running the test suite ## 4 Test Suite
#### Different kinds of tests: The test suite is implemented using CTest. CMake adds the target 'test' to the build. You can run ctest with the command `$ cmake --build . --target test`, or simply with `$ ctest`. The build creates 5 tests: bashTests, bugfixes, tiffTests, unit_tests and versionTest. You can run all tests or a subset.
| Description | Language | Location | Command<br>_(in build directory)_ | CMake Option to Build | ```bash
$ cmake --build . --target test
$ ctest # run all tests and display summary
$ ctest --output-on-failure # run all tests and output failures
$ ctest -R bugfixes # run only bugfixes and display summary
$ ctest -R bugfixes --verbose # run only bugfixes and display all output
```
#### Test Architecture
| Name | Language | Location | Command<br>_(in build directory)_ | CMake Option to Build |
|:-- |:-- |:-- |:-- |:-- | |:-- |:-- |:-- |:-- |:-- |
| Run all tests | | | $ make tests | | | bashTests | python | tests/bash\_tests | $ ctest -R bash | -DEXIV2\_BUILD\_SAMPLES=On |
| Run all tests | | **Visual Studio Users** | > cmake --build . --target tests | | | bugfixes | python | tests/bugfixes | $ ctest -R bugfixes | -DEXIV2\_BUILD\_SAMPLES=On |
| Bash tests | python | tests/bash\_tests | $ make bash_tests | -DEXIV2\_BUILD\_SAMPLES=On | | tiffTests | python | tests/tiff_test | $ ctest -R tiff | -DEXIV2\_BUILD\_SAMPLES=On |
| Python tests | python | tests | $ make python_tests | -DEXIV2\_BUILD\_SAMPLES=On | | unit_tests | C++ | unitTests | $ ctest -R unit | -DEXIV2\_BUILD\_UNIT\_TESTS=On |
| Unit tests | C++ | unitTests | $ make unit_test | -DEXIV2\_BUILD\_UNIT\_TESTS=On | | versionTest | C++ | src/version.cpp | $ ctest -R version | Always in library |
| Version test | C++ | src/version.cpp | $ make version_test | Always in library |
The term _**bash scripts**_ is historical. The implementation of the tests in this collection originally required bash. These The term _**bash scripts**_ is historical. The implementation of the tests in this collection originally required bash. These
scripts have been rewritten in python. Visual Studio Users will appreciate the python implementation as it avoids the scripts have been rewritten in python. Visual Studio Users will appreciate the python implementation as it avoids the
@ -899,17 +905,17 @@ The Variable EXIV2\_PORT or EXIV2\_HTTP can be set to None to skip http tests.
[TOC](#TOC) [TOC](#TOC)
<div id="4-1"> <div id="4-1">
### 4.1 Running tests on Unix-like systems
### 4.1 Running tests on a UNIX-like system
You can run tests directly from the build: You can run tests directly from the build:
```bash ```bash
$ cmake .. -G "Unix Makefiles" -DEXIV2_BUILD_UNIT_TESTS=On $ cmake .. -G "Unix Makefiles" -DEXIV2_BUILD_UNIT_TESTS=On
$ make ... lots of output and build summary ...
... lots of output ... $ cmake --build .
$ make tests
... lots of output ... ... lots of output ...
$ cmake --build . --target test
... test summary ...
$ $
``` ```
@ -917,13 +923,13 @@ You can run individual tests in the `test` directory. **Caution:** If you build
```bash ```bash
$ cd <exiv2dir>/build $ cd <exiv2dir>/build
$ make bash_tests $ ctest -R bash --verbose
addmoddel_test (testcases.TestCases) ... ok addmoddel_test (testcases.TestCases) ... ok
.... ....
Ran 176 tests in 9.526s Ran 176 tests in 9.526s
OK (skipped=6) OK (skipped=6)
$ make python_tests $ ctest -R bugfixes --verbose
... lots of output ... ... lots of output ...
test_run (tiff_test.test_tiff_test_program.TestTiffTestProg) ... ok test_run (tiff_test.test_tiff_test_program.TestTiffTestProg) ... ok
---------------------------------------------------------------------- ----------------------------------------------------------------------
@ -934,10 +940,9 @@ $
[TOC](#TOC) [TOC](#TOC)
<div id="4-2"> <div id="4-2">
### 4.2 Running tests on Visual Studio builds from cmd.exe ### 4.2 Running tests on Visual Studio builds from cmd.exe
**Caution:** _The python3 interpreter must be on the PATH, build for DOS, and called python3.exe. I copied the python.exe program: **Caution:** _The python3 interpreter must be on the PATH, build for DOS, and called python3.exe._ I copied the python.exe program:
```cmd ```cmd
> copy c:\Python37\python.exe c:\Python37\python3.exe > copy c:\Python37\python.exe c:\Python37\python3.exe
@ -945,13 +950,11 @@ $
``` ```
You can execute the test suite as described for UNIX-like systems. You can execute the test suite as described for UNIX-like systems.
The main difference is that you must use cmake to initiate the test
as make is not a system utility on Windows.
```bash ```bash
> cd <exiv2dir>/build > cd <exiv2dir>/build
> cmake --build . --target tests > cmake --build . --target test
> cmake --build . --target python_tests > ctest -R bugfixes --verbose
``` ```
##### Running tests from cmd.exe ##### Running tests from cmd.exe
@ -974,13 +977,12 @@ If you wish to use an environment variables, use set:
``` ```
set VERBOSE=1 set VERBOSE=1
cmake --build . --config Release --target tests cmake --build . --config Release --target test
set VERBOSE= set VERBOSE=
``` ```
[TOC](#TOC) [TOC](#TOC)
<div id="4-3"> <div id="4-3">
### 4.3 Unit tests ### 4.3 Unit tests
The code for the unit tests is in `<exiv2dir>/unitTests`. To include unit tests in the build, use the *cmake* option `-DEXIV2_BUILD_UNIT_TESTS=On`. The code for the unit tests is in `<exiv2dir>/unitTests`. To include unit tests in the build, use the *cmake* option `-DEXIV2_BUILD_UNIT_TESTS=On`.
@ -1000,24 +1002,23 @@ $ popd
[TOC](#TOC) [TOC](#TOC)
<div id="4-4"> <div id="4-4">
### 4.4 Bugfix tests
### 4.4 Python tests You can run the bugfix tests from the build directory:
You can run the python tests from the build directory:
```bash ```bash
$ cd <exiv2dir>/build $ cd <exiv2dir>/build
$ make python_tests $ ctest -R bugfixes
``` ```
If you wish to run in verbose mode: If you wish to run in verbose mode:
```bash ```bash
$ cd <exiv2dir>/build $ cd <exiv2dir>/build
$ make python_tests VERBOSE=1 $ $ ctest -R bugfixes --verbose
``` ```
The python tests are stored in the directory tests and you can run them all with the command: The bugfix tests are stored in the directory tests and you can run them all with the command:
```bash ```bash
$ cd <exiv2dir>/tests $ cd <exiv2dir>/tests
@ -1036,24 +1037,23 @@ You may wish to get a brief summary of failures with commands such as:
```bash ```bash
$ cd <exiv2dir>/build $ cd <exiv2dir>/build
$ make python_tests 2>&1 | grep FAIL $ ctest -R bugfixes --verbose 2>&1 | grep FAIL
``` ```
[TOC](#TOC) [TOC](#TOC)
<div id="4-5"> <div id="4-5">
### 4.5 Test Summary ### 4.5 Test Summary
| *Tests* | Unix Style Platforms _(bash)_ | Visual Studio _(cmd.exe)_ | | *Tests* | Execute using ctest | Execute using cmake |
|:-- |:--- |:-- | |:-- |:--- |:-- |
| | $ cd \<exiv2dir\>/build | \> cd \<exiv2dir\>/build | | | $ cd \<exiv2dir\>/build | \> cd \<exiv2dir\>/build |
| tests | $ make tests | \> cmake --build . --config Release --target tests | | test | $ ctest | \> cmake --build . --config Release --target test |
| bash_tests | $ make bash_tests | \> cmake --build . --config Release --target bash_tests | | bash_tests | $ ctest -R bash | \> cmake --build . --config Release --target bash_tests |
| python_tests | $ make python_tests | \> cmake --build . --config Release --target python_tests | | bugfixes | $ ctest -R bugfixes | \> cmake --build . --config Release --target bugfix_tests |
| unit_test | $ make unit_test | \> cmake --build . --config Release --target unit_test | | unit_test | $ ctest -R unit | \> cmake --build . --config Release --target unit_test |
| version_test | $ make version_test | \> cmake --build . --config Release --target version_test | | version_test | $ ctest -R version | \> cmake --build . --config Release --target version_test |
The name **bash_tests** is historical. They are implemented in python. The name **bashTests** is historical. The tests are implemented in python.
[TOC](#TOC) [TOC](#TOC)
<div id="4-6"> <div id="4-6">

Loading…
Cancel
Save