|
There should be an effort to encourage people to adopt a singletest program output format. This would make parsing results easier.
|
There should be an effort to encourage people to adopt a singletest program output format. This would make parsing results easier.
|
|
This page is about the output from the test programs, not the outputfrom Fuego itself. For that, see the Unified Results Format Project.
|
This page is about the output from the test programs, not the outputfrom Fuego itself. For that, see the [[Unified Results Format Project]].
|
|
Candidate formats are: * json - using a schema similar to what kernelci use * TAP - test anything protocol * *unit - for the xunit, junit, pyunit family of test frameworks * are these are consistent in their output?
|
Candidate formats are: * json - using a schema similar to what kernelci use * TAP - test anything protocol * *unit - for the xunit, junit, pyunit family of test frameworks * are these are consistent in their output?
|
|
Here are projects that promote or use a single test output format: * https://packages.debian.org/sid/autopkgtest
|
Here are projects that promote or use a single test output format: * https://packages.debian.org/sid/autopkgtest * https://wiki.yoctoproject.org/wiki/Ptest * https://testanything.org/ (Test Anything Protocol) * http://pyunit.sourceforge.net/
|
|
Notes on these:= ptest =Yocto/OpenEmbedded's ptest framework decouples the compiling phasefrom the testing phase and produces "a consistent output format".
|
Notes on these:= ptest =Yocto/OpenEmbedded's ptest framework decouples the compiling phasefrom the testing phase and produces "a consistent output format".
|
|
Ptest normally uses the test suite that comes with the original source code.For example, for the openssh recipe it uses some of the tests inside openssh's"regress" folder.
|
Ptest normally uses the test suite that comes with the original source code.For example, for the openssh recipe it uses some of the tests inside openssh's"regress" folder.
|
|
However, there are many recipes without their corresponding ptest definitions. I'm not sure if that is just because nobody added them yet, or becausethere was no test suite in the original source code.
|
However, there are many recipes without their corresponding ptest definitions. I'm not sure if that is just because nobody added them yet, or becausethere was no test suite in the original source code.
|
|
|
= TAP =Test Anything Protocol
|
|
https://testanything.org/tap-version-13-specification.html
|
https://testanything.org/tap-version-13-specification.html
|
|
Example:{{{TAP version 131..6## Create a new Board and Tile, then place# the Tile onto the board.#ok 1 - The object isa Boardok 2 - Board size is zerook 3 - The object isa Tileok 4 - Get possible places to put the Tilenot ok 5 - Placing the tile produces no error --- message: Could not place tile at row 1, column 1 severity: fail ...ok 6 - Board size is 1}}}
|
Example:{{{TAP version 131..6## Create a new Board and Tile, then place# the Tile onto the board.#ok 1 - The object isa Boardok 2 - Board size is zerook 3 - The object isa Tileok 4 - Get possible places to put the Tilenot ok 5 - Placing the tile produces no error --- message: Could not place tile at row 1, column 1 severity: fail ...ok 6 - Board size is 1}}}
|