ct_property_test - based tests.
Please see following description for synopsis
ct_property_test(3) Erlang Module Definition ct_property_test(3)
NAME
ct_property_test - Support in Common Test for running property-based
tests.
DESCRIPTION
This module helps running property-based tests in the Common Test
framework. One (or more) of the property testing tools
* QuickCheck,
* PropEr or
* Triq
is assumed to be installed.
The idea with this module is to have a Common Test test suite calling a
property testing tool with special property test suites as defined by
that tool. The tests are collected in the test directory of the appli-
cation. The test directory has a subdirectory property_test, where
everything needed for the property tests are collected. The usual
Erlang application directory structure is assumed.
A typical Common Test test suite using ct_property_test is organized as
follows:
-module(my_prop_test_SUITE).
-compile(export_all).
-include_lib("common_test/include/ct.hrl").
all() -> [prop_ftp_case].
init_per_suite(Config) ->
ct_property_test:init_per_suite(Config).
%%%---- test case
prop_ftp_case(Config) ->
ct_property_test:quickcheck(
ftp_simple_client_server:prop_ftp(),
Config
).
and the the property test module (in this example ftp_sim-
ple_client_server.erl) as almost a usual property testing module (More
examples are in the User's Guide):
-module(ftp_simple_client_server).
-export([prop_ftp/0...]).
-include_lib("common_test/include/ct_property_test.hrl").
prop_ftp() ->
?FORALL( ....
EXPORTS
init_per_suite(Config) -> Config | {skip, Reason}
Initializes and extends Config for property based testing.
This function investigates if support is available for either
QuickCheck, PropEr or Triq and compiles the properties with the
first tool found. It is supposed to be called in the
init_per_suite/1 function in a CommonTest test suite.
Which tools to check for, and in which order could be set with
the option {prop_tools, list(eqc|proper|triq)} in the CommonTest
configuration Config. The default value is [eqc, proper, triq]
with eqc being the first one searched for.
If no support is found for any tool, this function returns
{skip, Explanation}.
If support is found, the option {property_test_tool,ToolModule}
with the selected tool main module name (eqc, proper or triq) is
added to the list Config which then is returned.
The property tests are assumed to be in a subdirectory named
property_test. All found Erlang files in that directory are com-
piled with one of the macros 'EQC', 'PROPER' or 'TRIQ' set,
depending on which tool that is first found. This could make
parts of the Erlang property tests code to be included or
excluded with the macro directives -ifdef(Macro). or -ifn-
def(Macro)..
The file(s) in the property_test subdirectory could, or should,
include the ct_property_test include file:
-include_lib("common_test/include/ct_property_test.hrl").
This included file will:
* Include the correct tool's include file
* Set the macro 'MOD_eqc' to the correct module name for the
selected tool. That is, the macro 'MOD_eqc' is set to either
eqc, proper or triq.
quickcheck(Property, Config) -> true | {fail, Reason}
Calls the selected tool's function for running the Property. It
is usually and by historical reasons called quickcheck, and that
is why that name is used in this module (ct_property_test).
The result is returned in a form suitable for Common Test test
suites.
This function is intended to be called in test cases in test
suites.
present_result(Module, Cmds, Triple, Config) -> Result
Same as present_result(Module, Cmds, Triple, Config, [])
present_result(Module, Cmds, Triple, Config, Options) -> Result
Types:
Module = module()
Cmds =
the list of commands generated by the property testing
tool, for example by proper:commands/1 or by proper:paral-
lel_commands/1
Triple =
the output from for example proper:run_commands/2 or
proper:run_parallel_commands/2
Config =
the Common Test Config in test cases.
Options = [present_option()]
present_option() = {print_fun, fun(Format,Args)}
| {spec, StatisticsSpec}
The print_fun defines which function to do the actual
printout. The default is ct:log/2. The spec defines what
statistics are to be printed
Result = boolean()
Is false if the test failed and is true if the test passed
Presents the result of stateful (statem) property testing using
the aggregate function in PropEr, QuickCheck or other similar
property testing tool.
It is assumed to be called inside the property called by
quickcheck/2:
RunResult = run_parallel_commands(?MODULE, Cmds),
ct_property_test:present_result(?MODULE, Cmds, RunResult, Config)
See the User's Guide for an example of the usage and of the
default printout.
The StatisticsSpec is a list of the tuples:
* {Title::string(), CollectFun::fun/1}
* {Title::string(), FrequencyFun::/0, CollectFun::fun/1}
Each tuple will produce one table in the order of their places
in the list.
* Title will be the title of one result table
* CollectFun is called with one argument: the Cmds. It should
return a list of the values to be counted. The following
pre-defined functions exist:
* ct_property_test:cmnd_names/1 returns a list of commands
(function calls) generated in the Cmnd sequence, without
Module, Arguments and other details.
* ct_property_test:num_calls/1 returns a list of the length
of commands lists
* ct_property_test:sequential_parallel/1 returns a list with
information about sequential and parallel parts from
Tool:parallel_commands/1,2
* FrequencyFun/0 returns a fun/1 which is supposed to take a
list of items as input, and return an iolist wich will be
printed as the table. Per default, the number of each item
is counted and the percentage is printed for each. The list
[a,b,a,a,c] could for example return
["a 60%\n","b 20%\n","c 20%\n"]
a 60%
b 20%
c 20%
The default StatisticsSpec is:
* For sequential commands:
[{"Function calls", fun cmnd_names/1},
{"Length of command sequences", fun print_frequency_ranges/0,
fun num_calls/1}]
* For parallel commands:
[{"Distribution sequential/parallel", fun sequential_parallel/1},
{"Function calls", fun cmnd_names/1},
{"Length of command sequences", fun print_frequency_ranges/0,
fun num_calls/1}]
Ericsson AB common_test 1.22 ct_property_test(3)