Go to main content

man pages section 7: Standards, Environments, Macros, Character Sets, and Miscellany

Exit Print View

Updated: Wednesday, August 8, 2018
 
 

common_test (7erl)

Name

common_test - A framework for automated testing of any target nodes.

Synopsis

Please see following description for synopsis

Description

common_test(7)             Erlang Module Definition             common_test(7)



NAME
       common_test - A framework for automated testing of any target nodes.

DESCRIPTION
       The  Common  Test framework is an environment for implementing and per-
       forming automatic and semi-automatic execution of test cases.

       In brief, Common Test supports:

         * Automated execution of test suites (sets of test cases)

         * Logging of events during execution

         * HTML presentation of test suite results

         * HTML presentation of test suite code

         * Support functions for test suite authors

         * Step-by-step execution of test cases

       The following section describes the mandatory and optional  test  suite
       functions  that  Common  Test  calls  during  test  execution. For more
       details, see section Writing Test Suites in the User's Guide.

TEST CASE CALLBACK FUNCTIONS
       The following functions define the callback interface for a test suite.

EXPORTS
       Module:all() -> Tests | {skip,Reason}

              Types:

                 Tests  =  [TestCase  |  {group,GroupName}   |   {group,Group-
                 Name,Properties} | {group,GroupName,Properties,SubGroups}]
                 TestCase = atom()
                 GroupName = atom()
                 Properties = [parallel | sequence | Shuffle | {RepeatType,N}]
                 | default
                 SubGroups  =  [{GroupName,Properties}  |   {GroupName,Proper-
                 ties,SubGroups}]
                 Shuffle = shuffle | {shuffle,Seed}
                 Seed = {integer(),integer(),integer()}
                 RepeatType     =     repeat     |    repeat_until_all_ok    |
                 repeat_until_all_fail      |      repeat_until_any_ok       |
                 repeat_until_any_fail
                 N = integer() | forever
                 Reason = term()

              MANDATORY

              Returns  the  list of all test cases and test case groups in the
              test suite module to be executed. This list also  specifies  the
              order  the  cases and groups are executed by Common Test. A test
              case is represented by an atom, the name of the test case  func-
              tion.  A  test case group is represented by a group tuple, where
              GroupName, an atom,  is  the  name  of  the  group  (defined  in
              groups/0).  Execution  properties  for groups can also be speci-
              fied, both for a top-level group and for any of  its  subgroups.
              Group execution properties specified here override properties in
              the group definition (see groups/0). (With  value  default,  the
              group definition properties are used).

              If  {skip,Reason}  is returned, all test cases in the module are
              skipped and Reason is printed on the HTML result page.

              For details on groups, see  section  Test  Case  Groups  in  the
              User's Guide.

       Module:groups() -> GroupDefs

              Types:

                 GroupDefs = [Group]
                 Group = {GroupName,Properties,GroupsAndTestCases}
                 GroupName = atom()
                 Properties = [parallel | sequence | Shuffle | {RepeatType,N}]
                 GroupsAndTestCases = [Group | {group,GroupName} | TestCase]
                 TestCase = atom()
                 Shuffle = shuffle | {shuffle,Seed}
                 Seed = {integer(),integer(),integer()}
                 RepeatType     =     repeat     |    repeat_until_all_ok    |
                 repeat_until_all_fail      |      repeat_until_any_ok       |
                 repeat_until_any_fail
                 N = integer() | forever

              OPTIONAL

              Defines  test  case  groups.  For details, see section Test Case
              Groups in the User's Guide.

       Module:suite() -> [Info]

              Types:

                 Info    =    {timetrap,Time}    |    {require,Required}     |
                 {require,Name,Required}  | {userdata,UserData} | {silent_con-
                 nections,Conns} | {stylesheet,CSSFile} | {ct_hooks, CTHs}
                 Time = TimeVal | TimeFunc
                 TimeVal = MilliSec  |  {seconds,integer()}  |  {minutes,inte-
                 ger()} | {hours,integer()}
                 TimeFunc = {Mod,Func,Args} | Fun
                 MilliSec = integer()
                 Mod = atom()
                 Func = atom()
                 Args = list()
                 Fun = fun()
                 Required  =  Key  |  {Key,SubKeys} | {Key,SubKey} | {Key,Sub-
                 Key,SubKeys}
                 Key = atom()
                 SubKeys = SubKey | [SubKey]
                 SubKey = atom()
                 Name = atom()
                 UserData = term()
                 Conns = [atom()]
                 CSSFile = string()
                 CTHs = [CTHModule |
                  {CTHModule, CTHInitArgs} |
                  {CTHModule, CTHInitArgs, CTHPriority}]
                 CTHModule = atom()
                 CTHInitArgs = term()

              OPTIONAL

              The test suite information function. Returns a  list  of  tagged
              tuples specifying various properties related to the execution of
              this test suite (common for all test cases in the suite).

              Tag timetrap sets the  maximum  time  that  each  test  case  is
              allowed    to   execute   (including   init_per_testcase/2   and
              end_per_testcase/2). If the timetrap time is exceeded, the  test
              case fails with reason timetrap_timeout. A TimeFunc function can
              be used to set a new timetrap by returning  a  TimeVal.  It  can
              also  be  used to trigger a timetrap time-out by, at some point,
              returning a value other than a TimeVal. For details, see section
              Timetrap Time-Outs in the User's Guide.

              Tag  require  specifies configuration variables required by test
              cases (or configuration functions) in the suite. If the required
              configuration  variables  are not found in any of the configura-
              tion files, all test cases are skipped. For  details  about  the
              require functionality, see funtion ct:require/1,2.

              With  userdata,  the  user  can  specify  any test suite-related
              information, which can be read by calling ct:userdata/2.

              Tag ct_hooks specifies the Common Test Hooks to be run with this
              suite.

              Other tuples than the ones defined are ignored.

              For  details about the test suite information function, see sec-
              tion Test Suite Information Function in the User's Guide.

       Module:init_per_suite(Config)   ->   NewConfig   |   {skip,Reason}    |
       {skip_and_save,Reason,SaveConfig}

              Types:

                 Config = NewConfig = SaveConfig = [{Key,Value}]
                 Key = atom()
                 Value = term()
                 Reason = term()

              OPTIONAL; if this function is defined, then end_per_suite/1 must
              also be defined.

              This configuration function is called as the first  function  in
              the suite. It typically contains initializations that are common
              for all test cases in the suite, and  that  must  only  be  done
              once.  Parameter  Config  is  the configuration data that can be
              modified. Whatever is returned from this function  is  specified
              as  Config  to all configuration functions and test cases in the
              suite.

              If {skip,Reason} is returned, all test cases in  the  suite  are
              skipped and Reason is printed in the overview log for the suite.

              For  information  on  save_config and skip_and_save, see section
              Saving Configuration Data in the User's Guide.

       Module:end_per_suite(Config) -> term() | {save_config,SaveConfig}

              Types:

                 Config = SaveConfig = [{Key,Value}]
                 Key = atom()
                 Value = term()

              OPTIONAL; if this function  is  defined,  then  init_per_suite/1
              must also be defined.

              This  function  is called as the last test case in the suite. It
              is meant to be used for cleaning up after init_per_suite/1.

              For information on save_config, see section Saving Configuration
              Data in the User's Guide.

       Module:group(GroupName) -> [Info]

              Types:

                 Info     =    {timetrap,Time}    |    {require,Required}    |
                 {require,Name,Required} | {userdata,UserData} |  {silent_con-
                 nections,Conns} | {stylesheet,CSSFile} | {ct_hooks, CTHs}
                 Time = TimeVal | TimeFunc
                 TimeVal  =  MilliSec  |  {seconds,integer()} | {minutes,inte-
                 ger()} | {hours,integer()}
                 TimeFunc = {Mod,Func,Args} | Fun
                 MilliSec = integer()
                 Mod = atom()
                 Func = atom()
                 Args = list()
                 Fun = fun()
                 Required = Key | {Key,SubKeys}  |  {Key,Subkey}  |  {Key,Sub-
                 key,SubKeys}
                 Key = atom()
                 SubKeys = SubKey | [SubKey]
                 SubKey = atom()
                 Name = atom()
                 UserData = term()
                 Conns = [atom()]
                 CSSFile = string()
                 CTHs = [CTHModule |
                  {CTHModule, CTHInitArgs} |
                  {CTHModule, CTHInitArgs, CTHPriority}]
                 CTHModule = atom()
                 CTHInitArgs = term()

              OPTIONAL

              The  test  case  group  information  function. It is supposed to
              return a list of tagged tuples that specify  various  properties
              related to the execution of a test case group (that is, its test
              cases and subgroups). Properties set by group/1 override proper-
              ties with the same key that have been set previously by suite/0.

              Tag  timetrap  sets  the  maximum  time  that  each test case is
              allowed   to   execute   (including   init_per_testcase/2    and
              end_per_testcase/2).  If the timetrap time is exceeded, the test
              case fails with reason timetrap_timeout. A TimeFunc function can
              be  used  to  set  a new timetrap by returning a TimeVal. It can
              also be used to trigger a timetrap time-out by, at  some  point,
              returning a value other than a TimeVal. For details, see section
              Timetrap Time-Outs in the User's Guide.

              Tag require specifies configuration variables required  by  test
              cases (or configuration functions) in the suite. If the required
              configuration variables are not found in any of  the  configura-
              tion  files,  all  test  cases  in  this  group are skipped. For
              details  about   the   require   functionality,   see   function
              ct:require/1,2.

              With  userdata, the user can specify any test case group related
              information that can be read by calling ct:userdata/2.

              Tag ct_hooks specifies the Common Test Hooks to be run with this
              suite.

              Other tuples than the ones defined are ignored.

              For  details about the test case group information function, see
              section Group Information Function in the User's Guide.

       Module:init_per_group(GroupName, Config) -> NewConfig | {skip,Reason}

              Types:

                 GroupName = atom()
                 Config = NewConfig = [{Key,Value}]
                 Key = atom()
                 Value = term()
                 Reason = term()

              OPTIONAL; if this function is defined, then end_per_group/2 must
              also be defined.

              This configuration function is called before execution of a test
              case group. It typically contains initializations that are  com-
              mon for all test cases and subgroups in the group, and that must
              only be performed once. GroupName is the name of the  group,  as
              specified in the group definition (see groups/0). Parameter Con-
              fig is the configuration data that can be modified.  The  return
              value  of this function is given as Config to all test cases and
              subgroups in the group.

              If {skip,Reason} is returned, all test cases in  the  group  are
              skipped and Reason is printed in the overview log for the group.

              For  information  about  test case groups, see section Test Case
              Groups in the User's Guide.

       Module:end_per_group(GroupName,      Config)      ->      term()      |
       {return_group_result,Status}

              Types:

                 GroupName = atom()
                 Config = [{Key,Value}]
                 Key = atom()
                 Value = term()
                 Status = ok | skipped | failed

              OPTIONAL;  if  this  function  is defined, then init_per_group/2
              must also be defined.

              This function is called after the execution of a test case group
              is  finished.  It  is  meant  to  be  used for cleaning up after
              init_per_group/2. A status value for a nested  subgroup  can  be
              returned  with  {return_group_result,Status}.  The status can be
              retrieved in end_per_group/2 for the group on the  level  above.
              The status is also used by Common Test for deciding if execution
              of a group is to proceed if property sequence or  repeat_until_*
              is set.

              For details about test case groups, see section Test Case Groups
              in the User's Guide.

       Module:init_per_testcase(TestCase, Config) -> NewConfig | {fail,Reason}
       | {skip,Reason}

              Types:

                  TestCase = atom()
                  Config = NewConfig = [{Key,Value}]
                  Key = atom()
                  Value = term()
                  Reason = term()

              OPTIONAL;  if  this function is defined, then end_per_testcase/2
              must also be defined.

              This function is called before each test case. Argument TestCase
              is  the test case name, and Config (list of key-value tuples) is
              the configuration data that can be modified. The NewConfig  list
              returned from this function is given as Config to the test case.
              If {fail,Reason} is returned, the test case is marked as  failed
              without being executed.

              If  {skip,Reason} is returned, the test case is skipped and Rea-
              son is printed in the overview log for the suite.

       Module:end_per_testcase(TestCase, Config) -> term() |  {fail,Reason}  |
       {save_config,SaveConfig}

              Types:

                 TestCase = atom()
                 Config = SaveConfig = [{Key,Value}]
                 Key = atom()
                 Value = term()
                 Reason = term()

              OPTIONAL;  if this function is defined, then init_per_testcase/2
              must also be defined.

              This function is called after each test case, and can be used to
              clean up after init_per_testcase/2 and the test case. Any return
              value (besides {fail,Reason}  and  {save_config,SaveConfig})  is
              ignored.  By  returning  {fail,Reason},  TestCase  is  marked as
              faulty (even though it was  successful  in  the  sense  that  it
              returned a value instead of terminating).

              For information on save_config, see section Saving Configuration
              Data in the User's Guide.

       Module:Testcase() -> [Info]

              Types:

                 Info    =    {timetrap,Time}    |    {require,Required}     |
                 {require,Name,Required}  | {userdata,UserData} | {silent_con-
                 nections,Conns}
                 Time = TimeVal | TimeFunc
                 TimeVal = MilliSec  |  {seconds,integer()}  |  {minutes,inte-
                 ger()} | {hours,integer()}
                 TimeFunc = {Mod,Func,Args} | Fun
                 MilliSec = integer()
                 Mod = atom()
                 Func = atom()
                 Args = list()
                 Fun = fun()
                 Required  =  Key  |  {Key,SubKeys} | {Key,Subkey} | {Key,Sub-
                 key,SubKeys}
                 Key = atom()
                 SubKeys = SubKey | [SubKey]
                 SubKey = atom()
                 Name = atom()
                 UserData = term()
                 Conns = [atom()]

              OPTIONAL

              The test case information function. It is supposed to  return  a
              list of tagged tuples that specify various properties related to
              the execution of this particular test case.  Properties  set  by
              Testcase/0  override properties set previously for the test case
              by group/1 or suite/0.

              Tag timetrap sets the maximum time that the test case is allowed
              to  execute.  If  the  timetrap  time is exceeded, the test case
              fails  with  reason  timetrap_timeout.  init_per_testcase/2  and
              end_per_testcase/2 are included in the timetrap time. A TimeFunc
              function can be used to  set  a  new  timetrap  by  returning  a
              TimeVal.  It can also be used to trigger a timetrap time-out by,
              at some point, returning a  value  other  than  a  TimeVal.  For
              details, see section Timetrap Time-Outs in the User's Guide.

              Tag  require specifies configuration variables that are required
              by the test case (or init_per_testcase/2 or end_per_testcase/2).
              If  the required configuration variables are not found in any of
              the configuration files, the test case is skipped.  For  details
              about the require functionality, see function ct:require/1,2.

              If  timetrap or require is not set, the default values specified
              by suite/0 (or group/1) are used.

              With userdata, the user can specify any test case-related infor-
              mation that can be read by calling ct:userdata/3.

              Other tuples than the ones defined are ignored.

              For  details  about the test case information function, see sec-
              tion Test Case Information Function in the User's Guide.

       Module:Testcase(Config) -> term() | {skip,Reason} | {comment,Comment} |
       {save_config,SaveConfig} | {skip_and_save,Reason,SaveConfig} | exit()

              Types:

                 Config = SaveConfig = [{Key,Value}]
                 Key = atom()
                 Value = term()
                 Reason = term()
                 Comment = string()

              MANDATORY

              The  implementation  of  a test case. Call the functions to test
              and check the result. If something fails,  ensure  the  function
              causes  a  runtime  error or call ct:fail/1,2 (which also causes
              the test case process to terminate).

              Elements from the Config list can, for  example,  be  read  with
              proplists:get_value/2 in STDLIB (or the macro ?config defined in
              ct.hrl).

              If you decide not  to  run  the  test  case  after  all,  return
              {skip,Reason}.  Reason  is  then printed in field Comment on the
              HTML result page.

              To print some information in field Comment on  the  HTML  result
              page, return {comment,Comment}.

              If  the function returns anything else, the test case is consid-
              ered successful. The return value always  gets  printed  in  the
              test case log file.

              For  details  about  test  case implementation, see section Test
              Cases in the User's Guide.

              For information on save_config and  skip_and_save,  see  section
              Saving Configuration Data in the User's Guide.



Ericsson AB                    common_test 1.14                 common_test(7)