ct

(common_test)

Main user interface for the Common Test framework.

Main user interface for the Common Test framework.

This module implements the command line interface for running tests and some basic functions for common test case issues such as configuration and logging.

Test Suite Support Macros

The config macro is defined in ct.hrl. This macro should be used to retrieve information from the Config variable sent to all test cases. It is used with two arguments, where the first is the name of the configuration variable you wish to retrieve, and the second is the Config variable supplied to the test case.

Possible configuration variables include:

  • data_dir - Data file directory.

  • priv_dir - Scratch file directory.

  • Whatever added by init_per_suite/1 or init_per_testcase/2 in the test suite.

DATA TYPES

handle() = handle() (see module ct_gen_conn) | term()

The identity of a specific connection.

target_name() = var_name()

The name of a target.

var_name() = atom()

A variable name which is specified when ct:require/2 is called, e.g. ct:require(mynodename,{node,[telnet]})

Functions

abort_current_testcase(Reason) -> ok | {error, no_testcase_running}

  • Reason = term()

When calling this function, the currently executing test case will be aborted. It is the user's responsibility to know for sure which test case is currently executing. The function is therefore only safe to call from a function which has been called (or synchronously invoked) by the test case.

Reason, the reason for aborting the test case, is printed in the test case log.

add_config(Callback, Config) -> ok | {error, Reason}

  • Callback = atom()
  • Config = string()
  • Reason = term()

This function loads configuration variables using the given callback module and configuration string. Callback module should be either loaded or present in the code part. Loaded configuration variables can later be removed using remove_config/2 function.

comment(Comment) -> void()

  • Comment = term()

Print the given Comment in the comment field of the table on the test suite result page.

If called several times, only the last comment is printed. comment/1 is also overwritten by the return value {comment,Comment} or by the function fail/1 (which prints Reason as a comment).

decrypt_config_file(EncryptFileName, TargetFileName) -> ok | {error, Reason}

  • EncryptFileName = string()
  • TargetFileName = string()
  • Reason = term()

This function decrypts EncryptFileName, previously generated with encrypt_config_file/2/3. The original file contents is saved in the target file. The encryption key, a string, must be available in a text file named .ct_config.crypt in the current directory, or the home directory of the user (it is searched for in that order).

decrypt_config_file(EncryptFileName, TargetFileName, KeyOrFile) -> ok | {error, Reason}

  • EncryptFileName = string()
  • TargetFileName = string()
  • KeyOrFile = {key, string()} | {file, string()}
  • Reason = term()

This function decrypts EncryptFileName, previously generated with encrypt_config_file/2/3. The original file contents is saved in the target file. The key must have the the same value as that used for encryption.

encrypt_config_file(SrcFileName, EncryptFileName) -> ok | {error, Reason}

  • SrcFileName = string()
  • EncryptFileName = string()
  • Reason = term()

This function encrypts the source config file with DES3 and saves the result in file EncryptFileName. The key, a string, must be available in a text file named .ct_config.crypt in the current directory, or the home directory of the user (it is searched for in that order).

See the Common Test User's Guide for information about using encrypted config files when running tests.

See the crypto application for details on DES3 encryption/decryption.

encrypt_config_file(SrcFileName, EncryptFileName, KeyOrFile) -> ok | {error, Reason}

  • SrcFileName = string()
  • EncryptFileName = string()
  • KeyOrFile = {key, string()} | {file, string()}
  • Reason = term()

This function encrypts the source config file with DES3 and saves the result in the target file EncryptFileName. The encryption key to use is either the value in {key,Key} or the value stored in the file specified by {file,File}.

See the Common Test User's Guide for information about using encrypted config files when running tests.

See the crypto application for details on DES3 encryption/decryption.

fail(Reason) -> void()

  • Reason = term()

Terminate a test case with the given error Reason.

get_config(Required) -> Value

Equivalent to get_config(Required, undefined, []).

get_config(Required, Default) -> Value

Equivalent to get_config(Required, Default, []).

get_config(Required, Default, Opts) -> ValueOrElement

  • Required = KeyOrName | {KeyOrName, SubKey}
  • KeyOrName = atom()
  • SubKey = atom()
  • Default = term()
  • Opts = [Opt] | []
  • Opt = element | all
  • ValueOrElement = term() | Default

Read config data values.

This function returns the matching value(s) or config element(s), given a config variable key or its associated name (if one has been specified with require/2 or a require statement).

Example, given the following config file:

   {unix,[{telnet,IpAddr},
          {username,Username},
          {password,Password}]}.

get_config(unix,Default) -> [{telnet,IpAddr}, {username,Username}, {password,Password}]
get_config({unix,telnet},Default) -> IpAddr
get_config({unix,ftp},Default) -> Default
get_config(unknownkey,Default) -> Default

If a config variable key has been associated with a name (by means of require/2 or a require statement), the name may be used instead of the key to read the value:

require(myhost,unix) -> ok
get_config(myhost,Default) -> [{telnet,IpAddr}, {username,Username}, {password,Password}]

If a config variable is defined in multiple files and you want to access all possible values, use the all option. The values will be returned in a list and the order of the elements corresponds to the order that the config files were specified at startup.

If you want config elements (key-value tuples) returned as result instead of values, use the element option. The returned elements will then be on the form {KeyOrName,Value}, or (in case a subkey has been specified) {{KeyOrName,SubKey},Value}

See also: get_config/1, get_config/2, require/1, require/2.

get_status() -> TestStatus | {error, Reason} | no_tests_running

  • TestStatus = [StatusElem]
  • StatusElem = {current, {Suite, TestCase}} | {successful, Successful} | {failed, Failed} | {skipped, Skipped} | {total, Total}
  • Suite = atom()
  • TestCase = atom()
  • Successful = integer()
  • Failed = integer()
  • Skipped = {UserSkipped, AutoSkipped}
  • UserSkipped = integer()
  • AutoSkipped = integer()
  • Total = integer()
  • Reason = term()

Returns status of ongoing test. The returned list contains info about which test case is currently executing, as well as counters for successful, failed, skipped, and total test cases so far.

get_target_name(Handle) -> {ok, TargetName} | {error, Reason}

  • Handle = handle()
  • TargetName = target_name()

Return the name of the target that the given connection belongs to.

install(Opts) -> ok | {error, Reason}

  • Opts = [Opt]
  • Opt = {config, ConfigFiles} | {event_handler, Modules} | {decrypt, KeyOrFile}
  • ConfigFiles = [ConfigFile]
  • ConfigFile = string()
  • Modules = [atom()]
  • KeyOrFile = {key, Key} | {file, KeyFile}
  • Key = string()
  • KeyFile = string()

Install config files and event handlers.

Run this function once before first test.

Example:
install([{config,["config_node.ctc","config_user.ctc"]}]).

Note that this function is automatically run by the ct_run program.

listenv(Telnet) -> [Env]

  • Telnet = term()
  • Env = {Key, Value}
  • Key = string()
  • Value = string()

Performs the listenv command on the given telnet connection and returns the result as a list of Key-Value pairs.

log(Format) -> ok

Equivalent to log(default, Format, []).

log(X1, X2) -> ok

  • X1 = Category | Format
  • X2 = Format | Args

Equivalent to log(Category, Format, Args).

log(Category, Format, Args) -> ok

  • Category = atom()
  • Format = string()
  • Args = list()

Printout from a testcase to the log.

This function is meant for printing stuff directly from a testcase (i.e. not from within the CT framework) in the test log.

Default Category is default and default Args is [].

pal(Format) -> ok

Equivalent to pal(default, Format, []).

pal(X1, X2) -> ok

  • X1 = Category | Format
  • X2 = Format | Args

Equivalent to pal(Category, Format, Args).

pal(Category, Format, Args) -> ok

  • Category = atom()
  • Format = string()
  • Args = list()

Print and log from a testcase.

This function is meant for printing stuff from a testcase both in the log and on the console.

Default Category is default and default Args is [].

parse_table(Data) -> {Heading, Table}

  • Data = [string()]
  • Heading = tuple()
  • Table = [tuple()]

Parse the printout from an SQL table and return a list of tuples.

The printout to parse would typically be the result of a select command in SQL. The returned Table is a list of tuples, where each tuple is a row in the table.

Heading is a tuple of strings representing the headings of each column in the table.

print(Format) -> ok

Equivalent to print(default, Format, []).

print(X1, X2) -> term()

Equivalent to print(Category, Format, Args).

print(Category, Format, Args) -> ok

  • Category = atom()
  • Format = string()
  • Args = list()

Printout from a testcase to the console.

This function is meant for printing stuff from a testcase on the console.

Default Category is default and default Args is [].

reload_config(Required) -> ValueOrElement

  • Required = KeyOrName | {KeyOrName, SubKey}
  • KeyOrName = atom()
  • SubKey = atom()
  • ValueOrElement = term()

Reload config file which contains specified configuration key.

This function performs updating of the configuration data from which the given configuration variable was read, and returns the (possibly) new value of this variable.

Note that if some variables were present in the configuration but are not loaded using this function, they will be removed from the configuration table together with their aliases.

remove_config(Callback, Config) -> ok

  • Callback = atom()
  • Config = string()
  • Reason = term()

This function removes configuration variables (together with their aliases) which were loaded with specified callback module and configuration string.

require(Required) -> ok | {error, Reason}

  • Required = Key | {Key, SubKeys}
  • Key = atom()
  • SubKeys = SubKey | [SubKey]
  • SubKey = atom()

Check if the required configuration is available.

Example: require the variable myvar:
ok = ct:require(myvar)

In this case the config file must at least contain:

   {myvar,Value}.

Example: require the variable myvar with subvariable sub1:
ok = ct:require({myvar,sub1})

In this case the config file must at least contain:

   {myvar,[{sub1,Value}]}.

See also: get_config/1, get_config/2, get_config/3, require/2.

require(Name, Required) -> ok | {error, Reason}

  • Name = atom()
  • Required = Key | {Key, SubKeys}
  • Key = atom()
  • SubKeys = SubKey | [SubKey]
  • SubKey = atom()

Check if the required configuration is available, and give it a name.

If the requested data is available, the main entry will be associated with Name so that the value of the element can be read with get_config/1,2 provided Name instead of the Key.

Example: Require one node with a telnet connection and an ftp connection. Name the node a:
ok = ct:require(a,{node,[telnet,ftp]}).
All references to this node may then use the node name. E.g. you can fetch a file over ftp like this:
ok = ct:ftp_get(a,RemoteFile,LocalFile).

For this to work, the config file must at least contain:

   {node,[{telnet,IpAddr},
          {ftp,IpAddr}]}.

See also: get_config/1, get_config/2, get_config/3, require/1.

run(TestDirs) -> Result

  • TestDirs = TestDir | [TestDir]

Run all testcases in all suites in the given directories.

See also: run/3.

run(TestDir, Suite) -> Result

Run all testcases in the given suite.

See also: run/3.

run(TestDir, Suite, Cases) -> Result

  • TestDir = string()
  • Suite = atom()
  • Cases = atom() | [atom()]
  • Result = [TestResult] | {error, Reason}

Run the given testcase(s).

Requires that ct:install/1 has been run first.

Suites (*_SUITE.erl) files must be stored in TestDir or TestDir/test. All suites will be compiled when test is run.

run_test(Opts) -> Result

  • Opts = [OptTuples]
  • OptTuples = {dir, TestDirs} | {suite, Suites} | {group, Groups} | {testcase, Cases} | {spec, TestSpecs} | {label, Label} | {config, CfgFiles} | {userconfig, UserConfig} | {allow_user_terms, Bool} | {logdir, LogDir} | {silent_connections, Conns} | {stylesheet, CSSFile} | {cover, CoverSpecFile} | {step, StepOpts} | {event_handler, EventHandlers} | {include, InclDirs} | {auto_compile, Bool} | {multiply_timetraps, M} | {scale_timetraps, Bool} | {repeat, N} | {duration, DurTime} | {until, StopTime} | {force_stop, Bool} | {decrypt, DecryptKeyOrFile} | {refresh_logs, LogDir} | {basic_html, Bool} | {ct_hooks, CTHs}
  • TestDirs = [string()] | string()
  • Suites = [string()] | string()
  • Cases = [atom()] | atom()
  • Groups = [atom()] | atom()
  • TestSpecs = [string()] | string()
  • Label = string() | atom()
  • CfgFiles = [string()] | string()
  • UserConfig = [{CallbackMod, CfgStrings}] | {CallbackMod, CfgStrings}
  • CallbackMod = atom()
  • CfgStrings = [string()] | string()
  • LogDir = string()
  • Conns = all | [atom()]
  • CSSFile = string()
  • CoverSpecFile = string()
  • StepOpts = [StepOpt] | []
  • StepOpt = config | keep_inactive
  • EventHandlers = EH | [EH]
  • EH = atom() | {atom(), InitArgs} | {[atom()], InitArgs}
  • InitArgs = [term()]
  • InclDirs = [string()] | string()
  • M = integer()
  • N = integer()
  • DurTime = string(HHMMSS)
  • StopTime = string(YYMoMoDDHHMMSS) | string(HHMMSS)
  • DecryptKeyOrFile = {key, DecryptKey} | {file, DecryptFile}
  • DecryptKey = string()
  • DecryptFile = string()
  • CTHs = [CTHModule | {CTHModule, CTHInitArgs}]
  • CTHModule = atom()
  • CTHInitArgs = term()
  • Result = [TestResult] | {error, Reason}

Run tests as specified by the combination of options in Opts. The options are the same as those used with the ct_run program. Note that here a TestDir can be used to point out the path to a Suite. Note also that the option testcase corresponds to the -case option in the ct_run program. Configuration files specified in Opts will be installed automatically at startup.

run_testspec(TestSpec) -> Result

  • TestSpec = [term()]

Run test specified by TestSpec. The terms are the same as those used in test specification files.

sleep(Time) -> ok

  • Time = {hours, Hours} | {minutes, Mins} | {seconds, Secs} | Millisecs | infinity
  • Hours = integer()
  • Mins = integer()
  • Secs = integer()
  • Millisecs = integer() | float()

This function, similar to timer:sleep/1, suspends the test case for specified time. However, this function also multiplies Time with the 'multiply_timetraps' value (if set) and under certain circumstances also scales up the time automatically if 'scale_timetraps' is set to true (default is false).

start_interactive() -> ok

Start CT in interactive mode.

From this mode all test case support functions can be executed directly from the erlang shell. The interactive mode can also be started from the OS command line with ct_run -shell [-config File...].

If any functions using "required config data" (e.g. telnet or ftp functions) are to be called from the erlang shell, config data must first be required with ct:require/2.

Example:
> ct:require(unix_telnet, unix).
ok
> ct_telnet:open(unix_telnet).
{ok,<0.105.0>}
> ct_telnet:cmd(unix_telnet, "ls .").
{ok,["ls","file1 ...",...]}

step(TestDir, Suite, Case) -> Result

  • Case = atom()

Step through a test case with the debugger.

See also: run/3.

step(TestDir, Suite, Case, Opts) -> Result

  • Case = atom()
  • Opts = [Opt] | []
  • Opt = config | keep_inactive

Step through a test case with the debugger. If the config option has been given, breakpoints will be set also on the configuration functions in Suite.

See also: run/3.

stop_interactive() -> ok

Exit the interactive mode.

See also: start_interactive/0.

testcases(TestDir, Suite) -> Testcases | {error, Reason}

  • TestDir = string()
  • Suite = atom()
  • Testcases = list()
  • Reason = term()

Returns all testcases in the specified suite.

timetrap(Time) -> ok

  • Time = {hours, Hours} | {minutes, Mins} | {seconds, Secs} | Millisecs | infinity
  • Hours = integer()
  • Mins = integer()
  • Secs = integer()
  • Millisecs = integer() | float()

Use this function to set a new timetrap for the running test case.

userdata(TestDir, Suite) -> SuiteUserData | {error, Reason}

  • TestDir = string()
  • Suite = atom()
  • SuiteUserData = [term()]
  • Reason = term()

Returns any data specified with the tag userdata in the list of tuples returned from Suite:suite/0.

userdata(TestDir, Suite, Case) -> TCUserData | {error, Reason}

  • TestDir = string()
  • Suite = atom()
  • Case = atom()
  • TCUserData = [term()]
  • Reason = term()

Returns any data specified with the tag userdata in the list of tuples returned from Suite:Case/0.

View Functions