<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html xmlns:fn="http://www.w3.org/2005/02/xpath-functions">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<link rel="stylesheet" href="../../../../doc/otp_doc.css" type="text/css">
<title>Erlang -- Common Test Hooks</title>
</head>
<body bgcolor="white" text="#000000" link="#0000ff" vlink="#ff00ff" alink="#ff0000"><div id="container">
<script id="js" type="text/javascript" language="JavaScript" src="../../../../doc/js/flipmenu/flipmenu.js"></script><script id="js2" type="text/javascript" src="../../../../doc/js/erlresolvelinks.js"></script><script language="JavaScript" type="text/javascript">
<!--
function getWinHeight() {
var myHeight = 0;
if( typeof( window.innerHeight ) == 'number' ) {
//Non-IE
myHeight = window.innerHeight;
} else if( document.documentElement && ( document.documentElement.clientWidth ||
document.documentElement.clientHeight ) ) {
//IE 6+ in 'standards compliant mode'
myHeight = document.documentElement.clientHeight;
} else if( document.body && ( document.body.clientWidth || document.body.clientHeight ) ) {
//IE 4 compatible
myHeight = document.body.clientHeight;
}
return myHeight;
}
function setscrollpos() {
var objf=document.getElementById('loadscrollpos');
document.getElementById("leftnav").scrollTop = objf.offsetTop - getWinHeight()/2;
}
function addEvent(obj, evType, fn){
if (obj.addEventListener){
obj.addEventListener(evType, fn, true);
return true;
} else if (obj.attachEvent){
var r = obj.attachEvent("on"+evType, fn);
return r;
} else {
return false;
}
}
addEvent(window, 'load', setscrollpos);
//--></script><div id="leftnav"><div class="innertube">
<img alt="Erlang logo" src="../../../../doc/erlang-logo.png"><br><small><a href="users_guide.html">User's Guide</a><br><a href="index.html">Reference Manual</a><br><a href="release_notes.html">Release Notes</a><br><a href="../pdf/common_test-1.6.3.pdf">PDF</a><br><a href="../../../../doc/index.html">Top</a></small><p><strong>Common Test</strong><br><strong>User's Guide</strong><br><small>Version 1.6.3</small></p>
<br><a href="javascript:openAllFlips()">Expand All</a><br><a href="javascript:closeAllFlips()">Contract All</a><p><small><strong>Chapters</strong></small></p>
<ul class="flipMenu" imagepath="../../../../doc/js/flipmenu">
<li id="no" title="Common Test Basics" expanded="false">Common Test Basics<ul>
<li><a href="basics_chapter.html">
Top of chapter
</a></li>
<li title="Introduction"><a href="basics_chapter.html#id66143">Introduction</a></li>
<li title="Test Suite Organisation"><a href="basics_chapter.html#id61412">Test Suite Organisation</a></li>
<li title="Support Libraries"><a href="basics_chapter.html#id67203">Support Libraries</a></li>
<li title="Suites and Test Cases"><a href="basics_chapter.html#id61632">Suites and Test Cases</a></li>
<li title="External Interfaces"><a href="basics_chapter.html#id63930">External Interfaces</a></li>
</ul>
</li>
<li id="no" title="Getting Started" expanded="false">Getting Started<ul>
<li><a href="getting_started_chapter.html">
Top of chapter
</a></li>
<li title="Are you new around here?"><a href="getting_started_chapter.html#id61356">Are you new around here?</a></li>
<li title="Test case execution"><a href="getting_started_chapter.html#id63050">Test case execution</a></li>
<li title="A simple test suite"><a href="getting_started_chapter.html#id63909">A simple test suite</a></li>
<li title="A test suite with configuration functions"><a href="getting_started_chapter.html#id68094">A test suite with configuration functions</a></li>
<li title="What happens next?"><a href="getting_started_chapter.html#id62545">What happens next?</a></li>
</ul>
</li>
<li id="no" title="Installation" expanded="false">Installation<ul>
<li><a href="install_chapter.html">
Top of chapter
</a></li>
<li title="General information"><a href="install_chapter.html#id66699">General information</a></li>
</ul>
</li>
<li id="no" title="Writing Test Suites" expanded="false">Writing Test Suites<ul>
<li><a href="write_test_chapter.html">
Top of chapter
</a></li>
<li title="Support for test suite authors"><a href="write_test_chapter.html#id71900">Support for test suite authors</a></li>
<li title="Test suites"><a href="write_test_chapter.html#id70761">Test suites</a></li>
<li title="Init and end per suite"><a href="write_test_chapter.html#id70806">Init and end per suite</a></li>
<li title="Init and end per test case"><a href="write_test_chapter.html#id70917">Init and end per test case</a></li>
<li title="Test cases"><a href="write_test_chapter.html#id71162">Test cases</a></li>
<li title="Test case info function"><a href="write_test_chapter.html#id71322">Test case info function</a></li>
<li title="Test suite info function"><a href="write_test_chapter.html#id71521">Test suite info function</a></li>
<li title="Test case groups"><a href="write_test_chapter.html#id71611">Test case groups</a></li>
<li title="The parallel property and nested groups"><a href="write_test_chapter.html#id72823">The parallel property and nested groups</a></li>
<li title="Parallel test cases and IO"><a href="write_test_chapter.html#id72860">Parallel test cases and IO</a></li>
<li title="Repeated groups"><a href="write_test_chapter.html#id72908">Repeated groups</a></li>
<li title="Shuffled test case order"><a href="write_test_chapter.html#id73046">Shuffled test case order</a></li>
<li title="Group info function"><a href="write_test_chapter.html#id73105">Group info function</a></li>
<li title="Info functions for init- and end-configuration"><a href="write_test_chapter.html#id73139">Info functions for init- and end-configuration</a></li>
<li title="Data and Private Directories"><a href="write_test_chapter.html#id73202">Data and Private Directories</a></li>
<li title="Execution environment"><a href="write_test_chapter.html#id73327">Execution environment</a></li>
<li title="Timetrap timeouts"><a href="write_test_chapter.html#id73364">Timetrap timeouts</a></li>
<li title="Logging - categories and verbosity levels"><a href="write_test_chapter.html#id73549">Logging - categories and verbosity levels</a></li>
<li title="Illegal dependencies"><a href="write_test_chapter.html#id73756">Illegal dependencies</a></li>
</ul>
</li>
<li id="no" title="Test Structure" expanded="false">Test Structure<ul>
<li><a href="test_structure_chapter.html">
Top of chapter
</a></li>
<li title="Test structure"><a href="test_structure_chapter.html#id73930">Test structure</a></li>
<li title="Skipping test cases"><a href="test_structure_chapter.html#id73944">Skipping test cases</a></li>
<li title="Definition of terms"><a href="test_structure_chapter.html#id74020">Definition of terms</a></li>
</ul>
</li>
<li id="no" title="Examples and Templates" expanded="false">Examples and Templates<ul>
<li><a href="example_chapter.html">
Top of chapter
</a></li>
<li title="Test suite example"><a href="example_chapter.html#id74246">Test suite example</a></li>
<li title="Test suite templates"><a href="example_chapter.html#id74298">Test suite templates</a></li>
</ul>
</li>
<li id="no" title="Running Tests" expanded="false">Running Tests<ul>
<li><a href="run_test_chapter.html">
Top of chapter
</a></li>
<li title="Using the Common Test Framework"><a href="run_test_chapter.html#id74488">Using the Common Test Framework</a></li>
<li title="Automatic compilation of test suites and help modules"><a href="run_test_chapter.html#id74528">Automatic compilation of test suites and help modules</a></li>
<li title="Running tests from the OS command line"><a href="run_test_chapter.html#id74645">Running tests from the OS command line</a></li>
<li title="Running tests from the Erlang shell or from an Erlang program"><a href="run_test_chapter.html#id75142">Running tests from the Erlang shell or from an Erlang program</a></li>
<li title="Test case group execution"><a href="run_test_chapter.html#id75304">Test case group execution</a></li>
<li title="Running the interactive shell mode"><a href="run_test_chapter.html#id75821">Running the interactive shell mode</a></li>
<li title="Step by step execution of test cases with the Erlang Debugger"><a href="run_test_chapter.html#id76003">Step by step execution of test cases with the Erlang Debugger</a></li>
<li title="Test Specifications"><a href="run_test_chapter.html#id76102">Test Specifications</a></li>
<li title="Running tests from the Web based GUI"><a href="run_test_chapter.html#id76643">Running tests from the Web based GUI</a></li>
<li title="Log files"><a href="run_test_chapter.html#id76735">Log files</a></li>
<li title="HTML Style Sheets"><a href="run_test_chapter.html#id77053">HTML Style Sheets</a></li>
<li title="Repeating tests"><a href="run_test_chapter.html#id77201">Repeating tests</a></li>
<li title="Silent Connections"><a href="run_test_chapter.html#id77412">Silent Connections</a></li>
</ul>
</li>
<li id="no" title="External Configuration Data" expanded="false">External Configuration Data<ul>
<li><a href="config_file_chapter.html">
Top of chapter
</a></li>
<li title="General"><a href="config_file_chapter.html#id77631">General</a></li>
<li title="Syntax"><a href="config_file_chapter.html#id77671">Syntax</a></li>
<li title="Requiring and reading configuration data"><a href="config_file_chapter.html#id77695">Requiring and reading configuration data</a></li>
<li title="Using configuration variables defined in multiple files"><a href="config_file_chapter.html#id77839">Using configuration variables defined in multiple files</a></li>
<li title="Encrypted configuration files"><a href="config_file_chapter.html#id77870">Encrypted configuration files</a></li>
<li title="Opening connections by using configuration data"><a href="config_file_chapter.html#id77934">Opening connections by using configuration data</a></li>
<li title="User specific configuration data formats"><a href="config_file_chapter.html#id78002">User specific configuration data formats</a></li>
<li title="Examples of configuration data handling"><a href="config_file_chapter.html#id78216">Examples of configuration data handling</a></li>
<li title="Example of user specific configuration handler"><a href="config_file_chapter.html#id78268">Example of user specific configuration handler</a></li>
</ul>
</li>
<li id="no" title="Code Coverage Analysis" expanded="false">Code Coverage Analysis<ul>
<li><a href="cover_chapter.html">
Top of chapter
</a></li>
<li title="General"><a href="cover_chapter.html#id78384">General</a></li>
<li title="Usage"><a href="cover_chapter.html#id78404">Usage</a></li>
<li title="The cover specification file"><a href="cover_chapter.html#id78510">The cover specification file</a></li>
<li title="Logging"><a href="cover_chapter.html#id78563">Logging</a></li>
</ul>
</li>
<li id="no" title="Using Common Test for Large Scale Testing" expanded="false">Using Common Test for Large Scale Testing<ul>
<li><a href="ct_master_chapter.html">
Top of chapter
</a></li>
<li title="General"><a href="ct_master_chapter.html#id78634">General</a></li>
<li title="Usage"><a href="ct_master_chapter.html#id78668">Usage</a></li>
<li title="Test Specifications"><a href="ct_master_chapter.html#id78815">Test Specifications</a></li>
<li title="Automatic startup of test target nodes"><a href="ct_master_chapter.html#id78987">Automatic startup of test target nodes</a></li>
</ul>
</li>
<li id="no" title="Event Handling" expanded="false">Event Handling<ul>
<li><a href="event_handler_chapter.html">
Top of chapter
</a></li>
<li title="General"><a href="event_handler_chapter.html#id79190">General</a></li>
<li title="Usage"><a href="event_handler_chapter.html#id79226">Usage</a></li>
</ul>
</li>
<li id="no" title="Dependencies between Test Cases and Suites" expanded="false">Dependencies between Test Cases and Suites<ul>
<li><a href="dependencies_chapter.html">
Top of chapter
</a></li>
<li title="General"><a href="dependencies_chapter.html#id80143">General</a></li>
<li title="Saving configuration data"><a href="dependencies_chapter.html#id80265">Saving configuration data</a></li>
<li title="Sequences"><a href="dependencies_chapter.html#id80430">Sequences</a></li>
</ul>
</li>
<li id="loadscrollpos" title="Common Test Hooks" expanded="true">Common Test Hooks<ul>
<li><a href="ct_hooks_chapter.html">
Top of chapter
</a></li>
<li title="General"><a href="ct_hooks_chapter.html#id80622">General</a></li>
<li title="Installing a CTH"><a href="ct_hooks_chapter.html#id80676">Installing a CTH</a></li>
<li title="CTH Scope"><a href="ct_hooks_chapter.html#id80835">CTH Scope</a></li>
<li title="Manipulating tests"><a href="ct_hooks_chapter.html#id81189">Manipulating tests</a></li>
<li title="Example CTH"><a href="ct_hooks_chapter.html#id81482">Example CTH</a></li>
<li title="Built-in CTHs"><a href="ct_hooks_chapter.html#id81535">Built-in CTHs</a></li>
</ul>
</li>
<li id="no" title="Some thoughts about testing" expanded="false">Some thoughts about testing<ul>
<li><a href="why_test_chapter.html">
Top of chapter
</a></li>
<li title="Goals"><a href="why_test_chapter.html#id81722">Goals</a></li>
<li title="What to test?"><a href="why_test_chapter.html#id81742">What to test?</a></li>
</ul>
</li>
</ul>
</div></div>
<div id="content">
<div class="innertube">
<h1>13 Common Test Hooks</h1>
<a name="general"></a>
<h3><a name="id80622">13.1
General</a></h3>
<p>
The <strong>Common Test Hook</strong> (henceforth called CTH) framework allows
extensions of the default behaviour of Common Test by means of hooks
before and after all test suite calls. CTHs allow advanced Common Test
users to abstract out behaviour which is common to multiple test suites
without littering all test suites with library calls. Some example
usages are: logging, starting and monitoring external systems,
building C files needed by the tests and much more!</p>
<p>In brief, Common Test Hooks allows you to:</p>
<ul>
<li>Manipulate the runtime config before each suite
configuration call</li>
<li>Manipulate the return of all suite configuration calls and in
extension the result of the test themselves.</li>
</ul>
<p>The following sections describe how to use CTHs, when they are run
and how to manipulate your test results in a CTH</p>
<div class="warning">
<div class="label">Warning</div>
<div class="content"><p><p>When executing within a CTH all timetraps are shutoff. So
if your CTH never returns, the entire test run will be stalled!</p>
</p></div>
</div>
<a name="installing"></a>
<h3><a name="id80676">13.2
Installing a CTH</a></h3>
<p>There are multiple ways to install a CTH in your test run. You can do it
for all tests in a run, for specific test suites and for specific groups
within a test suite. If you want a CTH to be present in all test suites
within your test run there are three different ways to accomplish that.
</p>
<ul>
<li>Add <span class="code">-ct_hooks</span> as an argument to
<span class="bold_code"><a href="run_test_chapter.html#ct_run">ct_run</a></span>.
To add multiple CTHs using this method append them to each other
using the keyword <span class="code">and</span>, i.e.
<span class="code">ct_run -ct_hooks cth1 [{debug,true}] and cth2 ...</span>.</li>
<li>Add the <span class="code">ct_hooks</span> tag to your
<span class="bold_code"><a href="run_test_chapter.html#test_specifications">
Test Specification</a></span>
</li>
<li>Add the <span class="code">ct_hooks</span> tag to your call to
<span class="bold_code"><a href="ct.html#run_test-1">ct:run_test/1</a></span>
</li>
</ul>
<p>You can also add CTHs within a test suite. This is done by returning
<span class="code">{ct_hooks,[CTH]}</span> in the config list from
<span class="bold_code"><a href="common_test.html#Module:suite-0">suite/0</a></span>,
<span class="bold_code"><a href="common_test.html#Module:init_per_suite-1">
init_per_suite/1</a></span> or
<span class="bold_code"><a href="common_test.html#Module:init_per_group-2">
init_per_group/2</a></span>. <span class="code">CTH</span> in this case can be either
only the module name of the CTH or a tuple with the module name and the
initial arguments and optionally the hook priority of the CTH. Eg:
<span class="code">{ct_hooks,[my_cth_module]}</span> or
<span class="code">{ct_hooks,[{my_cth_module,[{debug,true}]}]}</span> or
<span class="code">{ct_hooks,[{my_cth_module,[{debug,true}],500}]}</span>
</p>
<h4>Overriding CTHs</h4>
<p>By default each installation of a CTH will cause a new instance of it
to be activated. This can cause problems if you want to be able to
override CTHs in test specifications while still having them in the
suite info function. The
<span class="bold_code"><a href="ct_hooks.html#Module:id-1">id/1</a></span>
callback exists to address this problem. By returning the same
<span class="code">id</span> in both places, Common Test knows that this CTH
has already been installed and will not try to install it again.</p>
<h4>CTH Execution order</h4>
<p>By default each CTH installed will be executed in the order which
they are installed for init calls, and then reversed for end calls.
This is not always wanted so common_test allows
the user to specify a priority for each hook. The priority can either
be specified in the CTH <span class="bold_code"><a href="ct_hooks.html#Module:init-2">init/2
</a></span> function or when installing the hook. The priority given at
installation will override the priority returned by the CTH. </p>
<a name="scope"></a>
<h3><a name="id80835">13.3
CTH Scope</a></h3>
<p>Once the CTH is installed into a certain test run it will be there until
its scope is expired. The scope of a CTH depends on when it is
installed.
The <span class="bold_code"><a href="ct_hooks.html#Module:init-2">init/2</a></span> is
called at the beginning of the scope and the
<span class="bold_code"><a href="ct_hooks.html#Module:terminate-1">terminate/1
</a></span> function is called when the scope ends.</p>
<table border="1" cellpadding="2" cellspacing="0">
<tr>
<td align="left" valign="middle"><strong>CTH Installed in</strong></td>
<td align="left" valign="middle"><strong>CTH scope begins before</strong></td>
<td align="left" valign="middle"><strong>CTH scope ends after</strong></td>
</tr>
<tr>
<td align="left" valign="middle"><span class="bold_code"><a href="run_test_chapter.html#ct_run">ct_run</a></span></td>
<td align="left" valign="middle">the first test suite is to be run.</td>
<td align="left" valign="middle">the last test suite has been run.</td>
</tr>
<tr>
<td align="left" valign="middle"><span class="bold_code"><a href="ct.html#run_test-1">ct:run_test</a></span></td>
<td align="left" valign="middle">the first test suite is to be run.</td>
<td align="left" valign="middle">the last test suite has been run.</td>
</tr>
<tr>
<td align="left" valign="middle"><span class="bold_code"><a href="run_test_chapter.html#test_specifications">
Test Specification</a></span></td>
<td align="left" valign="middle">the first test suite is to be run.</td>
<td align="left" valign="middle">the last test suite has been run.</td>
</tr>
<tr>
<td align="left" valign="middle"><span class="bold_code"><a href="common_test.html#Module:suite-0">suite/0
</a></span></td>
<td align="left" valign="middle">
<span class="bold_code"><a href="ct_hooks.html#Module:pre_init_per_suite-3">
pre_init_per_suite/3</a></span> is called.</td>
<td align="left" valign="middle">
<span class="bold_code"><a href="ct_hooks.html#Module:post_end_per_suite-4">
post_end_per_suite/4</a></span> has been called for that test suite.</td>
</tr>
<tr>
<td align="left" valign="middle"><span class="bold_code"><a href="common_test.html#Module:init_per_suite-1">
init_per_suite/1</a></span></td>
<td align="left" valign="middle">
<span class="bold_code"><a href="ct_hooks.html#Module:post_init_per_suite-4">
post_init_per_suite/4</a></span> is called.</td>
<td align="left" valign="middle">
<span class="bold_code"><a href="ct_hooks.html#Module:post_end_per_suite-4">
post_end_per_suite/4</a></span> has been called for that test suite.</td>
</tr>
<tr>
<td align="left" valign="middle"><span class="bold_code"><a href="common_test.html#Module:init_per_group-2">
init_per_group/2</a></span></td>
<td align="left" valign="middle">
<span class="bold_code"><a href="ct_hooks.html#Module:post_init_per_group-4">
post_init_per_group/4</a></span> is called.</td>
<td align="left" valign="middle">
<span class="bold_code"><a href="ct_hooks.html#Module:post_end_per_suite-4">
post_end_per_group/4</a></span> has been called for that group.</td>
</tr>
</table>
<em>Table
13.1:
Scope of a CTH</em>
<h4>CTH Processes and Tables</h4>
<p>CTHs are run with the same process scoping as normal test suites
i.e. a different process will execute the init_per_suite hooks then the
init_per_group or per_testcase hooks. So if you want to spawn a
process in the CTH you cannot link with the CTH process as it will exit
after the post hook ends. Also if you for some reason need an ETS
table with your CTH, you will have to spawn a process which handles
it.</p>
<h4>External configuration data and Logging</h4>
<p>It's possible in the CTH to read configuration data values
by calling <span class="code"><span class="bold_code"><a href="ct.html#get_config-1">ct:get_config/1/2/3</a></span></span> (as explained in the
<span class="bold_code"><a href="config_file_chapter.html#require_config_data">
External configuration data</a></span>
chapter). The config variables in question must, as always, first have been
<span class="code">required</span> by means of a suite-, group-, or test case info function,
or the <span class="code"><span class="bold_code"><a href="ct.html#require-1">ct:require/1/2</a></span></span> function. Note that the latter can also be used
in CT hook functions.</p>
<p>The CT hook functions may call any of the logging functions available
in the <span class="code">ct</span> interface to print information to the log files, or to
add comments in the suite overview page.
</p>
<a name="manipulating"></a>
<h3><a name="id81189">13.4
Manipulating tests</a></h3>
<p>It is through CTHs possible to manipulate the results of tests and
configuration functions. The main purpose of doing this with CTHs is to
allow common patterns to be abstracted out from test test suites and applied to
multiple test suites without duplicating any code. All of the callback
functions for a CTH follow a common interface, this interface is
described below.</p>
<p>Common Test will always call all available hook functions, even pre- and post
hooks for configuration functions that are not implemented in the suite.
For example, <span class="code">pre_init_per_suite(x_SUITE, ...)</span> and
<span class="code">post_init_per_suite(x_SUITE, ...)</span> will be called for test suite
<span class="code">x_SUITE</span>, even if it doesn't export <span class="code">init_per_suite/1</span>. This feature
makes it possible to use hooks as configuration fallbacks, or even
completely replace all configuration functions with hook functions.</p>
<a name="pre"></a>
<h4>Pre Hooks</h4>
<p>
It is possible in a CTH to hook in behaviour before
<span class="bold_code"><a href="common_test.html#Module:init_per_suite-1">init_per_suite</a></span>,
<span class="bold_code"><a href="common_test.html#Module:init_per_suite-1">init_per_group</a></span>,
<span class="bold_code"><a href="common_test.html#Module:init_per_suite-1">init_per_testcase</a></span>,
<span class="bold_code"><a href="common_test.html#Module:init_per_suite-1">end_per_group</a></span> and
<span class="bold_code"><a href="common_test.html#Module:init_per_suite-1">end_per_suite</a></span>.
This is done in the CTH functions called pre_<name of function>.
All of these functions take the same three arguments: <span class="code">Name</span>,
<span class="code">Config</span> and <span class="code">CTHState</span>. The return value of the CTH function
is always a combination of an result for the suite/group/test and an
updated <span class="code">CTHState</span>. If you want the test suite to continue on
executing you should return the config list which you want the test to
use as the result. If you for some reason want to skip/fail the test,
return a tuple with <span class="code">skip</span> or <span class="code">fail</span> and a reason as the
result. Example:
</p>
<div class="example"><pre>pre_init_per_suite(SuiteName, Config, CTHState) ->
case db:connect() of
{error,_Reason} ->
{{fail, "Could not connect to DB"}, CTHState};
{ok, Handle} ->
{[{db_handle, Handle} | Config], CTHState#state{ handle = Handle }}
end.</pre></div>
<div class="note">
<div class="label">Note</div>
<div class="content"><p>If using multiple CTHs, the first part of the return tuple will be
used as input for the next CTH. So in the case above the next CTH might
get <span class="code">{fail,Reason}</span> as the second parameter. If you have many CTHs
which interact, it might be a good idea to not let each CTH return
<span class="code">fail</span> or <span class="code">skip</span>. Instead return that an action should be taken
through the <span class="code">Config</span> list and implement a CTH which at the end takes
the correct action. </p></div>
</div>
<a name="post"></a>
<h4>Post Hooks</h4>
<p>It is also possible in a CTH to hook in behaviour after
<span class="bold_code"><a href="common_test.html#Module:init_per_suite-1">init_per_suite</a></span>,
<span class="bold_code"><a href="common_test.html#Module:init_per_suite-1">init_per_group</a></span>,
<span class="bold_code"><a href="common_test.html#Module:init_per_suite-1">end_per_testcase</a></span>,
<span class="bold_code"><a href="common_test.html#Module:init_per_suite-1">end_per_group</a></span> and
<span class="bold_code"><a href="common_test.html#Module:init_per_suite-1">end_per_suite</a></span>.
This is done in the CTH functions called post_<name of function>.
All of these function take the same four arguments: <span class="code">Name</span>,
<span class="code">Config</span>, <span class="code">Return</span> and <span class="code">CTHState</span>. <span class="code">Config</span> in this
case is the same <span class="code">Config</span> as the testcase is called with.
<span class="code">Return</span> is the value returned by the testcase. If the testcase
failed by crashing, <span class="code">Return</span> will be
<span class="code">{'EXIT',{{Error,Reason},Stacktrace}}</span>.</p>
<p>The return value of the CTH function is always a combination of an
result for the suite/group/test and an updated <span class="code">CTHState</span>. If
you want the callback to not affect the outcome of the test you should
return the <span class="code">Return</span> data as it is given to the CTH. You can also
modify the result of the test. By returning the <span class="code">Config</span> list
with the <span class="code">tc_status</span> element removed you can recover from a test
failure. As in all the pre hooks, it is also possible to fail/skip
the test case in the post hook. Example: </p>
<div class="example"><pre>post_end_per_testcase(_TC, Config, {'EXIT',{_,_}}, CTHState) ->
case db:check_consistency() of
true ->
%% DB is good, pass the test.
{proplists:delete(tc_status, Config), CTHState};
false ->
%% DB is not good, mark as skipped instead of failing
{{skip, "DB is inconsisten!"}, CTHState}
end;
post_end_per_testcase(_TC, Config, Return, CTHState) ->
%% Do nothing if tc does not crash.
{Return, CTHState}.</pre></div>
<div class="note">
<div class="label">Note</div>
<div class="content"><p>Recovering from a testcase failure using CTHs should only be done as
a last resort. If used wrongly it could become very difficult to
determine which tests pass or fail in a test run</p></div>
</div>
<a name="skip_n_fail"></a>
<h4>Skip and Fail hooks</h4>
<p>
After any post hook has been executed for all installed CTHs,
<span class="bold_code"><a href="ct_hooks.html#Module:on_tc_fail-3">on_tc_fail</a></span>
or <span class="bold_code"><a href="ct_hooks.html#Module:on_tc_fail-3">on_tc_skip</a></span>
might be called if the testcase failed or was skipped
respectively. You cannot affect the outcome of the tests any further at
this point.
</p>
<a name="example"></a>
<h3><a name="id81482">13.5
Example CTH</a></h3>
<p>The CTH below will log information about a test run into a format
parseable by <span class="bold_code"><a href="javascript:erlhref('../../../../doc/../','kernel','file.html#consult-1');">file:consult/1</a></span>.
</p>
<div class="example"><pre>%%% @doc Common Test Example Common Test Hook module.
-module(example_cth).
%% Callbacks
-export([id/1]).
-export([init/2]).
-export([pre_init_per_suite/3]).
-export([post_init_per_suite/4]).
-export([pre_end_per_suite/3]).
-export([post_end_per_suite/4]).
-export([pre_init_per_group/3]).
-export([post_init_per_group/4]).
-export([pre_end_per_group/3]).
-export([post_end_per_group/4]).
-export([pre_init_per_testcase/3]).
-export([post_end_per_testcase/4]).
-export([on_tc_fail/3]).
-export([on_tc_skip/3]).
-export([terminate/1]).
-record(state, { file_handle, total, suite_total, ts, tcs, data }).
%% @doc Return a unique id for this CTH.
id(Opts) ->
proplists:get_value(filename, Opts, "/tmp/file.log").
%% @doc Always called before any other callback function. Use this to initiate
%% any common state.
init(Id, Opts) ->
{ok,D} = file:open(Id,[write]),
{ok, #state{ file_handle = D, total = 0, data = [] }}.
%% @doc Called before init_per_suite is called.
pre_init_per_suite(Suite,Config,State) ->
{Config, State#state{ suite_total = 0, tcs = [] }}.
%% @doc Called after init_per_suite.
post_init_per_suite(Suite,Config,Return,State) ->
{Return, State}.
%% @doc Called before end_per_suite.
pre_end_per_suite(Suite,Config,State) ->
{Config, State}.
%% @doc Called after end_per_suite.
post_end_per_suite(Suite,Config,Return,State) ->
Data = {suites, Suite, State#state.suite_total, lists:reverse(State#state.tcs)},
{Return, State#state{ data = [Data | State#state.data] ,
total = State#state.total + State#state.suite_total } }.
%% @doc Called before each init_per_group.
pre_init_per_group(Group,Config,State) ->
{Config, State}.
%% @doc Called after each init_per_group.
post_init_per_group(Group,Config,Return,State) ->
{Return, State}.
%% @doc Called after each end_per_group.
pre_end_per_group(Group,Config,State) ->
{Config, State}.
%% @doc Called after each end_per_group.
post_end_per_group(Group,Config,Return,State) ->
{Return, State}.
%% @doc Called before each test case.
pre_init_per_testcase(TC,Config,State) ->
{Config, State#state{ ts = now(), total = State#state.suite_total + 1 } }.
%% @doc Called after each test case.
post_end_per_testcase(TC,Config,Return,State) ->
TCInfo = {testcase, TC, Return, timer:now_diff(now(), State#state.ts)},
{Return, State#state{ ts = undefined, tcs = [TCInfo | State#state.tcs] } }.
%% @doc Called after post_init_per_suite, post_end_per_suite, post_init_per_group,
%% post_end_per_group and post_end_per_testcase if the suite, group or test case failed.
on_tc_fail(TC, Reason, State) ->
State.
%% @doc Called when a test case is skipped by either user action
%% or due to an init function failing.
on_tc_skip(TC, Reason, State) ->
State.
%% @doc Called when the scope of the CTH is done
terminate(State) ->
io:format(State#state.file_handle, "~p.~n",
[{test_run, State#state.total, State#state.data}]),
file:close(State#state.file_handle),
ok.</pre></div>
<a name="builtin_cths"></a>
<h3><a name="id81535">13.6
Built-in CTHs</a></h3>
<p>Common Test is delivered with a couple of general purpose CTHs that
can be enabled by the user to provide some generic testing functionality.
Some of these are enabled by default when starting running common_test,
they can be disabled by setting <span class="code">enable_builtin_hooks</span> to
<span class="code">false</span> on the command line or in the test specification. In the
table below there is a list of all current CTHs which are delivered with
Common Test.</p>
<table border="1" cellpadding="2" cellspacing="0">
<tr>
<td align="left" valign="middle"><strong>CTH Name</strong></td>
<td align="left" valign="middle"><strong>Is Built-in</strong></td>
<td align="left" valign="middle"><strong>Description</strong></td>
</tr>
<tr>
<td align="left" valign="middle">cth_log_redirect</td>
<td align="left" valign="middle">yes</td>
<td align="left" valign="middle">Captures all error_logger and SASL logging events and prints them
to the current test case log. If an event can not be associated with a
testcase it will be printed in the common test framework log. This will
happen for testcases which are run in parallel and events which occur
inbetween testcases. You can configure the level of
<span class="bold_code"><a href="javascript:erlhref('../../../../doc/../','sasl','sasl_app.html');">SASL</a></span> events report
using the normal SASL mechanisms. </td>
</tr>
<tr>
<td align="left" valign="middle">cth_surefire</td>
<td align="left" valign="middle">no</td>
<td align="left" valign="middle">Captures all test results and outputs them as surefire XML into
a file. The file which is created is by default called junit_report.xml.
The name can be by setting the path option for this hook. e.g.
<div class="example"><pre>-ct_hooks cth_surefire [{path,"/tmp/report.xml"}]</pre></div>
Surefire XML can forinstance be used by Jenkins to display test
results.</td>
</tr>
</table>
</div>
<div class="footer">
<hr>
<p>Copyright © 2003-2012 Ericsson AB. All Rights Reserved.</p>
</div>
</div>
</div></body>
</html>
distrib
>
Fedora
>
17
>
i386
>
media
>
updates
>
by-pkgid
>
675c8c8167236dfcf8d66da674f931e8
>
files
>
239
