<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> <html xmlns:fn="http://www.w3.org/2005/02/xpath-functions"> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> <link rel="stylesheet" href="../../../../doc/otp_doc.css" type="text/css"> <title>Erlang -- Test Structure</title> </head> <body bgcolor="white" text="#000000" link="#0000ff" vlink="#ff00ff" alink="#ff0000"><div id="container"> <script id="js" type="text/javascript" language="JavaScript" src="../../../../doc/js/flipmenu/flipmenu.js"></script><script id="js2" type="text/javascript" src="../../../../doc/js/erlresolvelinks.js"></script><script language="JavaScript" type="text/javascript"> <!-- function getWinHeight() { var myHeight = 0; if( typeof( window.innerHeight ) == 'number' ) { //Non-IE myHeight = window.innerHeight; } else if( document.documentElement && ( document.documentElement.clientWidth || document.documentElement.clientHeight ) ) { //IE 6+ in 'standards compliant mode' myHeight = document.documentElement.clientHeight; } else if( document.body && ( document.body.clientWidth || document.body.clientHeight ) ) { //IE 4 compatible myHeight = document.body.clientHeight; } return myHeight; } function setscrollpos() { var objf=document.getElementById('loadscrollpos'); document.getElementById("leftnav").scrollTop = objf.offsetTop - getWinHeight()/2; } function addEvent(obj, evType, fn){ if (obj.addEventListener){ obj.addEventListener(evType, fn, true); return true; } else if (obj.attachEvent){ var r = obj.attachEvent("on"+evType, fn); return r; } else { return false; } } addEvent(window, 'load', setscrollpos); //--></script><div id="leftnav"><div class="innertube"> <img alt="Erlang logo" src="../../../../doc/erlang-logo.png"><br><small><a href="users_guide.html">User's Guide</a><br><a href="index.html">Reference Manual</a><br><a href="release_notes.html">Release Notes</a><br><a href="../pdf/common_test-1.4.7.pdf">PDF</a><br><a href="../../../../doc/index.html">Top</a></small><p><strong>Common Test</strong><br><strong>User's Guide</strong><br><small>Version 1.4.7</small></p> <br><a href="javascript:openAllFlips()">Expand All</a><br><a href="javascript:closeAllFlips()">Contract All</a><p><small><strong>Chapters</strong></small></p> <ul class="flipMenu" imagepath="../../../../doc/js/flipmenu"> <li id="no" title="Common Test Basics" expanded="false">Common Test Basics<ul> <li><a href="basics_chapter.html"> Top of chapter </a></li> <li title="Introduction"><a href="basics_chapter.html#id2257382">Introduction</a></li> <li title="Test Suite Organisation"><a href="basics_chapter.html#id2262699">Test Suite Organisation</a></li> <li title="Support Libraries"><a href="basics_chapter.html#id2262725">Support Libraries</a></li> <li title="Suites and Test Cases"><a href="basics_chapter.html#id2252225">Suites and Test Cases</a></li> <li title="External Interfaces"><a href="basics_chapter.html#id2252025">External Interfaces</a></li> </ul> </li> <li id="no" title="Installation" expanded="false">Installation<ul> <li><a href="install_chapter.html"> Top of chapter </a></li> <li title="General information"><a href="install_chapter.html#id2259791">General information</a></li> <li title="Unix/Linux"><a href="install_chapter.html#id2259969">Unix/Linux</a></li> <li title="Windows"><a href="install_chapter.html#id2252127">Windows</a></li> </ul> </li> <li id="no" title="Writing Test Suites" expanded="false">Writing Test Suites<ul> <li><a href="write_test_chapter.html"> Top of chapter </a></li> <li title="Support for test suite authors"><a href="write_test_chapter.html#id2263189">Support for test suite authors</a></li> <li title="Test suites"><a href="write_test_chapter.html#id2263446">Test suites</a></li> <li title="Init and end per suite"><a href="write_test_chapter.html#id2252033">Init and end per suite</a></li> <li title="Init and end per test case"><a href="write_test_chapter.html#id2263397">Init and end per test case</a></li> <li title="Test cases"><a href="write_test_chapter.html#id2252705">Test cases</a></li> <li title="Test case info function"><a href="write_test_chapter.html#id2252952">Test case info function</a></li> <li title="Test suite info function"><a href="write_test_chapter.html#id2264301">Test suite info function</a></li> <li title="Test case groups"><a href="write_test_chapter.html#id2264393">Test case groups</a></li> <li title="The parallel property and nested groups"><a href="write_test_chapter.html#id2264666">The parallel property and nested groups</a></li> <li title="Repeated groups"><a href="write_test_chapter.html#id2264711">Repeated groups</a></li> <li title="Shuffled test case order"><a href="write_test_chapter.html#id2264872">Shuffled test case order</a></li> <li title="Data and Private Directories"><a href="write_test_chapter.html#id2264941">Data and Private Directories</a></li> <li title="Execution environment"><a href="write_test_chapter.html#id2265020">Execution environment</a></li> <li title="Illegal dependencies"><a href="write_test_chapter.html#id2265070">Illegal dependencies</a></li> </ul> </li> <li id="loadscrollpos" title="Test Structure" expanded="true">Test Structure<ul> <li><a href="test_structure_chapter.html"> Top of chapter </a></li> <li title="Test structure"><a href="test_structure_chapter.html#id2265289">Test structure</a></li> <li title="Skipping test cases"><a href="test_structure_chapter.html#id2265306">Skipping test cases</a></li> <li title="Definition of terms"><a href="test_structure_chapter.html#id2265390">Definition of terms</a></li> </ul> </li> <li id="no" title="Examples and Templates" expanded="false">Examples and Templates<ul> <li><a href="example_chapter.html"> Top of chapter </a></li> <li title="Test suite example"><a href="example_chapter.html#id2266177">Test suite example</a></li> <li title="Test suite templates"><a href="example_chapter.html#id2266264">Test suite templates</a></li> </ul> </li> <li id="no" title="Running Test Suites" expanded="false">Running Test Suites<ul> <li><a href="run_test_chapter.html"> Top of chapter </a></li> <li title="Using the Common Test Framework"><a href="run_test_chapter.html#id2266562">Using the Common Test Framework</a></li> <li title="Automatic compilation of test suites and help modules"><a href="run_test_chapter.html#id2266607">Automatic compilation of test suites and help modules</a></li> <li title="Running tests from the UNIX command line"><a href="run_test_chapter.html#id2266741">Running tests from the UNIX command line</a></li> <li title="Running tests from the Web based GUI"><a href="run_test_chapter.html#id2267056">Running tests from the Web based GUI</a></li> <li title="Running tests from the Erlang shell or from an Erlang program"><a href="run_test_chapter.html#id2267143">Running tests from the Erlang shell or from an Erlang program</a></li> <li title="Running the interactive shell mode"><a href="run_test_chapter.html#id2267199">Running the interactive shell mode</a></li> <li title="Step by step execution of test cases with the Erlang Debugger"><a href="run_test_chapter.html#id2267375">Step by step execution of test cases with the Erlang Debugger</a></li> <li title="Using test specifications"><a href="run_test_chapter.html#id2267476">Using test specifications</a></li> <li title="Log files"><a href="run_test_chapter.html#id2267715">Log files</a></li> <li title="HTML Style Sheets"><a href="run_test_chapter.html#id2267921">HTML Style Sheets</a></li> <li title="Repeating tests"><a href="run_test_chapter.html#id2268107">Repeating tests</a></li> <li title="Silent Connections"><a href="run_test_chapter.html#id2268354">Silent Connections</a></li> </ul> </li> <li id="no" title="Config Files" expanded="false">Config Files<ul> <li><a href="config_file_chapter.html"> Top of chapter </a></li> <li title="General"><a href="config_file_chapter.html#id2268593">General</a></li> <li title="Syntax"><a href="config_file_chapter.html#id2268633">Syntax</a></li> <li title="Requiring and reading configuration data"><a href="config_file_chapter.html#id2268657">Requiring and reading configuration data</a></li> <li title="Using configuration variables defined in multiple files"><a href="config_file_chapter.html#id2268807">Using configuration variables defined in multiple files</a></li> <li title="Encrypted configuration files"><a href="config_file_chapter.html#id2268839">Encrypted configuration files</a></li> <li title="Opening connections by using configuration data"><a href="config_file_chapter.html#id2268904">Opening connections by using configuration data</a></li> <li title="Examples"><a href="config_file_chapter.html#id2268977">Examples</a></li> </ul> </li> <li id="no" title="Code Coverage Analysis" expanded="false">Code Coverage Analysis<ul> <li><a href="cover_chapter.html"> Top of chapter </a></li> <li title="General"><a href="cover_chapter.html#id2269096">General</a></li> <li title="Usage"><a href="cover_chapter.html#id2269123">Usage</a></li> <li title="The cover specification file"><a href="cover_chapter.html#id2269250">The cover specification file</a></li> <li title="Logging"><a href="cover_chapter.html#id2269317">Logging</a></li> </ul> </li> <li id="no" title="Using Common Test for Large Scale Testing" expanded="false">Using Common Test for Large Scale Testing<ul> <li><a href="ct_master_chapter.html"> Top of chapter </a></li> <li title="General"><a href="ct_master_chapter.html#id2269392">General</a></li> <li title="Usage"><a href="ct_master_chapter.html#id2269436">Usage</a></li> <li title="Test Specifications"><a href="ct_master_chapter.html#id2269594">Test Specifications</a></li> </ul> </li> <li id="no" title="Event Handling" expanded="false">Event Handling<ul> <li><a href="event_handler_chapter.html"> Top of chapter </a></li> <li title="General"><a href="event_handler_chapter.html#id2269865">General</a></li> <li title="Usage"><a href="event_handler_chapter.html#id2269911">Usage</a></li> </ul> </li> <li id="no" title="Dependencies between Test Cases and Suites" expanded="false">Dependencies between Test Cases and Suites<ul> <li><a href="dependencies_chapter.html"> Top of chapter </a></li> <li title="General"><a href="dependencies_chapter.html#id2270845">General</a></li> <li title="Saving configuration data"><a href="dependencies_chapter.html#id2271007">Saving configuration data</a></li> <li title="Sequences"><a href="dependencies_chapter.html#id2271201">Sequences</a></li> </ul> </li> <li id="no" title="Some thoughts about testing" expanded="false">Some thoughts about testing<ul> <li><a href="why_test_chapter.html"> Top of chapter </a></li> <li title="Goals"><a href="why_test_chapter.html#id2271413">Goals</a></li> <li title="What to test?"><a href="why_test_chapter.html#id2271438">What to test?</a></li> </ul> </li> </ul> </div></div> <div id="content"> <div class="innertube"> <h1>4 Test Structure</h1> <h3><a name="id2265289">4.1 Test structure</a></h3> <p>A test is performed by running one or more test suites. A test suite consists of test cases (as well as configuration functions and info functions). Test cases may be grouped in so called test case groups. A test suite is an Erlang module and test cases are implemented as Erlang functions. Test suites are stored in test directories.</p> <h3><a name="id2265306">4.2 Skipping test cases</a></h3> <p>It is possible to skip certain test cases, for example if you know beforehand that a specific test case fails. This might be functionality which isn't yet implemented, a bug that is known but not yet fixed or some functionality which doesn't work or isn't applicable on a specific platform.</p> <p>There are several different ways to state that one or more test cases should be skipped:</p> <ul> <li>Using <span class="code">skip_suites</span> and <span class="code">skip_cases</span> terms in <span class="bold_code"><a href="run_test_chapter.html#test_specifications">test specifications</a></span>. </li> <li>Returning <span class="code">{skip,Reason}</span> from the <span class="code">init_per_testcase/2</span> or <span class="code">init_per_suite/1</span> functions.</li> <li>Returning <span class="code">{skip,Reason}</span> from the execution clause of the test case.</li> </ul> <p>The latter of course means that the execution clause is actually called, so the author must make sure that the test case does not run.</p> <p>When a test case is skipped, it will be noted as <span class="code">SKIPPED</span> in the HTML log.</p> <h3><a name="id2265390">4.3 Definition of terms</a></h3> <dl> <dt><strong><strong>Auto skipped test case</strong></strong></dt> <dd> When a configuration function fails (i.e. terminates unexpectedly), the test cases that depend on the configuration function will be skipped automatically by Common Test. The status of the test cases is then "auto skipped". Test cases are also auto skipped by Common Test if required configuration data is not available at runtime. </dd> <dt><strong><strong>Configuration function</strong></strong></dt> <dd> A function in a test suite that is meant to be used for setting up, cleaning up, and/or verifying the state and environment on the SUT (System Under Test) and/or the Common Test host node, so that a test case (or a set of test cases) can execute correctly. </dd> <dt><strong><strong>Configuration file</strong></strong></dt> <dd> A file that contains data related to a test and/or an SUT (System Under Test), e.g. protocol server addresses, client login details, hardware interface addresses, etc - any data that should be handled as variable in the suite and not be hardcoded. </dd> <dt><strong><strong>Configuration variable</strong></strong></dt> <dd> A name (an Erlang atom) associated with a data value read from a configuration file. </dd> <dt><strong><strong>data_dir</strong></strong></dt> <dd> Data directory for a test suite. This directory contains any files used by the test suite, e.g. additional Erlang modules, binaries or data files. </dd> <dt><strong><strong>Info function</strong></strong></dt> <dd> A function in a test suite that returns a list of properties (read by the Common Test server) that describes the conditions for executing the test cases in the suite. </dd> <dt><strong><strong>Major log file</strong></strong></dt> <dd> An overview and summary log file for one or more test suites. </dd> <dt><strong><strong>Minor log file</strong></strong></dt> <dd> A log file for one particular test case. Also called the test case log file. </dd> <dt><strong><strong>priv_dir</strong></strong></dt> <dd> Private directory for a test suite. This directory should be used when the test suite needs to write to files. </dd> <dt><strong><strong>run_test</strong></strong></dt> <dd> The name of an executable Bourne shell script that may be used on Linux/Unix as an interface for specifying and running tests with Common Test. </dd> <dt><strong><strong>Test case</strong></strong></dt> <dd> A single test included in a test suite. A test case is implemented as a function in a test suite module. </dd> <dt><strong><strong>Test case group</strong></strong></dt> <dd> A set of test cases that share configuration functions and execution properties. The execution properties specify whether the test cases in the group should be executed in random order, in parallel, in sequence, and if the execution of the group should be repeated. Test case groups may also be nested (i.e. a group may, besides test cases, contain sub-groups). </dd> <dt><strong><strong>Test suite</strong></strong></dt> <dd> An erlang module containing a collection of test cases for a specific functional area. </dd> <dt><strong><strong>Test directory</strong></strong></dt> <dd> A directory that contains one or more test suite modules, i.e. a group of test suites. </dd> <dt><strong><strong>The Config argument</strong></strong></dt> <dd> A list of key-value tuples (i.e. a property list) containing runtime configuration data passed from the configuration functions to the test cases. </dd> <dt><strong><strong>User skipped test case</strong></strong></dt> <dd> This is the status of a test case that has been explicitly skipped in any of the ways described in the "Skipping test cases" section above. </dd> </dl> </div> <div class="footer"> <hr> <p>Copyright © 2003-2010 Ericsson AB. All Rights Reserved.</p> </div> </div> </div></body> </html>