<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html xmlns:fn="http://www.w3.org/2005/02/xpath-functions">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<link rel="stylesheet" href="../../../../doc/otp_doc.css" type="text/css">
<title>Erlang -- Test Server Basics</title>
</head>
<body bgcolor="white" text="#000000" link="#0000ff" vlink="#ff00ff" alink="#ff0000"><div id="container">
<script id="js" type="text/javascript" language="JavaScript" src="../../../../doc/js/flipmenu/flipmenu.js"></script><script id="js2" type="text/javascript" src="../../../../doc/js/erlresolvelinks.js"></script><script language="JavaScript" type="text/javascript">
<!--
function getWinHeight() {
var myHeight = 0;
if( typeof( window.innerHeight ) == 'number' ) {
//Non-IE
myHeight = window.innerHeight;
} else if( document.documentElement && ( document.documentElement.clientWidth ||
document.documentElement.clientHeight ) ) {
//IE 6+ in 'standards compliant mode'
myHeight = document.documentElement.clientHeight;
} else if( document.body && ( document.body.clientWidth || document.body.clientHeight ) ) {
//IE 4 compatible
myHeight = document.body.clientHeight;
}
return myHeight;
}
function setscrollpos() {
var objf=document.getElementById('loadscrollpos');
document.getElementById("leftnav").scrollTop = objf.offsetTop - getWinHeight()/2;
}
function addEvent(obj, evType, fn){
if (obj.addEventListener){
obj.addEventListener(evType, fn, true);
return true;
} else if (obj.attachEvent){
var r = obj.attachEvent("on"+evType, fn);
return r;
} else {
return false;
}
}
addEvent(window, 'load', setscrollpos);
//--></script><div id="leftnav"><div class="innertube">
<img alt="Erlang logo" src="../../../../doc/erlang-logo.png"><br><small><a href="users_guide.html">User's Guide</a><br><a href="index.html">Reference Manual</a><br><a href="release_notes.html">Release Notes</a><br><a href="../pdf/test_server-3.5.3.pdf">PDF</a><br><a href="../../../../doc/index.html">Top</a></small><p><strong>Test Server</strong><br><strong>User's Guide</strong><br><small>Version 3.5.3</small></p>
<br><a href="javascript:openAllFlips()">Expand All</a><br><a href="javascript:closeAllFlips()">Contract All</a><p><small><strong>Chapters</strong></small></p>
<ul class="flipMenu" imagepath="../../../../doc/js/flipmenu">
<li id="loadscrollpos" title="Test Server Basics" expanded="true">Test Server Basics<ul>
<li><a href="basics_chapter.html">
Top of chapter
</a></li>
<li title="Introduction"><a href="basics_chapter.html#id56984">Introduction</a></li>
<li title="Getting started"><a href="basics_chapter.html#id56907">Getting started</a></li>
<li title="Definition of terms"><a href="basics_chapter.html#id61500">Definition of terms</a></li>
</ul>
</li>
<li id="no" title="Test Structure and Test Specifications" expanded="false">Test Structure and Test Specifications<ul>
<li><a href="test_spec_chapter.html">
Top of chapter
</a></li>
<li title="Test structure"><a href="test_spec_chapter.html#id62057">Test structure</a></li>
<li title="Test specifications"><a href="test_spec_chapter.html#id57989">Test specifications</a></li>
<li title="Test Specification Files"><a href="test_spec_chapter.html#id63174">Test Specification Files</a></li>
<li title="Configuration cases"><a href="test_spec_chapter.html#id60965">Configuration cases</a></li>
<li title="The parallel property and nested configuration cases"><a href="test_spec_chapter.html#id56819">The parallel property and nested configuration cases</a></li>
<li title="Repeated execution of test cases"><a href="test_spec_chapter.html#id56843">Repeated execution of test cases</a></li>
<li title="Shuffled test case order"><a href="test_spec_chapter.html#id59974">Shuffled test case order</a></li>
<li title="Skipping test cases"><a href="test_spec_chapter.html#id60025">Skipping test cases</a></li>
</ul>
</li>
<li id="no" title="Writing Test Suites" expanded="false">Writing Test Suites<ul>
<li><a href="write_test_chapter.html">
Top of chapter
</a></li>
<li title="Support for test suite authors"><a href="write_test_chapter.html#id62420">Support for test suite authors</a></li>
<li title="Test suites"><a href="write_test_chapter.html#id62477">Test suites</a></li>
<li title="Init per test case"><a href="write_test_chapter.html#id62508">Init per test case</a></li>
<li title="Test cases"><a href="write_test_chapter.html#id62581">Test cases</a></li>
<li title="Data and Private Directories"><a href="write_test_chapter.html#id62718">Data and Private Directories</a></li>
<li title="Execution environment"><a href="write_test_chapter.html#id62776">Execution environment</a></li>
</ul>
</li>
<li id="no" title="Running Test Suites" expanded="false">Running Test Suites<ul>
<li><a href="run_test_chapter.html">
Top of chapter
</a></li>
<li title="Using the test server controller"><a href="run_test_chapter.html#id62898">Using the test server controller</a></li>
</ul>
</li>
<li id="no" title="Write you own test server framework" expanded="false">Write you own test server framework<ul>
<li><a href="write_framework_chapter.html">
Top of chapter
</a></li>
<li title="Introduction"><a href="write_framework_chapter.html#id62979">Introduction</a></li>
<li title="Interfacing the test server controller from Erlang"><a href="write_framework_chapter.html#id62997">Interfacing the test server controller from Erlang</a></li>
<li title="Interfacing the test server controller from the operating system."><a href="write_framework_chapter.html#id63107">Interfacing the test server controller from the operating system.</a></li>
<li title="Framework callback functions"><a href="write_framework_chapter.html#id63412">Framework callback functions</a></li>
<li title="Other concerns"><a href="write_framework_chapter.html#id63442">Other concerns</a></li>
</ul>
</li>
<li id="no" title="Examples" expanded="false">Examples<ul>
<li><a href="example_chapter.html">
Top of chapter
</a></li>
<li title="Test suite"><a href="example_chapter.html#id63550">Test suite</a></li>
<li title="Test specification file"><a href="example_chapter.html#id63589">Test specification file</a></li>
</ul>
</li>
</ul>
</div></div>
<div id="content">
<div class="innertube">
<h1>1 Test Server Basics</h1>
<h3><a name="id56984">1.1
Introduction</a></h3>
<p><strong>Test Server</strong> is a portable test tool for automated
testing of Erlang programs and OTP applications. It provides an
interface for running test programs directly with Test Server
as well as an interface for integrating Test Server
with a framework application. The latter makes it possible to use
Test Server as the engine of a higher level test tool
application.</p>
<p>It is strongly recommended that Test Server be used from inside
a framework application, rather than interfaced directly for
running test programs. Test Server can be pretty difficult to use
since it's a very general and quite extensive and complex
application. Furthermore, the <span class="code">test_server_ctrl</span> functions
are not meant to be used from within the actual test programs. The
framework should handle communication with Test Server and deal
with the more complex aspects of this interaction automatically so
that a higher level interface may be provided for the tester. For
test tool usage to be productive, a simpler, more intuitive and
(if required) more specific interface is required than what Test Server
can provide.</p>
<p>OTP delivers a general purpose framework for Test Server, called
<strong>Common Test</strong>. This application is a tool well suited for
automated black box testing of target systems of <strong>any kind</strong>
(not necessarily implemented in Erlang). Common Test is also a very
useful tool for white box testing of Erlang programs and OTP
applications. Unless a more specific functionality and/or user
interface is required (in which case you might need to implement
your own framework), Common Test should do the job for
you. Please read the Common Test User's Guide and reference manual
for more information.</p>
<p>Under normal circumstances, knowledge about the Test Server
application is not required for using the Common Test framework.
However, if you want to use Test Server without a framework,
or learn how to integrate it with your own framework, please read on...
</p>
<h3><a name="id56907">1.2
Getting started</a></h3>
<p>Testing when using Test Server is done by running test
suites. A test suite is a number of test cases, where each test
case tests one or more things. The test case is the smallest unit
that the test server deals with. One or more test cases are
grouped together into one ordinary Erlang module, which is called
a test suite. Several test suite modules can be grouped together
in special test specification files representing whole application
and/or system test "jobs".
</p>
<p>The test suite Erlang module must follow a certain interface,
which is specified by Test Server. See the section on writing
test suites for details about this.
</p>
<p>Each test case is considered a success if it returns to the
caller, no matter what the returned value is. An exception to this
is the return value <span class="code">{skip, Reason}</span> which indicates that the
test case is skipped. A failure is specified as a crash, no matter
what the crash reason is.
</p>
<p>As a test suite runs, all information (including output to
stdout) is recorded in several different log files. A minimum of
information is displayed to the user console. This only include
start and stop information, plus a note for each failed test case.
</p>
<p>The result from each test case is recorded in an HTML log file
which is created for each test run. Every test case gets one row
in a table presenting total time, whether the case was successful
or not, if it was skipped, and possibly also a comment. The HTML
file has links to each test case's logfile, which may be viewed
from e.g. Netscape or any other HTML capable browser.
</p>
<p>The Test Server consists of three parts:
</p>
<ul>
<li>The part that executes the test suites on target and
provides support for the test suite author is called
<span class="code">test_server</span>. This is described in the chapter about
writing test cases in this user's guide, and in the reference
manual for the <span class="code">test_server</span> module.</li>
<li>The controlling part, which provides the low level
operator interface, starts and stops the target node (if remote
target) and slave nodes and writes log files, is called
<span class="code">test_server_ctrl</span>. The Test Server Controller should not
be used directly when running tests. Instead a framework built
on top of it should be used. More information
about how to write your own framework can be found
in this user's guide and in the reference manual for the
<span class="code">test_server_ctrl</span> module.</li>
</ul>
<h3><a name="id61500">1.3
Definition of terms</a></h3>
<dl>
<dt><strong><strong>conf(iguration) case</strong></strong></dt>
<dd>This is a group of test cases which need some specific
configuration. A conf case contains an initiation function which
sets up a specific configuration, one or more test cases using
this configuration, and a cleanup function which restores the
configuration. A conf case is specified in a test specification
either like this:<span class="code">{conf,InitFunc,ListOfCases,CleanupFunc}</span>,
or this: <span class="code">{conf,Properties,InitFunc,ListOfCases,CleanupFunc}</span>
</dd>
<dt><strong><strong>datadir</strong></strong></dt>
<dd>Data directory for a test suite. This directory contains
any files used by the test suite, e.g. additional erlang
modules, c code or data files. If the data directory contains
code which must be compiled before the test suite is run, it
should also contain a makefile source called Makefile.src
defining how to compile.
</dd>
<dt><strong><strong>documentation clause</strong></strong></dt>
<dd>One of the function clauses in a test case. This clause
shall return a list of strings describing what the test case
tests.
</dd>
<dt><strong><strong>execution clause</strong></strong></dt>
<dd>One of the function clauses in a test case. This clause
implements the actual test case, i.e. calls the functions that
shall be tested and checks results. The clause shall crash if it
fails.
</dd>
<dt><strong><strong>major log file</strong></strong></dt>
<dd>This is the test suites log file.
</dd>
<dt><strong><strong>Makefile.src</strong></strong></dt>
<dd>This file is used by the test server framework to generate
a makefile for a datadir. It contains some special characters
which are replaced according to the platform currently tested.
</dd>
<dt><strong><strong>minor log file</strong></strong></dt>
<dd>This is a separate log file for each test case.
</dd>
<dt><strong><strong>privdir</strong></strong></dt>
<dd>Private directory for a test suite. This directory should
be used when the test suite needs to write to files.
</dd>
<dt><strong><strong>skip case</strong></strong></dt>
<dd>A test case which shall be skipped.
</dd>
<dt><strong><strong>specification clause</strong></strong></dt>
<dd>One of the function clauses in a test case. This clause
shall return an empty list, a test specification or
<span class="code">{skip,Reason}</span>. If an empty list is returned, it means
that the test case shall be executed, and so it must also have
an execution clause. Note that the specification clause is
always executed on the controller node, i.e. not on the target
node.
</dd>
<dt><strong><strong>test case</strong></strong></dt>
<dd>A single test included in a test suite. Typically it tests
one function in a module or application. A test case is
implemented as a function in a test suite module. The function
can have three clauses, the documentation-, specification- and
execution clause.
</dd>
<dt><strong><strong>test specification</strong></strong></dt>
<dd>A specification of which test suites and test cases to
run. There can be test specifications on three different levels
in a test. The top level is a test specification file which
roughly specifies what to test for a whole application. Then
there is a test specification for each test suite returned from
the <span class="code">all(suite)</span> function in the suite. And there can also
be a test specification returned from the specification clause
of a test case.
</dd>
<dt><strong><strong>test specification file</strong></strong></dt>
<dd>This is a text file containing the test specification for
an application. The file has the extension ".spec" or
".spec.Platform", where Platform is e.g. "vxworks".
</dd>
<dt><strong><strong>test suite</strong></strong></dt>
<dd>An erlang module containing a collection of test cases for
a specific application or module.
</dd>
<dt><strong><strong>topcase</strong></strong></dt>
<dd>The first "command" in a test specification file. This
command contains the test specification, like this:
<span class="code">{topcase,TestSpecification}</span>
</dd>
</dl>
</div>
<div class="footer">
<hr>
<p>Copyright © 2002-2012 Ericsson AB. All Rights Reserved.</p>
</div>
</div>
</div></body>
</html>
distrib
>
Fedora
>
17
>
i386
>
media
>
updates
>
by-pkgid
>
675c8c8167236dfcf8d66da674f931e8
>
files
>
1473
