<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> <html> <head> <link rel="stylesheet" href="style.css" type="text/css"> <meta content="text/html; charset=iso-8859-1" http-equiv="Content-Type"> <link rel="Start" href="index.html"> <link title="Index of types" rel=Appendix href="index_types.html"> <link title="Index of values" rel=Appendix href="index_values.html"> <link title="Index of modules" rel=Appendix href="index_modules.html"> <link title="Index of module types" rel=Appendix href="index_module_types.html"> <link title="OUnit" rel="Chapter" href="OUnit.html"> <link title="OUnitDiff" rel="Chapter" href="OUnitDiff.html"><title>OUnit user guide</title> </head> <body> <h1>OUnit user guide</h1> <ul class="indexlist"> <li><a href="index_types.html">Index of types</a></li> <li><a href="index_values.html">Index of values</a></li> <li><a href="index_modules.html">Index of modules</a></li> <li><a href="index_module_types.html">Index of module types</a></li> </ul> <p> <h2 id="2_WhatisunitTesting">What is unit Testing?</h2> <p> A test-oriented methodology for software development is most effective whent tests are easy to create, change, and execute. The JUnit tool pioneerded for test-first development in Java. OUnit is an adaptation of JUnit to OCaml. <p> With OUnit, as with JUnit, you can easily create tests, name them, group them into suites, and execute them, with the framework checking the results automatically. <p> <h2 id="2_GettingStarted">Getting Started</h2> <p> The basic principle of a test suite is to have a file <i>test.ml</i> which will contain the tests, and an OCaml module under test, named <i>foo.ml</i>. <p> File <i>foo.ml</i>: <pre class="codepre"><code class="code"><span class="comment">(* The functions we wish to test *)</span><br> <span class="keyword">let</span> unity x = x;;<br> <span class="keyword">let</span> funix ()= 0;;<br> <span class="keyword">let</span> fgeneric () = failwith <span class="string">"Not implemented"</span>;;<br> </code></pre> <p> The main point of a test is to check that the function under test has the expected behavior. You check the behavior using assert functions. The most simple one is <a href="OUnit.html#VALassert_equal"><code class="code"><span class="constructor">OUnit</span>.assert_equal</code></a>. This function compares the result of the function with an expected result. <p> The most useful functions are:<ul> <li><a href="OUnit.html#VALassert_equal"><code class="code"><span class="constructor">OUnit</span>.assert_equal</code></a> the basic assert function</li> <li><a href="OUnit.html#VAL(>:::)"><code class="code"><span class="constructor">OUnit</span>.(>:::)</code></a> to define a list of tests </li> <li><a href="OUnit.html#VAL(>::)"><code class="code"><span class="constructor">OUnit</span>.(>::)</code></a> to name a test</li> <li><a href="OUnit.html#VALrun_test_tt_main"><code class="code"><span class="constructor">OUnit</span>.run_test_tt_main</code></a> to run the test suite you define</li> <li><a href="OUnit.html#VALbracket"><code class="code"><span class="constructor">OUnit</span>.bracket</code></a> that can help to set and clean environment, it is especially useful if you deal with temporary filename.</li> </ul> File <i>test.ml</i>: <pre class="codepre"><code class="code"><span class="keyword">open</span> <span class="constructor">OUnit</span>;;<br> <br> <span class="keyword">let</span> test1 () = assert_equal <span class="string">"x"</span> (<span class="constructor">Foo</span>.unity <span class="string">"x"</span>);; <br> <br> <span class="keyword">let</span> test2 () = assert_equal 100 (<span class="constructor">Foo</span>.unity 100);;<br> <br> <span class="comment">(* Name the test cases and group them together *)</span><br> <span class="keyword">let</span> suite = <br> <span class="string">"suite"</span>>:::<br> [<span class="string">"test1"</span>>:: test1;<br> <span class="string">"test2"</span>>:: test2]<br> ;;<br> <br> <span class="keyword">let</span> _ = <br> run_test_tt_main suite<br> ;;<br> </code></pre> <p> And compile the module <p> <pre class="codepre"><code class="code">$ ocamlfind ocamlc -o test -package oUnit -linkpkg -g foo.ml test.ml<br> </code></pre> <p> A executable named "test" will be created. When run it produces the following output. <p> <pre class="codepre"><code class="code">$ ./tests<br> ..<br> <span class="constructor">Ran</span>: 2 tests <span class="keyword">in</span>: 0.00 <span class="constructor">Seconds</span><br> <span class="constructor">OK</span><br> </code></pre> <p> When using <a href="OUnit.html#VALrun_test_tt_main"><code class="code"><span class="constructor">OUnit</span>.run_test_tt_main</code></a>, a non zero exit code signals that the test suite was not successful. <p> <h2 id="2_Advancedusage">Advanced usage</h2> <p> The topics, cover here, are only for advanced users who wish to unravel the power of OUnit. <p> <br> <table class="indextable"> <tr><td class="module"><a href="OUnit.html">OUnit</a></td><td><div class="info"> Unit test building blocks </div> </td></tr> <tr><td class="module"><a href="OUnitDiff.html">OUnitDiff</a></td><td><div class="info"> Unit tests for collection of elements </div> </td></tr> </table> <p> <h3 id="3_Errorreporting">Error reporting</h3> <p> The error reporting part of OUnit is quite important. If you want to identify the failure, you should tune the display of the value and the test. <p> Here is a list of thing you can display:<ul> <li>name of the test: OUnit use numbers to define path's test. But an error reporting about a failed test "0:1:2" is less explicit than "OUnit:0:comparator:1:float_comparator:2"</li> <li><code class="code">~msg</code> parameter: it allows you to define say which assert has failed in your test. When you have more than one assert in a test, you should provide a <code class="code">~msg</code> to be able to make the difference</li> <li><code class="code">~printer</code> parameter: <a href="OUnit.html#VALassert_equal"><code class="code"><span class="constructor">OUnit</span>.assert_equal</code></a> allows you to define a printer for compared values. A message <code class="code"><span class="string">"abcd"</span> is not equal <span class="keyword">to</span> <span class="string">"defg"</span></code> is better than <code class="code">not equal</code></li> </ul> <pre class="codepre"><code class="code"><span class="keyword">open</span> <span class="constructor">OUnit</span>;;<br> <br> <span class="keyword">let</span> _ = <br> <span class="string">"mytest"</span>>::<br> (<span class="keyword">fun</span> () <span class="keywordsign">-></span><br> assert_equal<br> ~msg:<span class="string">"int value"</span><br> ~printer:string_of_int<br> 1<br> (<span class="constructor">Foo</span>.unity 1))<br> ;;<br> </code></pre> <p> <h3 id="3_Commandlinearguments">Command line arguments</h3> <p> <a href="OUnit.html#VALrun_test_tt_main"><code class="code"><span class="constructor">OUnit</span>.run_test_tt_main</code></a> already provides a set of command line argument to help user to run only the test he wants:<ul> <li><code class="code">-only-test</code>: skip all the tests except this one, you can use this flag several time to select more than one test to run </li> <li><code class="code">-list-test</code>: list all the available tests and exit</li> <li><code class="code">-verbose</code>: rather than displaying dots while running the test, be more verbose</li> <li><code class="code">-help</code>: display help message and exit</li> </ul> It is also possible to add your own command-line arguments. You should do it if you want to define some extra arguments. For example: <p> <pre class="codepre"><code class="code"><span class="keyword">open</span> <span class="constructor">OUnit</span>;;<br> <br> <span class="keyword">let</span> my_program_to_test = <br> ref <span class="constructor">None</span><br> ;;<br> <br> <span class="keyword">let</span> test1 () =<br> <span class="keyword">match</span> !my_program_to_test <span class="keyword">with</span><br> <span class="keywordsign">|</span> <span class="constructor">Some</span> prg <span class="keywordsign">-></span> <br> assert_command prg []<br> <span class="keywordsign">|</span> <span class="constructor">None</span> <span class="keywordsign">-></span><br> skip_if <span class="keyword">true</span> <span class="string">"My program is not defined"</span><br> ;;<br> <br> <span class="keyword">let</span> _ = <br> run_test_tt_main<br> ~arg_specs:[<span class="string">"-my-program"</span>,<br> <span class="constructor">Arg</span>.<span class="constructor">String</span> (<span class="keyword">fun</span> fn <span class="keywordsign">-></span> my_program_to_test := <span class="constructor">Some</span> fn),<br> <span class="string">"fn Program to test"</span>]<br> (<span class="string">"test1"</span> >:: test1)<br> ;;<br> </code></pre> <p> <h3 id="3_Skipandtodotests">Skip and todo tests</h3> <p> Tests are not always meaningful and can even fail because something is missing in the environment. In order to manage this, you can define a skip condition that will skip the test. <p> If you start by defining your tests rather than implementing the functions under test, you know that some tests will just fail. You can mark these tests as to do tests, this way they will be reported differently in your test suite. <p> <pre class="codepre"><code class="code"><span class="keyword">open</span> <span class="constructor">OUnit</span>;;<br> <br> <span class="keyword">let</span> _ =<br> <span class="string">"allfuns"</span> >:::<br> [<br> <span class="string">"funix"</span>>::<br> (<span class="keyword">fun</span> () <span class="keywordsign">-></span><br> skip_if (<span class="constructor">Sys</span>.os_type = <span class="string">"Win32"</span>) <span class="string">"Don't work on Windows"</span>;<br> assert_equal<br> 0<br> (<span class="constructor">Foo</span>.funix ()));<br> <br> <span class="string">"fgeneric"</span>>::<br> (<span class="keyword">fun</span> () <span class="keywordsign">-></span><br> todo <span class="string">"fgeneric not implemented"</span>;<br> assert_equal<br> 0<br> (<span class="constructor">Foo</span>.fgeneric ()));<br> ]<br> ;;<br> </code></pre> <p> <h3 id="3_EffectiveOUnit">Effective OUnit</h3> <p> This section is about general tips about unit testing and OUnit. It is the result of some years using OUnit in real world applications. <p> <ul> <li>test everything: the more you create tests, the better chance you have to catch early an error in your program. Every submitted bugs to your application should have a matching tests. This is a good practice, but it is not always easy to implement.</li> <li>test only what is really exported: on the long term, you have to maintain your test suite. If you test low-level functions, you'll have a lot of tests to rewrite. You should focus on creating tests for functions for which the behavior shouldn't change.</li> <li>test fast: the best test suite is the one that runs after every single build. You should set your default Makefile target to run the test suite. It means that your test suite should be fast to run, typically, a 10s test suite is fine.</li> <li>test long: contrary to the former tip, you should also have a complete test suite which can be very long to run. The best way to achieve both tips, is to define a command line arguments <code class="code">-long</code> and skip the tests that are too long in your test suite according to it. When you do a release, you should use run your long test suite. </li> <li>family tests: when testing behavior, most of the time you call exactly the same code with different arguments. In this case <code class="code"><span class="constructor">List</span>.map</code> and <a href="OUnit.html#VAL(>:::)"><code class="code"><span class="constructor">OUnit</span>.(>:::)</code></a> are your friends. For example:</li> </ul> <pre class="codepre"><code class="code"><span class="keyword">open</span> <span class="constructor">OUnit</span>;;<br> <br> <span class="keyword">let</span> _ =<br> <span class="string">"Family"</span>>:::<br> (<span class="constructor">List</span>.map <br> (<span class="keyword">fun</span> (arg,res) <span class="keywordsign">-></span> <br> <span class="keyword">let</span> title = <br> <span class="constructor">Printf</span>.sprintf <span class="string">"%s->%s"</span> arg res<br> <span class="keyword">in</span><br> title >::<br> (<span class="keyword">fun</span> () <span class="keywordsign">-></span><br> assert_equal res (<span class="constructor">Foo</span>.unity arg)))<br> [<span class="string">"abcd"</span>, <span class="string">"abcd"</span>;<br> <span class="string">"defg"</span>, <span class="string">"defg"</span>;<br> <span class="string">"wxyz"</span>, <span class="string">"wxyz"</span>])<br> ;;<br> </code></pre> <p> <ul> <li>test failures and successes: the most obvious thing you want to test are successes, i.e. that you get the expected behavior in the normal case. But most of the errors arise in corner cases and in the code of the test itself. For example, you can have a partial application of your <a href="OUnit.html#VALassert_equal"><code class="code"><span class="constructor">OUnit</span>.assert_equal</code></a> and never encounter any errors, just because the <code class="code">assert_equal</code> is not called. In this case, if you test errors as well, you will have a missing errors as well.</li> <li>set up and clean your environment in the test: you should not set up and clean your test environment outside the test. Ideally, if you run no tests, the program should do nothing. This is also a sane way to be sure that you are always testing in a clean environment, not polluted by the result of failed tests run before.</li> </ul> <pre class="codepre"><code class="code"><span class="keyword">open</span> <span class="constructor">OUnit</span>;;<br> <br> <span class="keyword">let</span> _ =<br> <span class="comment">(* We need to call a function in a particular directory *)</span><br> <span class="string">"change-dir-and-run"</span>>::<br> bracket<br> (<span class="keyword">fun</span> () <span class="keywordsign">-></span> <br> <span class="keyword">let</span> pwd = <span class="constructor">Sys</span>.getcwd () <span class="keyword">in</span><br> <span class="constructor">Sys</span>.chdir <span class="string">"test"</span>;<br> pwd)<br> (<span class="keyword">fun</span> _ <span class="keywordsign">-></span><br> assert_command <span class="string">"ls"</span> [])<br> (<span class="keyword">fun</span> pwd <span class="keywordsign">-></span><br> <span class="constructor">Sys</span>.chdir pwd)<br> ;;<br> </code></pre><ul> <li>separate your test: OUnit test code should live outside the code under tests. This allow to drop the dependency on OUnit when distributing your library/application. This also enables people to easily make a difference from what really matters (the main code) and what are only tests. It is possible to have it directly in the code, like in Quickcheck style tests. </li> </ul> The unit testing scope is always hard to define. Unit testing should be about testing a single features. But OUnit can help you to test higher level behavior, by running a full program for example. While it isn't real unit testing, you can use OUnit to do it and should not hesitate to do it. <p> In term of line of codes, a test suite can represent from 10% to 150% of the code under test. With time, your test suite will grow faster than your program/library. A good ratio is 33%.<br> <b>Author(s):</b> Maas-Maarten Zeeman, Sylvain Le Gall<br> </body> </html>
