<!DOCTYPE html> <!--[if IE 8]><html class="no-js lt-ie9" lang="en" > <![endif]--> <!--[if gt IE 8]><!--> <html class="no-js" lang="en" > <!--<![endif]--> <head> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <title>Writing and running tests — dune documentation</title> <link rel="stylesheet" href="_static/css/theme.css" type="text/css" /> <link rel="stylesheet" href="_static/pygments.css" type="text/css" /> <link rel="index" title="Index" href="genindex.html" /> <link rel="search" title="Search" href="search.html" /> <link rel="next" title="Generating Documentation" href="documentation.html" /> <link rel="prev" title="dune files" href="dune-files.html" /> <script src="_static/js/modernizr.min.js"></script> </head> <body class="wy-body-for-nav"> <div class="wy-grid-for-nav"> <nav data-toggle="wy-nav-shift" class="wy-nav-side"> <div class="wy-side-scroll"> <div class="wy-side-nav-search"> <a href="index.html" class="icon icon-home"> dune </a> <div role="search"> <form id="rtd-search-form" class="wy-form" action="search.html" method="get"> <input type="text" name="q" placeholder="Search docs" /> <input type="hidden" name="check_keywords" value="yes" /> <input type="hidden" name="area" value="default" /> </form> </div> </div> <div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation"> <ul class="current"> <li class="toctree-l1"><a class="reference internal" href="quick-start.html">Quickstart</a></li> <li class="toctree-l1"><a class="reference internal" href="overview.html">Overview</a></li> <li class="toctree-l1"><a class="reference internal" href="terminology.html">Terminology</a></li> <li class="toctree-l1"><a class="reference internal" href="project-layout-specification.html">Project Layout and Metadata Specification</a></li> <li class="toctree-l1"><a class="reference internal" href="dune-files.html">dune files</a></li> <li class="toctree-l1 current"><a class="current reference internal" href="#">Writing and running tests</a><ul> <li class="toctree-l2"><a class="reference internal" href="#running-tests">Running tests</a></li> <li class="toctree-l2"><a class="reference internal" href="#inline-tests">Inline tests</a><ul> <li class="toctree-l3"><a class="reference internal" href="#inline-expectation-tests">Inline expectation tests</a></li> <li class="toctree-l3"><a class="reference internal" href="#specifying-inline-test-dependencies">Specifying inline test dependencies</a></li> <li class="toctree-l3"><a class="reference internal" href="#passing-special-arguments-to-the-test-runner">Passing special arguments to the test runner</a></li> <li class="toctree-l3"><a class="reference internal" href="#using-additional-libraries-in-the-test-runner">Using additional libraries in the test runner</a></li> <li class="toctree-l3"><a class="reference internal" href="#defining-your-own-inline-test-backend">Defining your own inline test backend</a><ul> <li class="toctree-l4"><a class="reference internal" href="#example-of-backend">Example of backend</a></li> </ul> </li> </ul> </li> <li class="toctree-l2"><a class="reference internal" href="#custom-tests">Custom tests</a><ul> <li class="toctree-l3"><a class="reference internal" href="#diffing-the-result">Diffing the result</a></li> </ul> </li> </ul> </li> <li class="toctree-l1"><a class="reference internal" href="documentation.html">Generating Documentation</a></li> <li class="toctree-l1"><a class="reference internal" href="usage.html">Usage</a></li> <li class="toctree-l1"><a class="reference internal" href="advanced-topics.html">Advanced topics</a></li> <li class="toctree-l1"><a class="reference internal" href="configurator.html">Configurator</a></li> <li class="toctree-l1"><a class="reference internal" href="menhir.html">Menhir</a></li> <li class="toctree-l1"><a class="reference internal" href="faq.html">FAQ</a></li> <li class="toctree-l1"><a class="reference internal" href="known-issues.html">Known Issues</a></li> <li class="toctree-l1"><a class="reference internal" href="migration.html">Migration</a></li> </ul> </div> </div> </nav> <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap"> <nav class="wy-nav-top" aria-label="top navigation"> <i data-toggle="wy-nav-top" class="fa fa-bars"></i> <a href="index.html">dune</a> </nav> <div class="wy-nav-content"> <div class="rst-content"> <div role="navigation" aria-label="breadcrumbs navigation"> <ul class="wy-breadcrumbs"> <li><a href="index.html">Docs</a> »</li> <li>Writing and running tests</li> <li class="wy-breadcrumbs-aside"> <a href="_sources/tests.rst.txt" rel="nofollow"> View page source</a> </li> </ul> <hr/> </div> <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article"> <div itemprop="articleBody"> <div class="section" id="writing-and-running-tests"> <h1>Writing and running tests<a class="headerlink" href="#writing-and-running-tests" title="Permalink to this headline">¶</a></h1> <p>Dune tries to streamline the testing story as much as possible, so that you can focus on the tests themselves and not bother with setting up with various test frameworks.</p> <p>In this section, we will explain the workflow to deal with tests in dune. In particular we will see how to run the testsuite of a project, how to describe your tests to dune and how to promote tests result as expectation.</p> <p>We distinguish two kinds of tests: inline tests and custom tests. Inline tests are usually written directly inside the ml files of a library. They are the easiest to work with and usually requires nothing more than writing <code class="docutils literal notranslate"><span class="pre">(inline_tests)</span></code> inside your library stanza. Custom tests consist on executing an executable and sometimes do something afterwards, such as diffing its output.</p> <div class="section" id="running-tests"> <h2>Running tests<a class="headerlink" href="#running-tests" title="Permalink to this headline">¶</a></h2> <p>Whatever the tests of a project are, the usual way to run tests with dune is to call <code class="docutils literal notranslate"><span class="pre">dune</span> <span class="pre">runtest</span></code> from the shell. This will run all the tests defined in the current directory and any sub-directory recursively. You can also pass a directory argument to run the tests from a sub-tree. For instance <code class="docutils literal notranslate"><span class="pre">dune</span> <span class="pre">runtest</span> <span class="pre">test</span></code> will only run the tests from the <code class="docutils literal notranslate"><span class="pre">test</span></code> directory and any sub-directory of <code class="docutils literal notranslate"><span class="pre">test</span></code> recursively.</p> <p>Note that in any case, <code class="docutils literal notranslate"><span class="pre">dune</span> <span class="pre">runtest</span></code> is simply a short-hand for building the <code class="docutils literal notranslate"><span class="pre">runtest</span></code> alias, so you can always ask dune to run the tests in conjunction with other targets by passing <code class="docutils literal notranslate"><span class="pre">@runtest</span></code> to <code class="docutils literal notranslate"><span class="pre">dune</span> <span class="pre">build</span></code>. For instance:</p> <div class="code bash highlight-default notranslate"><div class="highlight"><pre><span></span>$ dune build @install @runtest $ dune build @install @test/runtest </pre></div> </div> </div> <div class="section" id="inline-tests"> <h2>Inline tests<a class="headerlink" href="#inline-tests" title="Permalink to this headline">¶</a></h2> <p>There are several inline tests framework available for OCaml, such as <a class="reference external" href="https://github.com/janestreet/ppx_inline_test">ppx_inline_test</a> and <a class="reference external" href="https://github.com/vincent-hugot/qtest">qtest</a>. We will use <a class="reference external" href="https://github.com/janestreet/ppx_inline_test">ppx_inline_test</a> as an example as at the time of writing this document it has the necessary setup to be used with dune out of the box.</p> <p><a class="reference external" href="https://github.com/janestreet/ppx_inline_test">ppx_inline_test</a> allows to write tests directly inside ml files as follow:</p> <div class="code ocaml highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">let</span> <span class="n">rec</span> <span class="n">fact</span> <span class="n">n</span> <span class="o">=</span> <span class="k">if</span> <span class="n">n</span> <span class="o">=</span> <span class="mi">1</span> <span class="n">then</span> <span class="mi">1</span> <span class="k">else</span> <span class="n">n</span> <span class="o">*</span> <span class="n">fact</span> <span class="p">(</span><span class="n">n</span> <span class="o">-</span> <span class="mi">1</span><span class="p">)</span> <span class="n">let</span><span class="o">%</span><span class="n">test</span> <span class="n">_</span> <span class="o">=</span> <span class="n">fact</span> <span class="mi">5</span> <span class="o">=</span> <span class="mi">120</span> </pre></div> </div> <p>The file has to be preprocessed with the ppx_inline_test ppx rewriter, so for instance the <code class="docutils literal notranslate"><span class="pre">jbuild</span></code> file might look like this:</p> <div class="code scheme highlight-default notranslate"><div class="highlight"><pre><span></span><span class="p">(</span><span class="n">library</span> <span class="p">((</span><span class="n">name</span> <span class="n">foo</span><span class="p">)</span> <span class="p">(</span><span class="n">preprocess</span> <span class="p">(</span><span class="n">pps</span> <span class="p">(</span><span class="n">ppx_inline_test</span><span class="p">)))))</span> </pre></div> </div> <p>In order to instruct dune that our library contains inline tests, all we have to do is add an <code class="docutils literal notranslate"><span class="pre">inline_tests</span></code> field:</p> <div class="code scheme highlight-default notranslate"><div class="highlight"><pre><span></span><span class="p">(</span><span class="n">library</span> <span class="p">((</span><span class="n">name</span> <span class="n">foo</span><span class="p">)</span> <span class="p">(</span><span class="n">inline_tests</span><span class="p">)</span> <span class="p">(</span><span class="n">preprocess</span> <span class="p">(</span><span class="n">pps</span> <span class="p">(</span><span class="n">ppx_inline_test</span><span class="p">)))))</span> </pre></div> </div> <p>We can now build and execute this test by running <code class="docutils literal notranslate"><span class="pre">dune</span> <span class="pre">runtest</span></code>. For instance, if we make the test fail by replacing <code class="docutils literal notranslate"><span class="pre">120</span></code> by <code class="docutils literal notranslate"><span class="pre">0</span></code> we get:</p> <div class="code bash highlight-default notranslate"><div class="highlight"><pre><span></span>$ dune runtest [...] File "src/fact.ml", line 3, characters 0-25: <<(fact 5) = 0>> is false. FAILED 1 / 1 tests </pre></div> </div> <p>Note that in this case Jbuild knew how to build and run the tests without any special configuration. This is because ppx_inline_test defines an inline tests backend and it is used by the library. Some other frameworks, such as <a class="reference external" href="https://github.com/vincent-hugot/qtest">qtest</a> don’t have any special library or ppx rewriter. To use such a framework, you must tell dune about it since it cannot guess it. You can do that by adding a <code class="docutils literal notranslate"><span class="pre">backend</span></code> field:</p> <div class="code scheme highlight-default notranslate"><div class="highlight"><pre><span></span><span class="p">(</span><span class="n">library</span> <span class="p">((</span><span class="n">name</span> <span class="n">foo</span><span class="p">)</span> <span class="p">(</span><span class="n">inline_tests</span> <span class="p">((</span><span class="n">backend</span> <span class="n">qtest</span><span class="p">)))))</span> </pre></div> </div> <div class="section" id="inline-expectation-tests"> <h3>Inline expectation tests<a class="headerlink" href="#inline-expectation-tests" title="Permalink to this headline">¶</a></h3> <p>Inline expectation tests are a special case of inline tests where you write a bit of OCaml code that prints something followed by what you expect this code to print. For instance, using <a class="reference external" href="https://github.com/janestreet/ppx_expect">ppx_expect</a>:</p> <div class="code ocaml highlight-default notranslate"><div class="highlight"><pre><span></span>let%expect_test _ = print_endline "Hello, world!"; [%expect{| Hello, world! |}] </pre></div> </div> <p>The test procedure consist of executing the OCaml code and replacing the contents of the <code class="docutils literal notranslate"><span class="pre">[%expect]</span></code> extension point by the real output. You then get a new file that you can compare to the original source file. Expectation tests are a neat way to write tests as the following test elements are clearly identified:</p> <ul class="simple"> <li>the code of the test</li> <li>the test expectation</li> <li>the test outcome</li> </ul> <p>You can have a look at <a class="reference external" href="https://blog.janestreet.com/testing-with-expectations/">this blog post</a> to find out more about expectation tests. To dune, the workflow for expectation tests is always as follows:</p> <ul class="simple"> <li>write the test with some empty expect nodes in it</li> <li>run the tests</li> <li>check the suggested correction and promote it as the original source file if you are happy with it</li> </ul> <p>Dune makes this workflow very easy, simply add <code class="docutils literal notranslate"><span class="pre">ppx_expect</span></code> to your list of ppx rewriters as follow:</p> <div class="code scheme highlight-default notranslate"><div class="highlight"><pre><span></span><span class="p">(</span><span class="n">library</span> <span class="p">((</span><span class="n">name</span> <span class="n">foo</span><span class="p">)</span> <span class="p">(</span><span class="n">inline_tests</span><span class="p">)</span> <span class="p">(</span><span class="n">preprocess</span> <span class="p">(</span><span class="n">pps</span> <span class="p">(</span><span class="n">ppx_expect</span><span class="p">)))))</span> </pre></div> </div> <p>Then calling <code class="docutils literal notranslate"><span class="pre">dune</span> <span class="pre">runtest</span></code> will run these tests and in case of mismatch dune will print a diff of the original source file and the suggested correction. For instance:</p> <div class="code bash highlight-default notranslate"><div class="highlight"><pre><span></span>$ dune runtest [...] -src/fact.ml +src/fact.ml.corrected File "src/fact.ml", line 5, characters 0-1: let rec fact n = if n = 1 then 1 else n * fact (n - 1) let%expect_test _ = print_int (fact 5); - [%expect] + [%expect{| 120 |}] </pre></div> </div> <p>In order to accept the correction, simply run:</p> <div class="code bash highlight-default notranslate"><div class="highlight"><pre><span></span>$ dune promote </pre></div> </div> <p>You can also make dune automatically accept the correction after running the tests by typing:</p> <div class="code bash highlight-default notranslate"><div class="highlight"><pre><span></span>$ dune runtest --auto-promote </pre></div> </div> <p>Finally, some editor integration is possible to make the editor do the promotion and make the workflow even smoother.</p> </div> <div class="section" id="specifying-inline-test-dependencies"> <h3>Specifying inline test dependencies<a class="headerlink" href="#specifying-inline-test-dependencies" title="Permalink to this headline">¶</a></h3> <p>If your tests are reading files, you must say it to dune by adding a <code class="docutils literal notranslate"><span class="pre">deps</span></code> field the the <code class="docutils literal notranslate"><span class="pre">inline_tests</span></code> field. The argument of this <code class="docutils literal notranslate"><span class="pre">deps</span></code> field follows the usual <a class="reference internal" href="dune-files.html#deps-field"><span class="std std-ref">Dependency specification</span></a>. For instance:</p> <div class="code ocaml highlight-default notranslate"><div class="highlight"><pre><span></span><span class="p">(</span><span class="n">library</span> <span class="p">((</span><span class="n">name</span> <span class="n">foo</span><span class="p">)</span> <span class="p">(</span><span class="n">inline_tests</span> <span class="p">((</span><span class="n">deps</span> <span class="p">(</span><span class="n">data</span><span class="o">.</span><span class="n">txt</span><span class="p">))))</span> <span class="p">(</span><span class="n">preprocess</span> <span class="p">(</span><span class="n">pps</span> <span class="p">(</span><span class="n">ppx_expect</span><span class="p">)))))</span> </pre></div> </div> </div> <div class="section" id="passing-special-arguments-to-the-test-runner"> <h3>Passing special arguments to the test runner<a class="headerlink" href="#passing-special-arguments-to-the-test-runner" title="Permalink to this headline">¶</a></h3> <p>Under the hood, a test executable is built by dune. Depending on the backend used this runner might take useful command line arguments. You can specify such flags by using a <code class="docutils literal notranslate"><span class="pre">flags</span></code> field, such as:</p> <div class="code ocaml highlight-default notranslate"><div class="highlight"><pre><span></span><span class="p">(</span><span class="n">library</span> <span class="p">((</span><span class="n">name</span> <span class="n">foo</span><span class="p">)</span> <span class="p">(</span><span class="n">inline_tests</span> <span class="p">((</span><span class="n">flags</span> <span class="p">(</span><span class="o">-</span><span class="n">foo</span> <span class="n">bar</span><span class="p">))))</span> <span class="p">(</span><span class="n">preprocess</span> <span class="p">(</span><span class="n">pps</span> <span class="p">(</span><span class="n">ppx_expect</span><span class="p">)))))</span> </pre></div> </div> <p>The argument of the <code class="docutils literal notranslate"><span class="pre">flags</span></code> field follows the <a class="reference internal" href="dune-files.html#ordered-set-language"><span class="std std-ref">Ordered set language</span></a>.</p> </div> <div class="section" id="using-additional-libraries-in-the-test-runner"> <h3>Using additional libraries in the test runner<a class="headerlink" href="#using-additional-libraries-in-the-test-runner" title="Permalink to this headline">¶</a></h3> <p>When tests are not part of the library code, it is possible that tests require additional libraries than the library being tested. This is the case with <a class="reference external" href="https://github.com/vincent-hugot/qtest">qtest</a> as tests are written in comments. You can specify such libraries using a <code class="docutils literal notranslate"><span class="pre">libraries</span></code> field, such as:</p> <div class="code ocaml highlight-default notranslate"><div class="highlight"><pre><span></span><span class="p">(</span><span class="n">library</span> <span class="p">((</span><span class="n">name</span> <span class="n">foo</span><span class="p">)</span> <span class="p">(</span><span class="n">inline_tests</span> <span class="p">((</span><span class="n">backend</span> <span class="n">qtest</span><span class="p">)</span> <span class="p">(</span><span class="n">libraries</span> <span class="p">(</span><span class="n">bar</span><span class="p">))))))</span> </pre></div> </div> </div> <div class="section" id="defining-your-own-inline-test-backend"> <h3>Defining your own inline test backend<a class="headerlink" href="#defining-your-own-inline-test-backend" title="Permalink to this headline">¶</a></h3> <p>If you are writing a test framework, or for specific cases, you might want to define your own inline tests backend. If your framework is naturally implemented by a library or ppx rewriter that the user must use when they want to write tests, then you should define this library has a backend. Otherwise simply create an empty library with the name you want to give for your backend.</p> <p>In order to define a library as an inline tests backend, simply add an <code class="docutils literal notranslate"><span class="pre">inline_tests.backend</span></code> field to the library stanza. An inline tests backend is specified by thee parameters:</p> <ol class="arabic simple"> <li>How to create the test runner</li> <li>How to build the test runner</li> <li>How to run the test runner</li> </ol> <p>These three parameters can be specified inside the <code class="docutils literal notranslate"><span class="pre">inline_tests.backend</span></code> field, which accepts the following fields:</p> <div class="code scheme highlight-default notranslate"><div class="highlight"><pre><span></span><span class="p">(</span><span class="n">generate_runner</span> <span class="o"><</span><span class="n">action</span><span class="o">></span><span class="p">)</span> <span class="p">(</span><span class="n">runner_libraries</span> <span class="p">(</span><span class="o"><</span><span class="n">ocaml</span><span class="o">-</span><span class="n">libraries</span><span class="o">></span><span class="p">))</span> <span class="p">(</span><span class="n">flags</span> <span class="o"><</span><span class="n">flags</span><span class="o">></span><span class="p">)</span> <span class="p">(</span><span class="n">extends</span> <span class="p">(</span><span class="o"><</span><span class="n">backends</span><span class="o">></span><span class="p">))</span> </pre></div> </div> <p>For instance:</p> <p><code class="docutils literal notranslate"><span class="pre"><action></span></code> follows the <a class="reference internal" href="dune-files.html#user-actions"><span class="std std-ref">User actions</span></a> specification. It describe an action that should be executed in the directory of libraries using this backend for their tests. It is expected that the action produces some OCaml code on its standard output. This code will constitute the test runner. The action can use the following additional variables:</p> <ul class="simple"> <li><code class="docutils literal notranslate"><span class="pre">${library-name}</span></code> which is the name of the library being tested</li> <li><code class="docutils literal notranslate"><span class="pre">${impl-files}</span></code> which is the list of implementation files in the library, i.e. all the <code class="docutils literal notranslate"><span class="pre">.ml</span></code> and <code class="docutils literal notranslate"><span class="pre">.re</span></code> files</li> <li><code class="docutils literal notranslate"><span class="pre">${intf-files}</span></code> which is the list of interface files in the library, i.e. all the <code class="docutils literal notranslate"><span class="pre">.mli</span></code> and <code class="docutils literal notranslate"><span class="pre">.rei</span></code> files</li> </ul> <p>The <code class="docutils literal notranslate"><span class="pre">runner_libraries</span></code> field specifies what OCaml libraries the test runner uses. For instance, if the <code class="docutils literal notranslate"><span class="pre">generate_runner</span></code> actions generates something like <code class="docutils literal notranslate"><span class="pre">My_test_framework.runtests</span> <span class="pre">()</span></code>, the you should probably put <code class="docutils literal notranslate"><span class="pre">my_test_framework</span></code> in the <code class="docutils literal notranslate"><span class="pre">runner_libraries</span></code> field.</p> <p>If you test runner needs specific flags, you should pass them in the <code class="docutils literal notranslate"><span class="pre">flags</span></code> field. You can use the <code class="docutils literal notranslate"><span class="pre">${library-name}</span></code> variable in this field.</p> <p>Finally, a backend can be an extension of another backend. In this case you must specify by in the <code class="docutils literal notranslate"><span class="pre">extends</span></code> field. For instance, <a class="reference external" href="https://github.com/janestreet/ppx_expect">ppx_expect</a> is an extension of <a class="reference external" href="https://github.com/janestreet/ppx_inline_test">ppx_inline_test</a>. It is possible to use a backend with several extensions in a library, however there must be exactly one <em>root backend</em>, i.e. exactly one backend that is not an extension of another one.</p> <p>When using a backend with extensions, the various fields are simply concatenated. The order in which they are concatenated is unspecified, however if a backend <code class="docutils literal notranslate"><span class="pre">b</span></code> extends of a backend <code class="docutils literal notranslate"><span class="pre">a</span></code>, then <code class="docutils literal notranslate"><span class="pre">a</span></code> will always come before <code class="docutils literal notranslate"><span class="pre">b</span></code>.</p> <div class="section" id="example-of-backend"> <h4>Example of backend<a class="headerlink" href="#example-of-backend" title="Permalink to this headline">¶</a></h4> <p>In this example, we put tests in comments of the form:</p> <div class="code ocaml highlight-default notranslate"><div class="highlight"><pre><span></span><span class="p">(</span><span class="o">*</span><span class="n">TEST</span><span class="p">:</span> <span class="k">assert</span> <span class="p">(</span><span class="n">fact</span> <span class="mi">5</span> <span class="o">=</span> <span class="mi">120</span><span class="p">)</span> <span class="o">*</span><span class="p">)</span> </pre></div> </div> <p>The backend for such a framework looks like this:</p> <div class="code scheme highlight-default notranslate"><div class="highlight"><pre><span></span><span class="p">(</span><span class="n">library</span> <span class="p">((</span><span class="n">name</span> <span class="n">simple_tests</span><span class="p">)</span> <span class="p">(</span><span class="n">inline_tests</span><span class="o">.</span><span class="n">backend</span> <span class="p">((</span><span class="n">generate_runner</span> <span class="p">(</span><span class="n">run</span> <span class="n">sed</span> <span class="s2">"s/(</span><span class="se">\\</span><span class="s2">*TEST:</span><span class="se">\\</span><span class="s2">(.*</span><span class="se">\\</span><span class="s2">)</span><span class="se">\\</span><span class="s2">*)/let () = </span><span class="se">\\</span><span class="s2">1;;/"</span> <span class="o">%</span><span class="p">{</span><span class="n">impl</span><span class="o">-</span><span class="n">files</span><span class="p">}))</span> <span class="p">))))</span> </pre></div> </div> <p>Now all you have to do is write <code class="docutils literal notranslate"><span class="pre">(inline_tests</span> <span class="pre">((backend</span> <span class="pre">simple_tests)))</span></code> wherever you want to write such tests. Note that this is only an example, we do not recommend using <code class="docutils literal notranslate"><span class="pre">sed</span></code> in your build as this would cause portability problems.</p> </div> </div> </div> <div class="section" id="custom-tests"> <h2>Custom tests<a class="headerlink" href="#custom-tests" title="Permalink to this headline">¶</a></h2> <p>We said in <a class="reference internal" href="#running-tests">Running tests</a> that to run tests dune simply builds the <code class="docutils literal notranslate"><span class="pre">runtest</span></code> alias. As a result, to define cutsom tests, you simply need to add an action to this alias in any directory. For instance if you have a binary <code class="docutils literal notranslate"><span class="pre">tests.exe</span></code> that you want to run as part of running your testsuite, simply add this to a jbuild file:</p> <div class="code scheme highlight-default notranslate"><div class="highlight"><pre><span></span><span class="p">(</span><span class="n">alias</span> <span class="p">((</span><span class="n">name</span> <span class="n">runtest</span><span class="p">)</span> <span class="p">(</span><span class="n">action</span> <span class="p">(</span><span class="n">run</span> <span class="o">./</span><span class="n">tests</span><span class="o">.</span><span class="n">exe</span><span class="p">))))</span> </pre></div> </div> <p>Hence to define an a test a pair of alias and executable stanzas are required. To simplify this common pattern, dune provides a <a class="reference internal" href="dune-files.html#tests-stanza"><span class="std std-ref">tests</span></a> stanza to define multiple tests and their aliases at once:</p> <div class="code scheme highlight-default notranslate"><div class="highlight"><pre><span></span><span class="p">(</span><span class="n">tests</span> <span class="p">(</span><span class="n">names</span> <span class="n">test1</span> <span class="n">test2</span><span class="p">))</span> </pre></div> </div> <div class="section" id="diffing-the-result"> <h3>Diffing the result<a class="headerlink" href="#diffing-the-result" title="Permalink to this headline">¶</a></h3> <p>It is often the case that we want to compare the output of a test to some expected one. For that, dune offers the <code class="docutils literal notranslate"><span class="pre">diff</span></code> command, which in essence is the same as running the <code class="docutils literal notranslate"><span class="pre">diff</span></code> tool, except that it is more integrated in dune and especially with the <code class="docutils literal notranslate"><span class="pre">promote</span></code> command. For instance let’s consider this test:</p> <div class="code scheme highlight-default notranslate"><div class="highlight"><pre><span></span><span class="p">(</span><span class="n">rule</span> <span class="p">(</span><span class="k">with</span><span class="o">-</span><span class="n">stdout</span><span class="o">-</span><span class="n">to</span> <span class="n">tests</span><span class="o">.</span><span class="n">output</span> <span class="p">(</span><span class="n">run</span> <span class="o">./</span><span class="n">tests</span><span class="o">.</span><span class="n">exe</span><span class="p">)))</span> <span class="p">(</span><span class="n">alias</span> <span class="p">((</span><span class="n">name</span> <span class="n">runtest</span><span class="p">)</span> <span class="p">(</span><span class="n">action</span> <span class="p">(</span><span class="n">diff</span> <span class="n">tests</span><span class="o">.</span><span class="n">expected</span> <span class="n">test</span><span class="o">.</span><span class="n">output</span><span class="p">))))</span> </pre></div> </div> <p>After having run <code class="docutils literal notranslate"><span class="pre">tests.exe</span></code> and dumping its output to <code class="docutils literal notranslate"><span class="pre">tests.output</span></code>, dune will compare the latter to <code class="docutils literal notranslate"><span class="pre">tests.expected</span></code>. In case of mismatch, dune will print a diff and then the <code class="docutils literal notranslate"><span class="pre">dune</span> <span class="pre">promote</span></code> command can be used to copy over the generated <code class="docutils literal notranslate"><span class="pre">test.output</span></code> file to <code class="docutils literal notranslate"><span class="pre">tests.expected</span></code> in the source tree.</p> <p>Alternatively, the <a class="reference internal" href="dune-files.html#tests-stanza"><span class="std std-ref">tests</span></a> also supports this style of tests.</p> <div class="code scheme highlight-default notranslate"><div class="highlight"><pre><span></span><span class="p">(</span><span class="n">tests</span> <span class="p">(</span><span class="n">names</span> <span class="n">tests</span><span class="p">))</span> </pre></div> </div> <p>Where dune expects a <code class="docutils literal notranslate"><span class="pre">tests.expected</span></code> file to exist to infer that this is an expect tests.</p> <p>This provides a nice way of dealing with the usual <em>write code</em>, <em>run</em>, <em>promote</em> cycle of testing. For instance:</p> <div class="code bash highlight-default notranslate"><div class="highlight"><pre><span></span>$ dune runtest [...] -tests.expected +tests.output File "tests.expected", line 1, characters 0-1: -Hello, world! +Good bye! $ dune promote Promoting _build/default/tests.output to tests.expected. </pre></div> </div> <p>Note that if available, the diffing is done using the <a class="reference external" href="https://github.com/janestreet/patdiff">patdiff</a> tool, which displays nicer looking diffs that the standard <code class="docutils literal notranslate"><span class="pre">diff</span></code> tool. You can change that by passing <code class="docutils literal notranslate"><span class="pre">--diff-command</span> <span class="pre">CMD</span></code> to dune.</p> </div> </div> </div> </div> </div> <footer> <div class="rst-footer-buttons" role="navigation" aria-label="footer navigation"> <a href="documentation.html" class="btn btn-neutral float-right" title="Generating Documentation" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right"></span></a> <a href="dune-files.html" class="btn btn-neutral" title="dune files" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left"></span> Previous</a> </div> <hr/> <div role="contentinfo"> <p> © Copyright 2017, Jérémie Dimino </p> </div> Built with <a href="http://sphinx-doc.org/">Sphinx</a> using a <a href="https://github.com/rtfd/sphinx_rtd_theme">theme</a> provided by <a href="https://readthedocs.org">Read the Docs</a>. </footer> </div> </div> </section> </div> <script type="text/javascript"> var DOCUMENTATION_OPTIONS = { URL_ROOT:'./', VERSION:'', LANGUAGE:'None', COLLAPSE_INDEX:false, FILE_SUFFIX:'.html', HAS_SOURCE: true, SOURCELINK_SUFFIX: '.txt' }; </script> <script type="text/javascript" src="_static/jquery.js"></script> <script type="text/javascript" src="_static/underscore.js"></script> <script type="text/javascript" src="_static/doctools.js"></script> <script type="text/javascript" src="_static/js/theme.js"></script> <script type="text/javascript"> jQuery(function () { SphinxRtdTheme.Navigation.enable(true); }); </script> </body> </html>