<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml"> <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <title>Parametrizing tests</title> <link rel="stylesheet" href="../_static/sphinxdoc.css" type="text/css" /> <link rel="stylesheet" href="../_static/pygments.css" type="text/css" /> <script type="text/javascript"> var DOCUMENTATION_OPTIONS = { URL_ROOT: '../', VERSION: '2.3.4.1', COLLAPSE_INDEX: false, FILE_SUFFIX: '.html', HAS_SOURCE: true }; </script> <script type="text/javascript" src="../_static/jquery.js"></script> <script type="text/javascript" src="../_static/underscore.js"></script> <script type="text/javascript" src="../_static/doctools.js"></script> <link rel="top" title="None" href="../index.html" /> <link rel="up" title="Usages and Examples" href="index.html" /> <link rel="next" title="Working with custom markers" href="markers.html" /> <link rel="prev" title="Basic patterns and examples" href="simple.html" /> </head> <body> <div class="related"> <h3>Navigation</h3> <ul> <li class="right" style="margin-right: 10px"> <a href="markers.html" title="Working with custom markers" accesskey="N">next</a></li> <li class="right" > <a href="simple.html" title="Basic patterns and examples" accesskey="P">previous</a> |</li> <li><a href="../contents.html">pytest-2.3.4.1</a> »</li> <li><a href="index.html" accesskey="U">Usages and Examples</a> »</li> <g:plusone></g:plusone> </ul> </div> <div class="sphinxsidebar"> <div class="sphinxsidebarwrapper"> <div id="searchbox" style="display: none"> <form class="search" action="../search.html" method="get"> <input type="text" name="q" size="18" /> <input type="submit" value="Search" /> <input type="hidden" name="check_keywords" value="yes" /> <input type="hidden" name="area" value="default" /> </form> </div> <script type="text/javascript">$('#searchbox').show(0);</script> <h3>quicklinks</h3> <div style="text-align: left; font-size: 100%; vertical-align: middle;"> <table> <tr> <td> <a href="../index.html">home</a> </td><td> <a href="../contents.html">TOC/contents</a> </td></tr><tr><td> <a href="../getting-started.html">install</a> </td><td> <a href="../changelog.html">changelog</a> </td></tr><tr><td> <a href="index.html">examples</a> </td><td> <a href="../customize.html">customize</a> </td></tr><tr><td> <a href="https://bitbucket.org/hpk42/pytest/issues?status=new&status=open">issues[bb]</a> </td><td> <a href="../contact.html">contact</a> </td></tr></table> </div> <h3><a href="../contents.html">Table Of Contents</a></h3> <ul> <li><a class="reference internal" href="#">Parametrizing tests</a><ul> <li><a class="reference internal" href="#generating-parameters-combinations-depending-on-command-line">Generating parameters combinations, depending on command line</a></li> <li><a class="reference internal" href="#a-quick-port-of-testscenarios">A quick port of “testscenarios”</a></li> <li><a class="reference internal" href="#deferring-the-setup-of-parametrized-resources">Deferring the setup of parametrized resources</a></li> <li><a class="reference internal" href="#parametrizing-test-methods-through-per-class-configuration">Parametrizing test methods through per-class configuration</a></li> <li><a class="reference internal" href="#indirect-parametrization-with-multiple-fixtures">Indirect parametrization with multiple fixtures</a></li> <li><a class="reference internal" href="#indirect-parametrization-of-optional-implementations-imports">Indirect parametrization of optional implementations/imports</a></li> </ul> </li> </ul> <h4>Previous topic</h4> <p class="topless"><a href="simple.html" title="previous chapter">Basic patterns and examples</a></p> <h4>Next topic</h4> <p class="topless"><a href="markers.html" title="next chapter">Working with custom markers</a></p> </div> </div> <div class="document"> <div class="documentwrapper"> <div class="bodywrapper"> <div class="body"> <div class="section" id="parametrizing-tests"> <span id="paramexamples"></span><h1>Parametrizing tests<a class="headerlink" href="#parametrizing-tests" title="Permalink to this headline">¶</a></h1> <p>py.test allows to easily parametrize test functions. For basic docs, see <a class="reference internal" href="../parametrize.html#parametrize-basics"><em>Parametrizing fixtures and test functions</em></a>.</p> <p>In the following we provide some examples using the builtin mechanisms.</p> <div class="section" id="generating-parameters-combinations-depending-on-command-line"> <h2>Generating parameters combinations, depending on command line<a class="headerlink" href="#generating-parameters-combinations-depending-on-command-line" title="Permalink to this headline">¶</a></h2> <p>Let’s say we want to execute a test with different computation parameters and the parameter range shall be determined by a command line argument. Let’s first write a simple (do-nothing) computation test:</p> <div class="highlight-python"><div class="highlight"><pre><span class="c"># content of test_compute.py</span> <span class="k">def</span> <span class="nf">test_compute</span><span class="p">(</span><span class="n">param1</span><span class="p">):</span> <span class="k">assert</span> <span class="n">param1</span> <span class="o"><</span> <span class="mi">4</span> </pre></div> </div> <p>Now we add a test configuration like this:</p> <div class="highlight-python"><div class="highlight"><pre><span class="c"># content of conftest.py</span> <span class="k">def</span> <span class="nf">pytest_addoption</span><span class="p">(</span><span class="n">parser</span><span class="p">):</span> <span class="n">parser</span><span class="o">.</span><span class="n">addoption</span><span class="p">(</span><span class="s">"--all"</span><span class="p">,</span> <span class="n">action</span><span class="o">=</span><span class="s">"store_true"</span><span class="p">,</span> <span class="n">help</span><span class="o">=</span><span class="s">"run all combinations"</span><span class="p">)</span> <span class="k">def</span> <span class="nf">pytest_generate_tests</span><span class="p">(</span><span class="n">metafunc</span><span class="p">):</span> <span class="k">if</span> <span class="s">'param1'</span> <span class="ow">in</span> <span class="n">metafunc</span><span class="o">.</span><span class="n">fixturenames</span><span class="p">:</span> <span class="k">if</span> <span class="n">metafunc</span><span class="o">.</span><span class="n">config</span><span class="o">.</span><span class="n">option</span><span class="o">.</span><span class="n">all</span><span class="p">:</span> <span class="n">end</span> <span class="o">=</span> <span class="mi">5</span> <span class="k">else</span><span class="p">:</span> <span class="n">end</span> <span class="o">=</span> <span class="mi">2</span> <span class="n">metafunc</span><span class="o">.</span><span class="n">parametrize</span><span class="p">(</span><span class="s">"param1"</span><span class="p">,</span> <span class="nb">range</span><span class="p">(</span><span class="n">end</span><span class="p">))</span> </pre></div> </div> <p>This means that we only run 2 tests if we do not pass <tt class="docutils literal"><span class="pre">--all</span></tt>:</p> <div class="highlight-python"><pre>$ py.test -q test_compute.py ..</pre> </div> <p>We run only two computations, so we see two dots. let’s run the full monty:</p> <div class="highlight-python"><pre>$ py.test -q --all ....F ================================= FAILURES ================================= _____________________________ test_compute[4] ______________________________ param1 = 4 def test_compute(param1): > assert param1 < 4 E assert 4 < 4 test_compute.py:3: AssertionError</pre> </div> <p>As expected when running the full range of <tt class="docutils literal"><span class="pre">param1</span></tt> values we’ll get an error on the last one.</p> </div> <div class="section" id="a-quick-port-of-testscenarios"> <h2>A quick port of “testscenarios”<a class="headerlink" href="#a-quick-port-of-testscenarios" title="Permalink to this headline">¶</a></h2> <p>Here is a quick port to run tests configured with <a class="reference external" href="http://pypi.python.org/pypi/testscenarios/">test scenarios</a>, an add-on from Robert Collins for the standard unittest framework. We only have to work a bit to construct the correct arguments for pytest’s <a class="reference internal" href="../parametrize.html#_pytest.python.Metafunc.parametrize" title="_pytest.python.Metafunc.parametrize"><tt class="xref py py-func docutils literal"><span class="pre">Metafunc.parametrize()</span></tt></a>:</p> <div class="highlight-python"><div class="highlight"><pre><span class="c"># content of test_scenarios.py</span> <span class="k">def</span> <span class="nf">pytest_generate_tests</span><span class="p">(</span><span class="n">metafunc</span><span class="p">):</span> <span class="n">idlist</span> <span class="o">=</span> <span class="p">[]</span> <span class="n">argvalues</span> <span class="o">=</span> <span class="p">[]</span> <span class="k">for</span> <span class="n">scenario</span> <span class="ow">in</span> <span class="n">metafunc</span><span class="o">.</span><span class="n">cls</span><span class="o">.</span><span class="n">scenarios</span><span class="p">:</span> <span class="n">idlist</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">scenario</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span> <span class="n">items</span> <span class="o">=</span> <span class="n">scenario</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">items</span><span class="p">()</span> <span class="n">argnames</span> <span class="o">=</span> <span class="p">[</span><span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">items</span><span class="p">]</span> <span class="n">argvalues</span><span class="o">.</span><span class="n">append</span><span class="p">(([</span><span class="n">x</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">items</span><span class="p">]))</span> <span class="n">metafunc</span><span class="o">.</span><span class="n">parametrize</span><span class="p">(</span><span class="n">argnames</span><span class="p">,</span> <span class="n">argvalues</span><span class="p">,</span> <span class="n">ids</span><span class="o">=</span><span class="n">idlist</span><span class="p">,</span> <span class="n">scope</span><span class="o">=</span><span class="s">"class"</span><span class="p">)</span> <span class="n">scenario1</span> <span class="o">=</span> <span class="p">(</span><span class="s">'basic'</span><span class="p">,</span> <span class="p">{</span><span class="s">'attribute'</span><span class="p">:</span> <span class="s">'value'</span><span class="p">})</span> <span class="n">scenario2</span> <span class="o">=</span> <span class="p">(</span><span class="s">'advanced'</span><span class="p">,</span> <span class="p">{</span><span class="s">'attribute'</span><span class="p">:</span> <span class="s">'value2'</span><span class="p">})</span> <span class="k">class</span> <span class="nc">TestSampleWithScenarios</span><span class="p">:</span> <span class="n">scenarios</span> <span class="o">=</span> <span class="p">[</span><span class="n">scenario1</span><span class="p">,</span> <span class="n">scenario2</span><span class="p">]</span> <span class="k">def</span> <span class="nf">test_demo1</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">attribute</span><span class="p">):</span> <span class="k">assert</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">attribute</span><span class="p">,</span> <span class="nb">str</span><span class="p">)</span> <span class="k">def</span> <span class="nf">test_demo2</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">attribute</span><span class="p">):</span> <span class="k">assert</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">attribute</span><span class="p">,</span> <span class="nb">str</span><span class="p">)</span> </pre></div> </div> <p>this is a fully self-contained example which you can run with:</p> <div class="highlight-python"><pre>$ py.test test_scenarios.py =========================== test session starts ============================ platform linux2 -- Python 2.7.3 -- pytest-2.3.5 collected 4 items test_scenarios.py .... ========================= 4 passed in 0.01 seconds =========================</pre> </div> <p>If you just collect tests you’ll also nicely see ‘advanced’ and ‘basic’ as variants for the test function:</p> <div class="highlight-python"><pre>$ py.test --collectonly test_scenarios.py =========================== test session starts ============================ platform linux2 -- Python 2.7.3 -- pytest-2.3.5 collected 4 items <Module 'test_scenarios.py'> <Class 'TestSampleWithScenarios'> <Instance '()'> <Function 'test_demo1[basic]'> <Function 'test_demo2[basic]'> <Function 'test_demo1[advanced]'> <Function 'test_demo2[advanced]'> ============================= in 0.01 seconds =============================</pre> </div> <p>Note that we told <tt class="docutils literal"><span class="pre">metafunc.parametrize()</span></tt> that your scenario values should be considered class-scoped. With pytest-2.3 this leads to a resource-based ordering.</p> </div> <div class="section" id="deferring-the-setup-of-parametrized-resources"> <h2>Deferring the setup of parametrized resources<a class="headerlink" href="#deferring-the-setup-of-parametrized-resources" title="Permalink to this headline">¶</a></h2> <p>The parametrization of test functions happens at collection time. It is a good idea to setup expensive resources like DB connections or subprocess only when the actual test is run. Here is a simple example how you can achieve that, first the actual test requiring a <tt class="docutils literal"><span class="pre">db</span></tt> object:</p> <div class="highlight-python"><div class="highlight"><pre><span class="c"># content of test_backends.py</span> <span class="kn">import</span> <span class="nn">pytest</span> <span class="k">def</span> <span class="nf">test_db_initialized</span><span class="p">(</span><span class="n">db</span><span class="p">):</span> <span class="c"># a dummy test</span> <span class="k">if</span> <span class="n">db</span><span class="o">.</span><span class="n">__class__</span><span class="o">.</span><span class="n">__name__</span> <span class="o">==</span> <span class="s">"DB2"</span><span class="p">:</span> <span class="n">pytest</span><span class="o">.</span><span class="n">fail</span><span class="p">(</span><span class="s">"deliberately failing for demo purposes"</span><span class="p">)</span> </pre></div> </div> <p>We can now add a test configuration that generates two invocations of the <tt class="docutils literal"><span class="pre">test_db_initialized</span></tt> function and also implements a factory that creates a database object for the actual test invocations:</p> <div class="highlight-python"><div class="highlight"><pre><span class="c"># content of conftest.py</span> <span class="kn">import</span> <span class="nn">pytest</span> <span class="k">def</span> <span class="nf">pytest_generate_tests</span><span class="p">(</span><span class="n">metafunc</span><span class="p">):</span> <span class="k">if</span> <span class="s">'db'</span> <span class="ow">in</span> <span class="n">metafunc</span><span class="o">.</span><span class="n">fixturenames</span><span class="p">:</span> <span class="n">metafunc</span><span class="o">.</span><span class="n">parametrize</span><span class="p">(</span><span class="s">"db"</span><span class="p">,</span> <span class="p">[</span><span class="s">'d1'</span><span class="p">,</span> <span class="s">'d2'</span><span class="p">],</span> <span class="n">indirect</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span> <span class="k">class</span> <span class="nc">DB1</span><span class="p">:</span> <span class="s">"one database object"</span> <span class="k">class</span> <span class="nc">DB2</span><span class="p">:</span> <span class="s">"alternative database object"</span> <span class="nd">@pytest.fixture</span> <span class="k">def</span> <span class="nf">db</span><span class="p">(</span><span class="n">request</span><span class="p">):</span> <span class="k">if</span> <span class="n">request</span><span class="o">.</span><span class="n">param</span> <span class="o">==</span> <span class="s">"d1"</span><span class="p">:</span> <span class="k">return</span> <span class="n">DB1</span><span class="p">()</span> <span class="k">elif</span> <span class="n">request</span><span class="o">.</span><span class="n">param</span> <span class="o">==</span> <span class="s">"d2"</span><span class="p">:</span> <span class="k">return</span> <span class="n">DB2</span><span class="p">()</span> <span class="k">else</span><span class="p">:</span> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="s">"invalid internal test config"</span><span class="p">)</span> </pre></div> </div> <p>Let’s first see how it looks like at collection time:</p> <div class="highlight-python"><pre>$ py.test test_backends.py --collectonly =========================== test session starts ============================ platform linux2 -- Python 2.7.3 -- pytest-2.3.5 collected 2 items <Module 'test_backends.py'> <Function 'test_db_initialized[d1]'> <Function 'test_db_initialized[d2]'> ============================= in 0.00 seconds =============================</pre> </div> <p>And then when we run the test:</p> <div class="highlight-python"><pre>$ py.test -q test_backends.py .F ================================= FAILURES ================================= _________________________ test_db_initialized[d2] __________________________ db = <conftest.DB2 instance at 0x2038f80> def test_db_initialized(db): # a dummy test if db.__class__.__name__ == "DB2": > pytest.fail("deliberately failing for demo purposes") E Failed: deliberately failing for demo purposes test_backends.py:6: Failed</pre> </div> <p>The first invocation with <tt class="docutils literal"><span class="pre">db</span> <span class="pre">==</span> <span class="pre">"DB1"</span></tt> passed while the second with <tt class="docutils literal"><span class="pre">db</span> <span class="pre">==</span> <span class="pre">"DB2"</span></tt> failed. Our <tt class="docutils literal"><span class="pre">db</span></tt> fixture function has instantiated each of the DB values during the setup phase while the <tt class="docutils literal"><span class="pre">pytest_generate_tests</span></tt> generated two according calls to the <tt class="docutils literal"><span class="pre">test_db_initialized</span></tt> during the collection phase.</p> </div> <div class="section" id="parametrizing-test-methods-through-per-class-configuration"> <h2>Parametrizing test methods through per-class configuration<a class="headerlink" href="#parametrizing-test-methods-through-per-class-configuration" title="Permalink to this headline">¶</a></h2> <p>Here is an example <tt class="docutils literal"><span class="pre">pytest_generate_function</span></tt> function implementing a parametrization scheme similar to Michael Foord’s <a class="reference external" href="http://code.google.com/p/unittest-ext/source/browse/trunk/params.py">unittest parameterizer</a> but in a lot less code:</p> <div class="highlight-python"><div class="highlight"><pre><span class="c"># content of ./test_parametrize.py</span> <span class="kn">import</span> <span class="nn">pytest</span> <span class="k">def</span> <span class="nf">pytest_generate_tests</span><span class="p">(</span><span class="n">metafunc</span><span class="p">):</span> <span class="c"># called once per each test function</span> <span class="n">funcarglist</span> <span class="o">=</span> <span class="n">metafunc</span><span class="o">.</span><span class="n">cls</span><span class="o">.</span><span class="n">params</span><span class="p">[</span><span class="n">metafunc</span><span class="o">.</span><span class="n">function</span><span class="o">.</span><span class="n">__name__</span><span class="p">]</span> <span class="n">argnames</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="n">funcarglist</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span> <span class="n">metafunc</span><span class="o">.</span><span class="n">parametrize</span><span class="p">(</span><span class="n">argnames</span><span class="p">,</span> <span class="p">[[</span><span class="n">funcargs</span><span class="p">[</span><span class="n">name</span><span class="p">]</span> <span class="k">for</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">argnames</span><span class="p">]</span> <span class="k">for</span> <span class="n">funcargs</span> <span class="ow">in</span> <span class="n">funcarglist</span><span class="p">])</span> <span class="k">class</span> <span class="nc">TestClass</span><span class="p">:</span> <span class="c"># a map specifying multiple argument sets for a test method</span> <span class="n">params</span> <span class="o">=</span> <span class="p">{</span> <span class="s">'test_equals'</span><span class="p">:</span> <span class="p">[</span><span class="nb">dict</span><span class="p">(</span><span class="n">a</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">b</span><span class="o">=</span><span class="mi">2</span><span class="p">),</span> <span class="nb">dict</span><span class="p">(</span><span class="n">a</span><span class="o">=</span><span class="mi">3</span><span class="p">,</span> <span class="n">b</span><span class="o">=</span><span class="mi">3</span><span class="p">),</span> <span class="p">],</span> <span class="s">'test_zerodivision'</span><span class="p">:</span> <span class="p">[</span><span class="nb">dict</span><span class="p">(</span><span class="n">a</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">b</span><span class="o">=</span><span class="mi">0</span><span class="p">),</span> <span class="p">],</span> <span class="p">}</span> <span class="k">def</span> <span class="nf">test_equals</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">):</span> <span class="k">assert</span> <span class="n">a</span> <span class="o">==</span> <span class="n">b</span> <span class="k">def</span> <span class="nf">test_zerodivision</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">):</span> <span class="n">pytest</span><span class="o">.</span><span class="n">raises</span><span class="p">(</span><span class="ne">ZeroDivisionError</span><span class="p">,</span> <span class="s">"a/b"</span><span class="p">)</span> </pre></div> </div> <p>Our test generator looks up a class-level definition which specifies which argument sets to use for each test function. Let’s run it:</p> <div class="highlight-python"><pre>$ py.test -q F.. ================================= FAILURES ================================= ________________________ TestClass.test_equals[1-2] ________________________ self = <test_parametrize.TestClass instance at 0x1338f80>, a = 1, b = 2 def test_equals(self, a, b): > assert a == b E assert 1 == 2 test_parametrize.py:18: AssertionError</pre> </div> </div> <div class="section" id="indirect-parametrization-with-multiple-fixtures"> <h2>Indirect parametrization with multiple fixtures<a class="headerlink" href="#indirect-parametrization-with-multiple-fixtures" title="Permalink to this headline">¶</a></h2> <p>Here is a stripped down real-life example of using parametrized testing for testing serialization of objects between different python interpreters. We define a <tt class="docutils literal"><span class="pre">test_basic_objects</span></tt> function which is to be run with different sets of arguments for its three arguments:</p> <ul class="simple"> <li><tt class="docutils literal"><span class="pre">python1</span></tt>: first python interpreter, run to pickle-dump an object to a file</li> <li><tt class="docutils literal"><span class="pre">python2</span></tt>: second interpreter, run to pickle-load an object from a file</li> <li><tt class="docutils literal"><span class="pre">obj</span></tt>: object to be dumped/loaded</li> </ul> <div class="highlight-python"><div class="highlight"><pre><span class="sd">"""</span> <span class="sd">module containing a parametrized tests testing cross-python</span> <span class="sd">serialization via the pickle module.</span> <span class="sd">"""</span> <span class="kn">import</span> <span class="nn">py</span><span class="o">,</span> <span class="nn">pytest</span> <span class="n">pythonlist</span> <span class="o">=</span> <span class="p">[</span><span class="s">'python2.4'</span><span class="p">,</span> <span class="s">'python2.5'</span><span class="p">,</span> <span class="s">'python2.6'</span><span class="p">,</span> <span class="s">'python2.7'</span><span class="p">,</span> <span class="s">'python2.8'</span><span class="p">]</span> <span class="nd">@pytest.fixture</span><span class="p">(</span><span class="n">params</span><span class="o">=</span><span class="n">pythonlist</span><span class="p">)</span> <span class="k">def</span> <span class="nf">python1</span><span class="p">(</span><span class="n">request</span><span class="p">,</span> <span class="n">tmpdir</span><span class="p">):</span> <span class="n">picklefile</span> <span class="o">=</span> <span class="n">tmpdir</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="s">"data.pickle"</span><span class="p">)</span> <span class="k">return</span> <span class="n">Python</span><span class="p">(</span><span class="n">request</span><span class="o">.</span><span class="n">param</span><span class="p">,</span> <span class="n">picklefile</span><span class="p">)</span> <span class="nd">@pytest.fixture</span><span class="p">(</span><span class="n">params</span><span class="o">=</span><span class="n">pythonlist</span><span class="p">)</span> <span class="k">def</span> <span class="nf">python2</span><span class="p">(</span><span class="n">request</span><span class="p">,</span> <span class="n">python1</span><span class="p">):</span> <span class="k">return</span> <span class="n">Python</span><span class="p">(</span><span class="n">request</span><span class="o">.</span><span class="n">param</span><span class="p">,</span> <span class="n">python1</span><span class="o">.</span><span class="n">picklefile</span><span class="p">)</span> <span class="k">class</span> <span class="nc">Python</span><span class="p">:</span> <span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">version</span><span class="p">,</span> <span class="n">picklefile</span><span class="p">):</span> <span class="bp">self</span><span class="o">.</span><span class="n">pythonpath</span> <span class="o">=</span> <span class="n">py</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">local</span><span class="o">.</span><span class="n">sysfind</span><span class="p">(</span><span class="n">version</span><span class="p">)</span> <span class="k">if</span> <span class="ow">not</span> <span class="bp">self</span><span class="o">.</span><span class="n">pythonpath</span><span class="p">:</span> <span class="n">py</span><span class="o">.</span><span class="n">test</span><span class="o">.</span><span class="n">skip</span><span class="p">(</span><span class="s">"</span><span class="si">%r</span><span class="s"> not found"</span> <span class="o">%</span><span class="p">(</span><span class="n">version</span><span class="p">,))</span> <span class="bp">self</span><span class="o">.</span><span class="n">picklefile</span> <span class="o">=</span> <span class="n">picklefile</span> <span class="k">def</span> <span class="nf">dumps</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">obj</span><span class="p">):</span> <span class="n">dumpfile</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">picklefile</span><span class="o">.</span><span class="n">dirpath</span><span class="p">(</span><span class="s">"dump.py"</span><span class="p">)</span> <span class="n">dumpfile</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="n">py</span><span class="o">.</span><span class="n">code</span><span class="o">.</span><span class="n">Source</span><span class="p">(</span><span class="s">"""</span> <span class="s"> import pickle</span> <span class="s"> f = open(</span><span class="si">%r</span><span class="s">, 'wb')</span> <span class="s"> s = pickle.dump(</span><span class="si">%r</span><span class="s">, f)</span> <span class="s"> f.close()</span> <span class="s"> """</span> <span class="o">%</span> <span class="p">(</span><span class="nb">str</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">picklefile</span><span class="p">),</span> <span class="n">obj</span><span class="p">)))</span> <span class="n">py</span><span class="o">.</span><span class="n">process</span><span class="o">.</span><span class="n">cmdexec</span><span class="p">(</span><span class="s">"</span><span class="si">%s</span><span class="s"> </span><span class="si">%s</span><span class="s">"</span> <span class="o">%</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">pythonpath</span><span class="p">,</span> <span class="n">dumpfile</span><span class="p">))</span> <span class="k">def</span> <span class="nf">load_and_is_true</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">expression</span><span class="p">):</span> <span class="n">loadfile</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">picklefile</span><span class="o">.</span><span class="n">dirpath</span><span class="p">(</span><span class="s">"load.py"</span><span class="p">)</span> <span class="n">loadfile</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="n">py</span><span class="o">.</span><span class="n">code</span><span class="o">.</span><span class="n">Source</span><span class="p">(</span><span class="s">"""</span> <span class="s"> import pickle</span> <span class="s"> f = open(</span><span class="si">%r</span><span class="s">, 'rb')</span> <span class="s"> obj = pickle.load(f)</span> <span class="s"> f.close()</span> <span class="s"> res = eval(</span><span class="si">%r</span><span class="s">)</span> <span class="s"> if not res:</span> <span class="s"> raise SystemExit(1)</span> <span class="s"> """</span> <span class="o">%</span> <span class="p">(</span><span class="nb">str</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">picklefile</span><span class="p">),</span> <span class="n">expression</span><span class="p">)))</span> <span class="k">print</span> <span class="p">(</span><span class="n">loadfile</span><span class="p">)</span> <span class="n">py</span><span class="o">.</span><span class="n">process</span><span class="o">.</span><span class="n">cmdexec</span><span class="p">(</span><span class="s">"</span><span class="si">%s</span><span class="s"> </span><span class="si">%s</span><span class="s">"</span> <span class="o">%</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">pythonpath</span><span class="p">,</span> <span class="n">loadfile</span><span class="p">))</span> <span class="nd">@pytest.mark.parametrize</span><span class="p">(</span><span class="s">"obj"</span><span class="p">,</span> <span class="p">[</span><span class="mi">42</span><span class="p">,</span> <span class="p">{},</span> <span class="p">{</span><span class="mi">1</span><span class="p">:</span><span class="mi">3</span><span class="p">},])</span> <span class="k">def</span> <span class="nf">test_basic_objects</span><span class="p">(</span><span class="n">python1</span><span class="p">,</span> <span class="n">python2</span><span class="p">,</span> <span class="n">obj</span><span class="p">):</span> <span class="n">python1</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="n">obj</span><span class="p">)</span> <span class="n">python2</span><span class="o">.</span><span class="n">load_and_is_true</span><span class="p">(</span><span class="s">"obj == </span><span class="si">%s</span><span class="s">"</span> <span class="o">%</span> <span class="n">obj</span><span class="p">)</span> </pre></div> </div> <p>Running it results in some skips if we don’t have all the python interpreters installed and otherwise runs all combinations (5 interpreters times 5 interpreters times 3 objects to serialize/deserialize):</p> <div class="highlight-python"><pre>. $ py.test -rs -q multipython.py ............sss............sss............sss............ssssssssssssssssss ========================= short test summary info ========================== SKIP [27] /home/hpk/p/pytest/doc/en/example/multipython.py:21: 'python2.8' not found</pre> </div> </div> <div class="section" id="indirect-parametrization-of-optional-implementations-imports"> <h2>Indirect parametrization of optional implementations/imports<a class="headerlink" href="#indirect-parametrization-of-optional-implementations-imports" title="Permalink to this headline">¶</a></h2> <p>If you want to compare the outcomes of several implementations of a given API, you can write test functions that receive the already imported implementations and get skipped in case the implementation is not importable/available. Let’s say we have a “base” implementation and the other (possibly optimized ones) need to provide similar results:</p> <div class="highlight-python"><div class="highlight"><pre><span class="c"># content of conftest.py</span> <span class="kn">import</span> <span class="nn">pytest</span> <span class="nd">@pytest.fixture</span><span class="p">(</span><span class="n">scope</span><span class="o">=</span><span class="s">"session"</span><span class="p">)</span> <span class="k">def</span> <span class="nf">basemod</span><span class="p">(</span><span class="n">request</span><span class="p">):</span> <span class="k">return</span> <span class="n">pytest</span><span class="o">.</span><span class="n">importorskip</span><span class="p">(</span><span class="s">"base"</span><span class="p">)</span> <span class="nd">@pytest.fixture</span><span class="p">(</span><span class="n">scope</span><span class="o">=</span><span class="s">"session"</span><span class="p">,</span> <span class="n">params</span><span class="o">=</span><span class="p">[</span><span class="s">"opt1"</span><span class="p">,</span> <span class="s">"opt2"</span><span class="p">])</span> <span class="k">def</span> <span class="nf">optmod</span><span class="p">(</span><span class="n">request</span><span class="p">):</span> <span class="k">return</span> <span class="n">pytest</span><span class="o">.</span><span class="n">importorskip</span><span class="p">(</span><span class="n">request</span><span class="o">.</span><span class="n">param</span><span class="p">)</span> </pre></div> </div> <p>And then a base implementation of a simple function:</p> <div class="highlight-python"><div class="highlight"><pre><span class="c"># content of base.py</span> <span class="k">def</span> <span class="nf">func1</span><span class="p">():</span> <span class="k">return</span> <span class="mi">1</span> </pre></div> </div> <p>And an optimized version:</p> <div class="highlight-python"><div class="highlight"><pre><span class="c"># content of opt1.py</span> <span class="k">def</span> <span class="nf">func1</span><span class="p">():</span> <span class="k">return</span> <span class="mf">1.0001</span> </pre></div> </div> <p>And finally a little test module:</p> <div class="highlight-python"><div class="highlight"><pre><span class="c"># content of test_module.py</span> <span class="k">def</span> <span class="nf">test_func1</span><span class="p">(</span><span class="n">basemod</span><span class="p">,</span> <span class="n">optmod</span><span class="p">):</span> <span class="k">assert</span> <span class="nb">round</span><span class="p">(</span><span class="n">basemod</span><span class="o">.</span><span class="n">func1</span><span class="p">(),</span> <span class="mi">3</span><span class="p">)</span> <span class="o">==</span> <span class="nb">round</span><span class="p">(</span><span class="n">optmod</span><span class="o">.</span><span class="n">func1</span><span class="p">(),</span> <span class="mi">3</span><span class="p">)</span> </pre></div> </div> <p>If you run this with reporting for skips enabled:</p> <div class="highlight-python"><pre>$ py.test -rs test_module.py =========================== test session starts ============================ platform linux2 -- Python 2.7.3 -- pytest-2.3.5 collected 2 items test_module.py .s ========================= short test summary info ========================== SKIP [1] /tmp/doc-exec-275/conftest.py:10: could not import 'opt2' =================== 1 passed, 1 skipped in 0.01 seconds ====================</pre> </div> <p>You’ll see that we don’t have a <tt class="docutils literal"><span class="pre">opt2</span></tt> module and thus the second test run of our <tt class="docutils literal"><span class="pre">test_func1</span></tt> was skipped. A few notes:</p> <ul class="simple"> <li>the fixture functions in the <tt class="docutils literal"><span class="pre">conftest.py</span></tt> file are “session-scoped” because we don’t need to import more than once</li> <li>if you have multiple test functions and a skipped import, you will see the <tt class="docutils literal"><span class="pre">[1]</span></tt> count increasing in the report</li> <li>you can put <a class="reference internal" href="../parametrize.html#pytest-mark-parametrize"><em>@pytest.mark.parametrize</em></a> style parametrization on the test functions to parametrize input/output values as well.</li> </ul> </div> </div> </div> </div> </div> <div class="clearer"></div> </div> <div class="related"> <h3>Navigation</h3> <ul> <li class="right" style="margin-right: 10px"> <a href="markers.html" title="Working with custom markers" >next</a></li> <li class="right" > <a href="simple.html" title="Basic patterns and examples" >previous</a> |</li> <li><a href="../contents.html">pytest-2.3.4.1</a> »</li> <li><a href="index.html" >Usages and Examples</a> »</li> <g:plusone></g:plusone> </ul> </div> <div class="footer"> © Copyright 2012, holger krekel. Created using <a href="http://sphinx.pocoo.org/">Sphinx</a> 1.1.3. </div> <script type="text/javascript"> var _gaq = _gaq || []; _gaq.push(['_setAccount', 'UA-7597274-13']); _gaq.push(['_trackPageview']); (function() { var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); })(); </script> <script type="text/javascript" src="https://apis.google.com/js/plusone.js"></script> </body> </html>