<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Parametrizing fixtures and test functions</title>
<link rel="stylesheet" href="_static/sphinxdoc.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '',
VERSION: '2.3.4.1',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
};
</script>
<script type="text/javascript" src="_static/jquery.js"></script>
<script type="text/javascript" src="_static/underscore.js"></script>
<script type="text/javascript" src="_static/doctools.js"></script>
<link rel="top" title="None" href="index.html" />
<link rel="up" title="py.test reference documentation" href="apiref.html" />
<link rel="next" title="classic xunit-style setup" href="xunit_setup.html" />
<link rel="prev" title="pytest fixtures: explicit, modular, scalable" href="fixture.html" />
</head>
<body>
<div class="related">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="xunit_setup.html" title="classic xunit-style setup"
accesskey="N">next</a></li>
<li class="right" >
<a href="fixture.html" title="pytest fixtures: explicit, modular, scalable"
accesskey="P">previous</a> |</li>
<li><a href="contents.html">pytest-2.3.4.1</a> »</li>
<li><a href="apiref.html" accesskey="U">py.test reference documentation</a> »</li>
<g:plusone></g:plusone>
</ul>
</div>
<div class="sphinxsidebar">
<div class="sphinxsidebarwrapper">
<div id="searchbox" style="display: none">
<form class="search" action="search.html" method="get">
<input type="text" name="q" size="18" />
<input type="submit" value="Search" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
<h3>quicklinks</h3>
<div style="text-align: left; font-size: 100%; vertical-align: middle;">
<table>
<tr>
<td>
<a href="index.html">home</a>
</td><td>
<a href="contents.html">TOC/contents</a>
</td></tr><tr><td>
<a href="getting-started.html">install</a>
</td><td>
<a href="changelog.html">changelog</a>
</td></tr><tr><td>
<a href="example/index.html">examples</a>
</td><td>
<a href="customize.html">customize</a>
</td></tr><tr><td>
<a href="https://bitbucket.org/hpk42/pytest/issues?status=new&status=open">issues[bb]</a>
</td><td>
<a href="contact.html">contact</a>
</td></tr></table>
</div>
<h3><a href="contents.html">Table Of Contents</a></h3>
<ul>
<li><a class="reference internal" href="#">Parametrizing fixtures and test functions</a><ul>
<li><a class="reference internal" href="#pytest-mark-parametrize-parametrizing-test-functions"><tt class="docutils literal"><span class="pre">@pytest.mark.parametrize</span></tt>: parametrizing test functions</a></li>
<li><a class="reference internal" href="#basic-pytest-generate-tests-example">Basic <tt class="docutils literal"><span class="pre">pytest_generate_tests</span></tt> example</a></li>
<li><a class="reference internal" href="#the-metafunc-object">The <strong>metafunc</strong> object</a></li>
</ul>
</li>
</ul>
<h4>Previous topic</h4>
<p class="topless"><a href="fixture.html"
title="previous chapter">pytest fixtures: explicit, modular, scalable</a></p>
<h4>Next topic</h4>
<p class="topless"><a href="xunit_setup.html"
title="next chapter">classic xunit-style setup</a></p>
</div>
</div>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body">
<div class="section" id="parametrizing-fixtures-and-test-functions">
<span id="parametrize-basics"></span><span id="parametrize"></span><span id="parametrized-test-functions"></span><span id="parametrizing-tests"></span><span id="test-generators"></span><h1>Parametrizing fixtures and test functions<a class="headerlink" href="#parametrizing-fixtures-and-test-functions" title="Permalink to this headline">¶</a></h1>
<p>pytest supports test parametrization in several well-integrated ways:</p>
<ul class="simple">
<li><tt class="xref py py-func docutils literal"><span class="pre">pytest.fixture()</span></tt> allows to define <a class="reference internal" href="fixture.html#fixture-parametrize"><em>parametrization
at the level of fixture functions</em></a>.</li>
</ul>
<ul class="simple">
<li><a class="reference internal" href="#pytest-mark-parametrize">@pytest.mark.parametrize</a> allows to define parametrization at the
function or class level, provides multiple argument/fixture sets
for a particular test function or class.</li>
<li><a class="reference internal" href="#pytest-generate-tests">pytest_generate_tests</a> enables implementing your own custom
dynamic parametrization scheme or extensions.</li>
</ul>
<div class="section" id="pytest-mark-parametrize-parametrizing-test-functions">
<span id="pytest-mark-parametrize"></span><span id="parametrizemark"></span><h2><tt class="docutils literal"><span class="pre">@pytest.mark.parametrize</span></tt>: parametrizing test functions<a class="headerlink" href="#pytest-mark-parametrize-parametrizing-test-functions" title="Permalink to this headline">¶</a></h2>
<p class="versionadded">
<span class="versionmodified">New in version 2.2.</span></p>
<p>The builtin <tt class="docutils literal"><span class="pre">pytest.mark.parametrize</span></tt> decorator enables
parametrization of arguments for a test function. Here is a typical example
of a test function that implements checking that a certain input leads
to an expected output:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="c"># content of test_expectation.py</span>
<span class="kn">import</span> <span class="nn">pytest</span>
<span class="nd">@pytest.mark.parametrize</span><span class="p">((</span><span class="s">"input"</span><span class="p">,</span> <span class="s">"expected"</span><span class="p">),</span> <span class="p">[</span>
<span class="p">(</span><span class="s">"3+5"</span><span class="p">,</span> <span class="mi">8</span><span class="p">),</span>
<span class="p">(</span><span class="s">"2+4"</span><span class="p">,</span> <span class="mi">6</span><span class="p">),</span>
<span class="p">(</span><span class="s">"6*9"</span><span class="p">,</span> <span class="mi">42</span><span class="p">),</span>
<span class="p">])</span>
<span class="k">def</span> <span class="nf">test_eval</span><span class="p">(</span><span class="nb">input</span><span class="p">,</span> <span class="n">expected</span><span class="p">):</span>
<span class="k">assert</span> <span class="nb">eval</span><span class="p">(</span><span class="nb">input</span><span class="p">)</span> <span class="o">==</span> <span class="n">expected</span>
</pre></div>
</div>
<p>Here, the <tt class="docutils literal"><span class="pre">@parametrize</span></tt> decorator defines three different argument
sets for the two <tt class="docutils literal"><span class="pre">(input,</span> <span class="pre">output)</span></tt> arguments of the <tt class="docutils literal"><span class="pre">test_eval</span></tt> function
which will thus run three times:</p>
<div class="highlight-python"><pre>$ py.test
=========================== test session starts ============================
platform linux2 -- Python 2.7.3 -- pytest-2.3.5
collected 3 items
test_expectation.py ..F
================================= FAILURES =================================
____________________________ test_eval[6*9-42] _____________________________
input = '6*9', expected = 42
@pytest.mark.parametrize(("input", "expected"), [
("3+5", 8),
("2+4", 6),
("6*9", 42),
])
def test_eval(input, expected):
> assert eval(input) == expected
E assert 54 == 42
E + where 54 = eval('6*9')
test_expectation.py:8: AssertionError
==================== 1 failed, 2 passed in 0.01 seconds ====================</pre>
</div>
<p>As expected only one pair of input/output values fails the simple test function.
And as usual with test function arguments, you can see the <tt class="docutils literal"><span class="pre">input</span></tt> and <tt class="docutils literal"><span class="pre">output</span></tt> values in the traceback.</p>
<p>Note that there ways how you can mark a class or a module,
see <a class="reference internal" href="mark.html#mark"><em>Marking test functions with attributes</em></a>.</p>
</div>
<div class="section" id="basic-pytest-generate-tests-example">
<span id="pytest-generate-tests"></span><h2>Basic <tt class="docutils literal"><span class="pre">pytest_generate_tests</span></tt> example<a class="headerlink" href="#basic-pytest-generate-tests-example" title="Permalink to this headline">¶</a></h2>
<p>Sometimes you may want to implement your own parametrization scheme
or implement some dynamism for determining the parameters or scope
of a fixture. For this, you can use the <tt class="docutils literal"><span class="pre">pytest_generate_tests</span></tt> hook
which is called when collecting a test function. Through the passed in
<cite>metafunc</cite> object you can inspect the requesting test context and, most
importantly, you can call <tt class="docutils literal"><span class="pre">metafunc.parametrize()</span></tt> to cause
parametrization.</p>
<p>For example, let’s say we want to run a test taking string inputs which
we want to set via a new py.test command line option. Let’s first write
a simple test accepting a <tt class="docutils literal"><span class="pre">stringinput</span></tt> fixture function argument:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="c"># content of test_strings.py</span>
<span class="k">def</span> <span class="nf">test_valid_string</span><span class="p">(</span><span class="n">stringinput</span><span class="p">):</span>
<span class="k">assert</span> <span class="n">stringinput</span><span class="o">.</span><span class="n">isalpha</span><span class="p">()</span>
</pre></div>
</div>
<p>Now we add a <tt class="docutils literal"><span class="pre">conftest.py</span></tt> file containing the addition of a
command line option and the parametrization of our test function:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="c"># content of conftest.py</span>
<span class="k">def</span> <span class="nf">pytest_addoption</span><span class="p">(</span><span class="n">parser</span><span class="p">):</span>
<span class="n">parser</span><span class="o">.</span><span class="n">addoption</span><span class="p">(</span><span class="s">"--stringinput"</span><span class="p">,</span> <span class="n">action</span><span class="o">=</span><span class="s">"append"</span><span class="p">,</span> <span class="n">default</span><span class="o">=</span><span class="p">[],</span>
<span class="n">help</span><span class="o">=</span><span class="s">"list of stringinputs to pass to test functions"</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">pytest_generate_tests</span><span class="p">(</span><span class="n">metafunc</span><span class="p">):</span>
<span class="k">if</span> <span class="s">'stringinput'</span> <span class="ow">in</span> <span class="n">metafunc</span><span class="o">.</span><span class="n">fixturenames</span><span class="p">:</span>
<span class="n">metafunc</span><span class="o">.</span><span class="n">parametrize</span><span class="p">(</span><span class="s">"stringinput"</span><span class="p">,</span>
<span class="n">metafunc</span><span class="o">.</span><span class="n">config</span><span class="o">.</span><span class="n">option</span><span class="o">.</span><span class="n">stringinput</span><span class="p">)</span>
</pre></div>
</div>
<p>If we now pass two stringinput values, our test will run twice:</p>
<div class="highlight-python"><pre>$ py.test -q --stringinput="hello" --stringinput="world" test_strings.py
..</pre>
</div>
<p>Let’s also run with a stringinput that will lead to a failing test:</p>
<div class="highlight-python"><pre>$ py.test -q --stringinput="!" test_strings.py
F
================================= FAILURES =================================
___________________________ test_valid_string[!] ___________________________
stringinput = '!'
def test_valid_string(stringinput):
> assert stringinput.isalpha()
E assert <built-in method isalpha of str object at 0x2ba729dab300>()
E + where <built-in method isalpha of str object at 0x2ba729dab300> = '!'.isalpha
test_strings.py:3: AssertionError</pre>
</div>
<p>As expected our test function fails.</p>
<p>If you don’t specify a stringinput it will be skipped because
<tt class="docutils literal"><span class="pre">metafunc.parametrize()</span></tt> will be called with an empty parameter
listlist:</p>
<div class="highlight-python"><pre>$ py.test -q -rs test_strings.py
s
========================= short test summary info ==========================
SKIP [1] /home/hpk/p/pytest/.tox/regen/local/lib/python2.7/site-packages/_pytest/python.py:974: got empty parameter set, function test_valid_string at /tmp/doc-exec-240/test_strings.py:1</pre>
</div>
<p>For further examples, you might want to look at <a class="reference internal" href="example/parametrize.html#paramexamples"><em>more
parametrization examples</em></a>.</p>
</div>
<div class="section" id="the-metafunc-object">
<span id="metafunc-object"></span><h2>The <strong>metafunc</strong> object<a class="headerlink" href="#the-metafunc-object" title="Permalink to this headline">¶</a></h2>
<p>metafunc objects are passed to the <tt class="docutils literal"><span class="pre">pytest_generate_tests</span></tt> hook.
They help to inspect a testfunction and to generate tests
according to test configuration or values specified
in the class or module where a test function is defined:</p>
<p><tt class="docutils literal"><span class="pre">metafunc.fixturenames</span></tt>: set of required function arguments for given function</p>
<p><tt class="docutils literal"><span class="pre">metafunc.function</span></tt>: underlying python test function</p>
<p><tt class="docutils literal"><span class="pre">metafunc.cls</span></tt>: class object where the test function is defined in or None.</p>
<p><tt class="docutils literal"><span class="pre">metafunc.module</span></tt>: the module object where the test function is defined in.</p>
<p><tt class="docutils literal"><span class="pre">metafunc.config</span></tt>: access to command line opts and general config</p>
<p><tt class="docutils literal"><span class="pre">metafunc.funcargnames</span></tt>: alias for <tt class="docutils literal"><span class="pre">fixturenames</span></tt>, for pre-2.3 compatibility</p>
<dl class="method">
<dt id="_pytest.python.Metafunc.parametrize">
<tt class="descclassname">Metafunc.</tt><tt class="descname">parametrize</tt><big>(</big><em>argnames</em>, <em>argvalues</em>, <em>indirect=False</em>, <em>ids=None</em>, <em>scope=None</em><big>)</big><a class="reference internal" href="_modules/_pytest/python.html#Metafunc.parametrize"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#_pytest.python.Metafunc.parametrize" title="Permalink to this definition">¶</a></dt>
<dd><p>Add new invocations to the underlying test function using the list
of argvalues for the given argnames. Parametrization is performed
during the collection phase. If you need to setup expensive resources
see about setting indirect=True to do it rather at test setup time.</p>
<table class="docutils field-list" frame="void" rules="none">
<col class="field-name" />
<col class="field-body" />
<tbody valign="top">
<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><ul class="first last simple">
<li><strong>argnames</strong> – an argument name or a list of argument names</li>
<li><strong>argvalues</strong> – The list of argvalues determines how often a test is invoked
with different argument values. If only one argname was specified argvalues
is a list of simple values. If N argnames were specified, argvalues must
be a list of N-tuples, where each tuple-element specifies a value for its
respective argname.</li>
<li><strong>indirect</strong> – if True each argvalue corresponding to an argname will
be passed as request.param to its respective argname fixture
function so that it can perform more expensive setups during the
setup phase of a test rather than at collection time.</li>
<li><strong>ids</strong> – list of string ids each corresponding to the argvalues so
that they are part of the test id. If no ids are provided they will
be generated automatically from the argvalues.</li>
<li><strong>scope</strong> – if specified it denotes the scope of the parameters.
The scope is used for grouping tests by parameter instances.
It will also override any fixture-function defined scope, allowing
to set a dynamic scope using test context or configuration.</li>
</ul>
</td>
</tr>
</tbody>
</table>
</dd></dl>
<dl class="method">
<dt id="_pytest.python.Metafunc.addcall">
<tt class="descclassname">Metafunc.</tt><tt class="descname">addcall</tt><big>(</big><em>funcargs=None</em>, <em>id=_notexists</em>, <em>param=_notexists</em><big>)</big><a class="reference internal" href="_modules/_pytest/python.html#Metafunc.addcall"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#_pytest.python.Metafunc.addcall" title="Permalink to this definition">¶</a></dt>
<dd><p>(deprecated, use parametrize) Add a new call to the underlying
test function during the collection phase of a test run. Note that
request.addcall() is called during the test collection phase prior and
independently to actual test execution. You should only use addcall()
if you need to specify multiple arguments of a test function.</p>
<table class="docutils field-list" frame="void" rules="none">
<col class="field-name" />
<col class="field-body" />
<tbody valign="top">
<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><ul class="first last simple">
<li><strong>funcargs</strong> – argument keyword dictionary used when invoking
the test function.</li>
<li><strong>id</strong> – used for reporting and identification purposes. If you
don’t supply an <cite>id</cite> an automatic unique id will be generated.</li>
<li><strong>param</strong> – a parameter which will be exposed to a later fixture function
invocation through the <tt class="docutils literal"><span class="pre">request.param</span></tt> attribute.</li>
</ul>
</td>
</tr>
</tbody>
</table>
</dd></dl>
</div>
</div>
</div>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="related">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="xunit_setup.html" title="classic xunit-style setup"
>next</a></li>
<li class="right" >
<a href="fixture.html" title="pytest fixtures: explicit, modular, scalable"
>previous</a> |</li>
<li><a href="contents.html">pytest-2.3.4.1</a> »</li>
<li><a href="apiref.html" >py.test reference documentation</a> »</li>
<g:plusone></g:plusone>
</ul>
</div>
<div class="footer">
© Copyright 2012, holger krekel.
Created using <a href="http://sphinx.pocoo.org/">Sphinx</a> 1.1.3.
</div>
<script type="text/javascript">
var _gaq = _gaq || [];
_gaq.push(['_setAccount', 'UA-7597274-13']);
_gaq.push(['_trackPageview']);
(function() {
var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
})();
</script>
<script type="text/javascript" src="https://apis.google.com/js/plusone.js"></script>
</body>
</html>
