<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>The writing and reporting of assertions in tests</title>
<link rel="stylesheet" href="_static/sphinxdoc.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '',
VERSION: '2.3.4.1',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
};
</script>
<script type="text/javascript" src="_static/jquery.js"></script>
<script type="text/javascript" src="_static/underscore.js"></script>
<script type="text/javascript" src="_static/doctools.js"></script>
<link rel="top" title="None" href="index.html" />
<link rel="up" title="py.test reference documentation" href="apiref.html" />
<link rel="next" title="pytest fixtures: explicit, modular, scalable" href="fixture.html" />
<link rel="prev" title="Basic test configuration" href="customize.html" />
</head>
<body>
<div class="related">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="fixture.html" title="pytest fixtures: explicit, modular, scalable"
accesskey="N">next</a></li>
<li class="right" >
<a href="customize.html" title="Basic test configuration"
accesskey="P">previous</a> |</li>
<li><a href="contents.html">pytest-2.3.4.1</a> »</li>
<li><a href="apiref.html" accesskey="U">py.test reference documentation</a> »</li>
<g:plusone></g:plusone>
</ul>
</div>
<div class="sphinxsidebar">
<div class="sphinxsidebarwrapper">
<div id="searchbox" style="display: none">
<form class="search" action="search.html" method="get">
<input type="text" name="q" size="18" />
<input type="submit" value="Search" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
<h3>quicklinks</h3>
<div style="text-align: left; font-size: 100%; vertical-align: middle;">
<table>
<tr>
<td>
<a href="index.html">home</a>
</td><td>
<a href="contents.html">TOC/contents</a>
</td></tr><tr><td>
<a href="getting-started.html">install</a>
</td><td>
<a href="changelog.html">changelog</a>
</td></tr><tr><td>
<a href="example/index.html">examples</a>
</td><td>
<a href="customize.html">customize</a>
</td></tr><tr><td>
<a href="https://bitbucket.org/hpk42/pytest/issues?status=new&status=open">issues[bb]</a>
</td><td>
<a href="contact.html">contact</a>
</td></tr></table>
</div>
<h3><a href="contents.html">Table Of Contents</a></h3>
<ul>
<li><a class="reference internal" href="#">The writing and reporting of assertions in tests</a><ul>
<li><a class="reference internal" href="#asserting-with-the-assert-statement">Asserting with the <tt class="docutils literal"><span class="pre">assert</span></tt> statement</a></li>
<li><a class="reference internal" href="#assertions-about-expected-exceptions">Assertions about expected exceptions</a></li>
<li><a class="reference internal" href="#making-use-of-context-sensitive-comparisons">Making use of context-sensitive comparisons</a></li>
<li><a class="reference internal" href="#defining-your-own-assertion-comparison">Defining your own assertion comparison</a></li>
<li><a class="reference internal" href="#advanced-assertion-introspection">Advanced assertion introspection</a></li>
</ul>
</li>
</ul>
<h4>Previous topic</h4>
<p class="topless"><a href="customize.html"
title="previous chapter">Basic test configuration</a></p>
<h4>Next topic</h4>
<p class="topless"><a href="fixture.html"
title="next chapter">pytest fixtures: explicit, modular, scalable</a></p>
</div>
</div>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body">
<div class="section" id="the-writing-and-reporting-of-assertions-in-tests">
<h1>The writing and reporting of assertions in tests<a class="headerlink" href="#the-writing-and-reporting-of-assertions-in-tests" title="Permalink to this headline">¶</a></h1>
<div class="section" id="asserting-with-the-assert-statement">
<span id="assert"></span><span id="assert-with-the-assert-statement"></span><span id="assertfeedback"></span><h2>Asserting with the <tt class="docutils literal"><span class="pre">assert</span></tt> statement<a class="headerlink" href="#asserting-with-the-assert-statement" title="Permalink to this headline">¶</a></h2>
<p><tt class="docutils literal"><span class="pre">py.test</span></tt> allows you to use the standard python <tt class="docutils literal"><span class="pre">assert</span></tt> for verifying
expectations and values in Python tests. For example, you can write the
following:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="c"># content of test_assert1.py</span>
<span class="k">def</span> <span class="nf">f</span><span class="p">():</span>
<span class="k">return</span> <span class="mi">3</span>
<span class="k">def</span> <span class="nf">test_function</span><span class="p">():</span>
<span class="k">assert</span> <span class="n">f</span><span class="p">()</span> <span class="o">==</span> <span class="mi">4</span>
</pre></div>
</div>
<p>to assert that your function returns a certain value. If this assertion fails
you will see the return value of the function call:</p>
<div class="highlight-python"><pre>$ py.test test_assert1.py
=========================== test session starts ============================
platform linux2 -- Python 2.7.3 -- pytest-2.3.5
collected 1 items
test_assert1.py F
================================= FAILURES =================================
______________________________ test_function _______________________________
def test_function():
> assert f() == 4
E assert 3 == 4
E + where 3 = f()
test_assert1.py:5: AssertionError
========================= 1 failed in 0.01 seconds =========================</pre>
</div>
<p>py.test has support for showing the values of the most common subexpressions
including calls, attributes, comparisons, and binary and unary
operators. (See <a class="reference internal" href="example/reportingdemo.html#tbreportdemo"><em>Demo of Python failure reports with py.test</em></a>). This allows you to use the
idiomatic python constructs without boilerplate code while not losing
introspection information.</p>
<p>However, if you specify a message with the assertion like this:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="k">assert</span> <span class="n">a</span> <span class="o">%</span> <span class="mi">2</span> <span class="o">==</span> <span class="mi">0</span><span class="p">,</span> <span class="s">"value was odd, should be even"</span>
</pre></div>
</div>
<p>then no assertion introspection takes places at all and the message
will be simply shown in the traceback.</p>
<p>See <a class="reference internal" href="#assert-details"><em>Advanced assertion introspection</em></a> for more information on assertion introspection.</p>
</div>
<div class="section" id="assertions-about-expected-exceptions">
<span id="assertraises"></span><h2>Assertions about expected exceptions<a class="headerlink" href="#assertions-about-expected-exceptions" title="Permalink to this headline">¶</a></h2>
<p>In order to write assertions about raised exceptions, you can use
<tt class="docutils literal"><span class="pre">pytest.raises</span></tt> as a context manager like this:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="kn">import</span> <span class="nn">pytest</span>
<span class="k">with</span> <span class="n">pytest</span><span class="o">.</span><span class="n">raises</span><span class="p">(</span><span class="ne">ZeroDivisionError</span><span class="p">):</span>
<span class="mi">1</span> <span class="o">/</span> <span class="mi">0</span>
</pre></div>
</div>
<p>and if you need to have access to the actual exception info you may use:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="k">with</span> <span class="n">pytest</span><span class="o">.</span><span class="n">raises</span><span class="p">(</span><span class="ne">RuntimeError</span><span class="p">)</span> <span class="k">as</span> <span class="n">excinfo</span><span class="p">:</span>
<span class="k">def</span> <span class="nf">f</span><span class="p">():</span>
<span class="n">f</span><span class="p">()</span>
<span class="n">f</span><span class="p">()</span>
<span class="c"># do checks related to excinfo.type, excinfo.value, excinfo.traceback</span>
</pre></div>
</div>
<p>If you want to write test code that works on Python 2.4 as well,
you may also use two other ways to test for an expected exception:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="n">pytest</span><span class="o">.</span><span class="n">raises</span><span class="p">(</span><span class="n">ExpectedException</span><span class="p">,</span> <span class="n">func</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
<span class="n">pytest</span><span class="o">.</span><span class="n">raises</span><span class="p">(</span><span class="n">ExpectedException</span><span class="p">,</span> <span class="s">"func(*args, **kwargs)"</span><span class="p">)</span>
</pre></div>
</div>
<p>both of which execute the specified function with args and kwargs and
asserts that the given <tt class="docutils literal"><span class="pre">ExpectedException</span></tt> is raised. The reporter will
provide you with helpful output in case of failures such as <em>no
exception</em> or <em>wrong exception</em>.</p>
</div>
<div class="section" id="making-use-of-context-sensitive-comparisons">
<span id="newreport"></span><h2>Making use of context-sensitive comparisons<a class="headerlink" href="#making-use-of-context-sensitive-comparisons" title="Permalink to this headline">¶</a></h2>
<p class="versionadded">
<span class="versionmodified">New in version 2.0.</span></p>
<p>py.test has rich support for providing context-sensitive information
when it encounters comparisons. For example:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="c"># content of test_assert2.py</span>
<span class="k">def</span> <span class="nf">test_set_comparison</span><span class="p">():</span>
<span class="n">set1</span> <span class="o">=</span> <span class="nb">set</span><span class="p">(</span><span class="s">"1308"</span><span class="p">)</span>
<span class="n">set2</span> <span class="o">=</span> <span class="nb">set</span><span class="p">(</span><span class="s">"8035"</span><span class="p">)</span>
<span class="k">assert</span> <span class="n">set1</span> <span class="o">==</span> <span class="n">set2</span>
</pre></div>
</div>
<p>if you run this module:</p>
<div class="highlight-python"><pre>$ py.test test_assert2.py
=========================== test session starts ============================
platform linux2 -- Python 2.7.3 -- pytest-2.3.5
collected 1 items
test_assert2.py F
================================= FAILURES =================================
___________________________ test_set_comparison ____________________________
def test_set_comparison():
set1 = set("1308")
set2 = set("8035")
> assert set1 == set2
E assert set(['0', '1', '3', '8']) == set(['0', '3', '5', '8'])
E Extra items in the left set:
E '1'
E Extra items in the right set:
E '5'
test_assert2.py:5: AssertionError
========================= 1 failed in 0.01 seconds =========================</pre>
</div>
<p>Special comparisons are done for a number of cases:</p>
<ul class="simple">
<li>comparing long strings: a context diff is shown</li>
<li>comparing long sequences: first failing indices</li>
<li>comparing dicts: different entries</li>
</ul>
<p>See the <a class="reference internal" href="example/reportingdemo.html#tbreportdemo"><em>reporting demo</em></a> for many more examples.</p>
</div>
<div class="section" id="defining-your-own-assertion-comparison">
<h2>Defining your own assertion comparison<a class="headerlink" href="#defining-your-own-assertion-comparison" title="Permalink to this headline">¶</a></h2>
<p>It is possible to add your own detailed explanations by implementing
the <tt class="docutils literal"><span class="pre">pytest_assertrepr_compare</span></tt> hook.</p>
<dl class="function">
<dt id="_pytest.hookspec.pytest_assertrepr_compare">
<tt class="descname">pytest_assertrepr_compare</tt><big>(</big><em>config</em>, <em>op</em>, <em>left</em>, <em>right</em><big>)</big><a class="reference internal" href="_modules/_pytest/hookspec.html#pytest_assertrepr_compare"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#_pytest.hookspec.pytest_assertrepr_compare" title="Permalink to this definition">¶</a></dt>
<dd><p>return explanation for comparisons in failing assert expressions.</p>
<p>Return None for no custom explanation, otherwise return a list
of strings. The strings will be joined by newlines but any newlines
<em>in</em> a string will be escaped. Note that all but the first line will
be indented sligthly, the intention is for the first line to be a summary.</p>
</dd></dl>
<p>As an example consider adding the following hook in a conftest.py which
provides an alternative explanation for <tt class="docutils literal"><span class="pre">Foo</span></tt> objects:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="c"># content of conftest.py</span>
<span class="kn">from</span> <span class="nn">test_foocompare</span> <span class="kn">import</span> <span class="n">Foo</span>
<span class="k">def</span> <span class="nf">pytest_assertrepr_compare</span><span class="p">(</span><span class="n">op</span><span class="p">,</span> <span class="n">left</span><span class="p">,</span> <span class="n">right</span><span class="p">):</span>
<span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">left</span><span class="p">,</span> <span class="n">Foo</span><span class="p">)</span> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">right</span><span class="p">,</span> <span class="n">Foo</span><span class="p">)</span> <span class="ow">and</span> <span class="n">op</span> <span class="o">==</span> <span class="s">"=="</span><span class="p">:</span>
<span class="k">return</span> <span class="p">[</span><span class="s">'Comparing Foo instances:'</span><span class="p">,</span>
<span class="s">' vals: </span><span class="si">%s</span><span class="s"> != </span><span class="si">%s</span><span class="s">'</span> <span class="o">%</span> <span class="p">(</span><span class="n">left</span><span class="o">.</span><span class="n">val</span><span class="p">,</span> <span class="n">right</span><span class="o">.</span><span class="n">val</span><span class="p">)]</span>
</pre></div>
</div>
<p>now, given this test module:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="c"># content of test_foocompare.py</span>
<span class="k">class</span> <span class="nc">Foo</span><span class="p">:</span>
<span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">val</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">val</span> <span class="o">=</span> <span class="n">val</span>
<span class="k">def</span> <span class="nf">test_compare</span><span class="p">():</span>
<span class="n">f1</span> <span class="o">=</span> <span class="n">Foo</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span>
<span class="n">f2</span> <span class="o">=</span> <span class="n">Foo</span><span class="p">(</span><span class="mi">2</span><span class="p">)</span>
<span class="k">assert</span> <span class="n">f1</span> <span class="o">==</span> <span class="n">f2</span>
</pre></div>
</div>
<p>you can run the test module and get the custom output defined in
the conftest file:</p>
<div class="highlight-python"><pre>$ py.test -q test_foocompare.py
F
================================= FAILURES =================================
_______________________________ test_compare _______________________________
def test_compare():
f1 = Foo(1)
f2 = Foo(2)
> assert f1 == f2
E assert Comparing Foo instances:
E vals: 1 != 2
test_foocompare.py:8: AssertionError</pre>
</div>
</div>
<div class="section" id="advanced-assertion-introspection">
<span id="assert-introspection"></span><span id="assert-details"></span><h2>Advanced assertion introspection<a class="headerlink" href="#advanced-assertion-introspection" title="Permalink to this headline">¶</a></h2>
<p class="versionadded">
<span class="versionmodified">New in version 2.1.</span></p>
<p>Reporting details about a failing assertion is achieved either by rewriting
assert statements before they are run or re-evaluating the assert expression and
recording the intermediate values. Which technique is used depends on the
location of the assert, py.test’s configuration, and Python version being used
to run py.test. Note that for assert statements with a manually provided
message, i.e. <tt class="docutils literal"><span class="pre">assert</span> <span class="pre">expr,</span> <span class="pre">message</span></tt>, no assertion introspection takes place
and the manually provided message will be rendered in tracebacks.</p>
<p>By default, if the Python version is greater than or equal to 2.6, py.test
rewrites assert statements in test modules. Rewritten assert statements put
introspection information into the assertion failure message. py.test only
rewrites test modules directly discovered by its test collection process, so
asserts in supporting modules which are not themselves test modules will not be
rewritten.</p>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">py.test rewrites test modules on import. It does this by using an import hook
to write a new pyc files. Most of the time this works transparently. However,
if you are messing with import yourself, the import hook may interfere. If
this is the case, simply use <tt class="docutils literal"><span class="pre">--assert=reinterp</span></tt> or
<tt class="docutils literal"><span class="pre">--assert=plain</span></tt>. Additionally, rewriting will fail silently if it cannot
write new pycs, i.e. in a read-only filesystem or a zipfile.</p>
</div>
<p>If an assert statement has not been rewritten or the Python version is less than
2.6, py.test falls back on assert reinterpretation. In assert reinterpretation,
py.test walks the frame of the function containing the assert statement to
discover sub-expression results of the failing assert statement. You can force
py.test to always use assertion reinterpretation by passing the
<tt class="docutils literal"><span class="pre">--assert=reinterp</span></tt> option.</p>
<p>Assert reinterpretation has a caveat not present with assert rewriting: If
evaluating the assert expression has side effects you may get a warning that the
intermediate values could not be determined safely. A common example of this
issue is an assertion which reads from a file:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="k">assert</span> <span class="n">f</span><span class="o">.</span><span class="n">read</span><span class="p">()</span> <span class="o">!=</span> <span class="s">'...'</span>
</pre></div>
</div>
<p>If this assertion fails then the re-evaluation will probably succeed!
This is because <tt class="docutils literal"><span class="pre">f.read()</span></tt> will return an empty string when it is
called the second time during the re-evaluation. However, it is
easy to rewrite the assertion and avoid any trouble:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="n">content</span> <span class="o">=</span> <span class="n">f</span><span class="o">.</span><span class="n">read</span><span class="p">()</span>
<span class="k">assert</span> <span class="n">content</span> <span class="o">!=</span> <span class="s">'...'</span>
</pre></div>
</div>
<p>All assert introspection can be turned off by passing <tt class="docutils literal"><span class="pre">--assert=plain</span></tt>.</p>
<p>For further information, Benjamin Peterson wrote up <a class="reference external" href="http://pybites.blogspot.com/2011/07/behind-scenes-of-pytests-new-assertion.html">Behind the scenes of py.test’s new assertion rewriting</a>.</p>
<p class="versionadded">
<span class="versionmodified">New in version 2.1: </span>Add assert rewriting as an alternate introspection technique.</p>
<p class="versionchanged">
<span class="versionmodified">Changed in version 2.1: </span>Introduce the <tt class="docutils literal"><span class="pre">--assert</span></tt> option. Deprecate <tt class="docutils literal"><span class="pre">--no-assert</span></tt> and
<tt class="docutils literal"><span class="pre">--nomagic</span></tt>.</p>
</div>
</div>
</div>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="related">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="fixture.html" title="pytest fixtures: explicit, modular, scalable"
>next</a></li>
<li class="right" >
<a href="customize.html" title="Basic test configuration"
>previous</a> |</li>
<li><a href="contents.html">pytest-2.3.4.1</a> »</li>
<li><a href="apiref.html" >py.test reference documentation</a> »</li>
<g:plusone></g:plusone>
</ul>
</div>
<div class="footer">
© Copyright 2012, holger krekel.
Created using <a href="http://sphinx.pocoo.org/">Sphinx</a> 1.1.3.
</div>
<script type="text/javascript">
var _gaq = _gaq || [];
_gaq.push(['_setAccount', 'UA-7597274-13']);
_gaq.push(['_trackPageview']);
(function() {
var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
})();
</script>
<script type="text/javascript" src="https://apis.google.com/js/plusone.js"></script>
</body>
</html>
