<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml"> <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <title>Usage and Invocations</title> <link rel="stylesheet" href="_static/sphinxdoc.css" type="text/css" /> <link rel="stylesheet" href="_static/pygments.css" type="text/css" /> <script type="text/javascript"> var DOCUMENTATION_OPTIONS = { URL_ROOT: '', VERSION: '2.3.4.1', COLLAPSE_INDEX: false, FILE_SUFFIX: '.html', HAS_SOURCE: true }; </script> <script type="text/javascript" src="_static/jquery.js"></script> <script type="text/javascript" src="_static/underscore.js"></script> <script type="text/javascript" src="_static/doctools.js"></script> <link rel="top" title="None" href="index.html" /> <link rel="up" title="Getting started basics" href="overview.html" /> <link rel="next" title="Good Integration Practises" href="goodpractises.html" /> <link rel="prev" title="Installation and Getting Started" href="getting-started.html" /> </head> <body> <div class="related"> <h3>Navigation</h3> <ul> <li class="right" style="margin-right: 10px"> <a href="goodpractises.html" title="Good Integration Practises" accesskey="N">next</a></li> <li class="right" > <a href="getting-started.html" title="Installation and Getting Started" accesskey="P">previous</a> |</li> <li><a href="contents.html">pytest-2.3.4.1</a> »</li> <li><a href="overview.html" accesskey="U">Getting started basics</a> »</li> <g:plusone></g:plusone> </ul> </div> <div class="sphinxsidebar"> <div class="sphinxsidebarwrapper"> <div id="searchbox" style="display: none"> <form class="search" action="search.html" method="get"> <input type="text" name="q" size="18" /> <input type="submit" value="Search" /> <input type="hidden" name="check_keywords" value="yes" /> <input type="hidden" name="area" value="default" /> </form> </div> <script type="text/javascript">$('#searchbox').show(0);</script> <h3>quicklinks</h3> <div style="text-align: left; font-size: 100%; vertical-align: middle;"> <table> <tr> <td> <a href="index.html">home</a> </td><td> <a href="contents.html">TOC/contents</a> </td></tr><tr><td> <a href="getting-started.html">install</a> </td><td> <a href="changelog.html">changelog</a> </td></tr><tr><td> <a href="example/index.html">examples</a> </td><td> <a href="customize.html">customize</a> </td></tr><tr><td> <a href="https://bitbucket.org/hpk42/pytest/issues?status=new&status=open">issues[bb]</a> </td><td> <a href="contact.html">contact</a> </td></tr></table> </div> <h3><a href="contents.html">Table Of Contents</a></h3> <ul> <li><a class="reference internal" href="#">Usage and Invocations</a><ul> <li><a class="reference internal" href="#calling-pytest-through-python-m-pytest">Calling pytest through <tt class="docutils literal"><span class="pre">python</span> <span class="pre">-m</span> <span class="pre">pytest</span></tt></a></li> <li><a class="reference internal" href="#getting-help-on-version-option-names-environment-variables">Getting help on version, option names, environment variables</a></li> <li><a class="reference internal" href="#stopping-after-the-first-or-n-failures">Stopping after the first (or N) failures</a></li> <li><a class="reference internal" href="#specifying-tests-selecting-tests">Specifying tests / selecting tests</a></li> <li><a class="reference internal" href="#modifying-python-traceback-printing">Modifying Python traceback printing</a></li> <li><a class="reference internal" href="#dropping-to-pdb-python-debugger-on-failures">Dropping to PDB (Python Debugger) on failures</a></li> <li><a class="reference internal" href="#setting-a-breakpoint-aka-set-trace">Setting a breakpoint / aka <tt class="docutils literal"><span class="pre">set_trace()</span></tt></a></li> <li><a class="reference internal" href="#profiling-test-execution-duration">Profiling test execution duration</a></li> <li><a class="reference internal" href="#creating-junitxml-format-files">Creating JUnitXML format files</a></li> <li><a class="reference internal" href="#creating-resultlog-format-files">Creating resultlog format files</a></li> <li><a class="reference internal" href="#sending-test-report-to-online-pastebin-service">Sending test report to online pastebin service</a></li> <li><a class="reference internal" href="#calling-pytest-from-python-code">Calling pytest from Python code</a></li> </ul> </li> </ul> <h4>Previous topic</h4> <p class="topless"><a href="getting-started.html" title="previous chapter">Installation and Getting Started</a></p> <h4>Next topic</h4> <p class="topless"><a href="goodpractises.html" title="next chapter">Good Integration Practises</a></p> </div> </div> <div class="document"> <div class="documentwrapper"> <div class="bodywrapper"> <div class="body"> <div class="section" id="usage-and-invocations"> <span id="usage"></span><h1>Usage and Invocations<a class="headerlink" href="#usage-and-invocations" title="Permalink to this headline">¶</a></h1> <div class="section" id="calling-pytest-through-python-m-pytest"> <span id="cmdline"></span><h2>Calling pytest through <tt class="docutils literal"><span class="pre">python</span> <span class="pre">-m</span> <span class="pre">pytest</span></tt><a class="headerlink" href="#calling-pytest-through-python-m-pytest" title="Permalink to this headline">¶</a></h2> <p class="versionadded"> <span class="versionmodified">New in version 2.0.</span></p> <p>If you use Python-2.5 or later you can invoke testing through the Python interpreter from the command line:</p> <div class="highlight-python"><pre>python -m pytest [...]</pre> </div> <p>This is equivalent to invoking the command line script <tt class="docutils literal"><span class="pre">py.test</span> <span class="pre">[...]</span></tt> directly.</p> </div> <div class="section" id="getting-help-on-version-option-names-environment-variables"> <h2>Getting help on version, option names, environment variables<a class="headerlink" href="#getting-help-on-version-option-names-environment-variables" title="Permalink to this headline">¶</a></h2> <div class="highlight-python"><div class="highlight"><pre><span class="n">py</span><span class="o">.</span><span class="n">test</span> <span class="o">--</span><span class="n">version</span> <span class="c"># shows where pytest was imported from</span> <span class="n">py</span><span class="o">.</span><span class="n">test</span> <span class="o">--</span><span class="n">fixtures</span> <span class="c"># show available builtin function arguments</span> <span class="n">py</span><span class="o">.</span><span class="n">test</span> <span class="o">-</span><span class="n">h</span> <span class="o">|</span> <span class="o">--</span><span class="n">help</span> <span class="c"># show help on command line and config file options</span> </pre></div> </div> </div> <div class="section" id="stopping-after-the-first-or-n-failures"> <h2>Stopping after the first (or N) failures<a class="headerlink" href="#stopping-after-the-first-or-n-failures" title="Permalink to this headline">¶</a></h2> <p>To stop the testing process after the first (N) failures:</p> <div class="highlight-python"><div class="highlight"><pre><span class="n">py</span><span class="o">.</span><span class="n">test</span> <span class="o">-</span><span class="n">x</span> <span class="c"># stop after first failure</span> <span class="n">py</span><span class="o">.</span><span class="n">test</span> <span class="o">--</span><span class="n">maxfail</span><span class="o">=</span><span class="mi">2</span> <span class="c"># stop after two failures</span> </pre></div> </div> </div> <div class="section" id="specifying-tests-selecting-tests"> <h2>Specifying tests / selecting tests<a class="headerlink" href="#specifying-tests-selecting-tests" title="Permalink to this headline">¶</a></h2> <p>Several test run options:</p> <div class="highlight-python"><pre>py.test test_mod.py # run tests in module py.test somepath # run all tests below path py.test -k string # only run tests whose names contain a string</pre> </div> <p>Import ‘pkg’ and use its filesystem location to find and run tests:</p> <div class="highlight-python"><pre>py.test --pyargs pkg # run all tests found below directory of pypkg</pre> </div> </div> <div class="section" id="modifying-python-traceback-printing"> <h2>Modifying Python traceback printing<a class="headerlink" href="#modifying-python-traceback-printing" title="Permalink to this headline">¶</a></h2> <p>Examples for modifying traceback printing:</p> <div class="highlight-python"><div class="highlight"><pre><span class="n">py</span><span class="o">.</span><span class="n">test</span> <span class="o">--</span><span class="n">showlocals</span> <span class="c"># show local variables in tracebacks</span> <span class="n">py</span><span class="o">.</span><span class="n">test</span> <span class="o">-</span><span class="n">l</span> <span class="c"># show local variables (shortcut)</span> <span class="n">py</span><span class="o">.</span><span class="n">test</span> <span class="o">--</span><span class="n">tb</span><span class="o">=</span><span class="nb">long</span> <span class="c"># the default informative traceback formatting</span> <span class="n">py</span><span class="o">.</span><span class="n">test</span> <span class="o">--</span><span class="n">tb</span><span class="o">=</span><span class="n">native</span> <span class="c"># the Python standard library formatting</span> <span class="n">py</span><span class="o">.</span><span class="n">test</span> <span class="o">--</span><span class="n">tb</span><span class="o">=</span><span class="n">short</span> <span class="c"># a shorter traceback format</span> <span class="n">py</span><span class="o">.</span><span class="n">test</span> <span class="o">--</span><span class="n">tb</span><span class="o">=</span><span class="n">line</span> <span class="c"># only one line per failure</span> </pre></div> </div> </div> <div class="section" id="dropping-to-pdb-python-debugger-on-failures"> <h2>Dropping to PDB (Python Debugger) on failures<a class="headerlink" href="#dropping-to-pdb-python-debugger-on-failures" title="Permalink to this headline">¶</a></h2> <p>Python comes with a builtin Python debugger called <a class="reference external" href="http://docs.python.org/library/pdb.html">PDB</a>. <tt class="docutils literal"><span class="pre">py.test</span></tt> allows one to drop into the PDB prompt via a command line option:</p> <div class="highlight-python"><div class="highlight"><pre><span class="n">py</span><span class="o">.</span><span class="n">test</span> <span class="o">--</span><span class="n">pdb</span> </pre></div> </div> <p>This will invoke the Python debugger on every failure. Often you might only want to do this for the first failing test to understand a certain failure situation:</p> <div class="highlight-python"><div class="highlight"><pre><span class="n">py</span><span class="o">.</span><span class="n">test</span> <span class="o">-</span><span class="n">x</span> <span class="o">--</span><span class="n">pdb</span> <span class="c"># drop to PDB on first failure, then end test session</span> <span class="n">py</span><span class="o">.</span><span class="n">test</span> <span class="o">--</span><span class="n">pdb</span> <span class="o">--</span><span class="n">maxfail</span><span class="o">=</span><span class="mi">3</span> <span class="c"># drop to PDB for the first three failures</span> </pre></div> </div> </div> <div class="section" id="setting-a-breakpoint-aka-set-trace"> <h2>Setting a breakpoint / aka <tt class="docutils literal"><span class="pre">set_trace()</span></tt><a class="headerlink" href="#setting-a-breakpoint-aka-set-trace" title="Permalink to this headline">¶</a></h2> <p>If you want to set a breakpoint and enter the <tt class="docutils literal"><span class="pre">pdb.set_trace()</span></tt> you can use a helper:</p> <div class="highlight-python"><div class="highlight"><pre><span class="kn">import</span> <span class="nn">pytest</span> <span class="k">def</span> <span class="nf">test_function</span><span class="p">():</span> <span class="o">...</span> <span class="n">pytest</span><span class="o">.</span><span class="n">set_trace</span><span class="p">()</span> <span class="c"># invoke PDB debugger and tracing</span> </pre></div> </div> <p>In previous versions you could only enter PDB tracing if you disabled capturing on the command line via <tt class="docutils literal"><span class="pre">py.test</span> <span class="pre">-s</span></tt>.</p> </div> <div class="section" id="profiling-test-execution-duration"> <span id="durations"></span><h2>Profiling test execution duration<a class="headerlink" href="#profiling-test-execution-duration" title="Permalink to this headline">¶</a></h2> <p>To get a list of the slowest 10 test durations:</p> <div class="highlight-python"><div class="highlight"><pre><span class="n">py</span><span class="o">.</span><span class="n">test</span> <span class="o">--</span><span class="n">durations</span><span class="o">=</span><span class="mi">10</span> </pre></div> </div> </div> <div class="section" id="creating-junitxml-format-files"> <h2>Creating JUnitXML format files<a class="headerlink" href="#creating-junitxml-format-files" title="Permalink to this headline">¶</a></h2> <p>To create result files which can be read by <a class="reference external" href="http://hudson-ci.org/">Hudson</a> or other Continuous integration servers, use this invocation:</p> <div class="highlight-python"><div class="highlight"><pre><span class="n">py</span><span class="o">.</span><span class="n">test</span> <span class="o">--</span><span class="n">junitxml</span><span class="o">=</span><span class="n">path</span> </pre></div> </div> <p>to create an XML file at <tt class="docutils literal"><span class="pre">path</span></tt>.</p> </div> <div class="section" id="creating-resultlog-format-files"> <h2>Creating resultlog format files<a class="headerlink" href="#creating-resultlog-format-files" title="Permalink to this headline">¶</a></h2> <p>To create plain-text machine-readable result files you can issue:</p> <div class="highlight-python"><div class="highlight"><pre><span class="n">py</span><span class="o">.</span><span class="n">test</span> <span class="o">--</span><span class="n">resultlog</span><span class="o">=</span><span class="n">path</span> </pre></div> </div> <p>and look at the content at the <tt class="docutils literal"><span class="pre">path</span></tt> location. Such files are used e.g. by the <a class="reference external" href="http://buildbot.pypy.org/summary">PyPy-test</a> web page to show test results over several revisions.</p> </div> <div class="section" id="sending-test-report-to-online-pastebin-service"> <h2>Sending test report to online pastebin service<a class="headerlink" href="#sending-test-report-to-online-pastebin-service" title="Permalink to this headline">¶</a></h2> <p><strong>Creating a URL for each test failure</strong>:</p> <div class="highlight-python"><div class="highlight"><pre><span class="n">py</span><span class="o">.</span><span class="n">test</span> <span class="o">--</span><span class="n">pastebin</span><span class="o">=</span><span class="n">failed</span> </pre></div> </div> <p>This will submit test run information to a remote Paste service and provide a URL for each failure. You may select tests as usual or add for example <tt class="docutils literal"><span class="pre">-x</span></tt> if you only want to send one particular failure.</p> <p><strong>Creating a URL for a whole test session log</strong>:</p> <div class="highlight-python"><div class="highlight"><pre><span class="n">py</span><span class="o">.</span><span class="n">test</span> <span class="o">--</span><span class="n">pastebin</span><span class="o">=</span><span class="nb">all</span> </pre></div> </div> <p>Currently only pasting to the <a class="reference external" href="http://bpaste.net">http://bpaste.net</a> service is implemented.</p> </div> <div class="section" id="calling-pytest-from-python-code"> <span id="pytest-main-usage"></span><h2>Calling pytest from Python code<a class="headerlink" href="#calling-pytest-from-python-code" title="Permalink to this headline">¶</a></h2> <p class="versionadded"> <span class="versionmodified">New in version 2.0.</span></p> <p>You can invoke <tt class="docutils literal"><span class="pre">py.test</span></tt> from Python code directly:</p> <div class="highlight-python"><div class="highlight"><pre><span class="n">pytest</span><span class="o">.</span><span class="n">main</span><span class="p">()</span> </pre></div> </div> <p>this acts as if you would call “py.test” from the command line. It will not raise <tt class="docutils literal"><span class="pre">SystemExit</span></tt> but return the exitcode instead. You can pass in options and arguments:</p> <div class="highlight-python"><div class="highlight"><pre><span class="n">pytest</span><span class="o">.</span><span class="n">main</span><span class="p">([</span><span class="s">'-x'</span><span class="p">,</span> <span class="s">'mytestdir'</span><span class="p">])</span> </pre></div> </div> <p>or pass in a string:</p> <div class="highlight-python"><div class="highlight"><pre><span class="n">pytest</span><span class="o">.</span><span class="n">main</span><span class="p">(</span><span class="s">"-x mytestdir"</span><span class="p">)</span> </pre></div> </div> <p>You can specify additional plugins to <tt class="docutils literal"><span class="pre">pytest.main</span></tt>:</p> <div class="highlight-python"><div class="highlight"><pre><span class="c"># content of myinvoke.py</span> <span class="kn">import</span> <span class="nn">pytest</span> <span class="k">class</span> <span class="nc">MyPlugin</span><span class="p">:</span> <span class="k">def</span> <span class="nf">pytest_sessionfinish</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span> <span class="k">print</span><span class="p">(</span><span class="s">"*** test run reporting finishing"</span><span class="p">)</span> <span class="n">pytest</span><span class="o">.</span><span class="n">main</span><span class="p">(</span><span class="s">"-qq"</span><span class="p">,</span> <span class="n">plugins</span><span class="o">=</span><span class="p">[</span><span class="n">MyPlugin</span><span class="p">()])</span> </pre></div> </div> <p>Running it will show that <tt class="docutils literal"><span class="pre">MyPlugin</span></tt> was added and its hook was invoked:</p> <div class="highlight-python"><pre>$ python myinvoke.py *** test run reporting finishing</pre> </div> </div> </div> </div> </div> </div> <div class="clearer"></div> </div> <div class="related"> <h3>Navigation</h3> <ul> <li class="right" style="margin-right: 10px"> <a href="goodpractises.html" title="Good Integration Practises" >next</a></li> <li class="right" > <a href="getting-started.html" title="Installation and Getting Started" >previous</a> |</li> <li><a href="contents.html">pytest-2.3.4.1</a> »</li> <li><a href="overview.html" >Getting started basics</a> »</li> <g:plusone></g:plusone> </ul> </div> <div class="footer"> © Copyright 2012, holger krekel. Created using <a href="http://sphinx.pocoo.org/">Sphinx</a> 1.1.3. </div> <script type="text/javascript"> var _gaq = _gaq || []; _gaq.push(['_setAccount', 'UA-7597274-13']); _gaq.push(['_trackPageview']); (function() { var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); })(); </script> <script type="text/javascript" src="https://apis.google.com/js/plusone.js"></script> </body> </html>