.. _plugins:
Working with plugins and conftest files
=============================================
py.test implements all aspects of configuration, collection, running and reporting by calling `well specified hooks`_. Virtually any Python module can be registered as a plugin. It can implement any number of hook functions (usually two or three) which all have a ``pytest_`` prefix, making hook functions easy to distinguish and find. There are three basic location types:
* `builtin plugins`_: loaded from py.test's internal ``_pytest`` directory.
* `external plugins`_: modules discovered through `setuptools entry points`_
* `conftest.py plugins`_: modules auto-discovered in test directories
.. _`pytest/plugin`: http://bitbucket.org/hpk42/pytest/src/tip/pytest/plugin/
.. _`conftest.py plugins`:
.. _`conftest.py`:
.. _`localplugin`:
.. _`conftest`:
conftest.py: local per-directory plugins
--------------------------------------------------------------
local ``conftest.py`` plugins contain directory-specific hook
implementations. Session and test running activities will
invoke all hooks defined in ``conftest.py`` files closer to the
root of the filesystem. Example: Assume the following layout
and content of files::
a/conftest.py:
def pytest_runtest_setup(item):
# called for running each test in 'a' directory
print ("setting up", item)
a/test_in_subdir.py:
def test_sub():
pass
test_flat.py:
def test_flat():
pass
Here is how you might run it::
py.test test_flat.py # will not show "setting up"
py.test a/test_sub.py # will show "setting up"
.. Note::
If you have ``conftest.py`` files which do not reside in a
python package directory (i.e. one containing an ``__init__.py``) then
"import conftest" can be ambiguous because there might be other
``conftest.py`` files as well on your PYTHONPATH or ``sys.path``.
It is thus good practise for projects to either put ``conftest.py``
under a package scope or to never import anything from a
conftest.py file.
.. _`external plugins`:
.. _`extplugins`:
Installing External Plugins / Searching
------------------------------------------------------
Installing a plugin happens through any usual Python installation
tool, for example::
pip install pytest-NAME
pip uninstall pytest-NAME
If a plugin is installed, py.test automatically finds and integrates it,
there is no need to activate it. Here is a initial list of known plugins:
.. _`django`: https://www.djangoproject.com/
* `pytest-django <http://pypi.python.org/pypi/pytest-django>`_: write tests
for `django`_ apps, using pytest integration.
* `pytest-twisted <http://pypi.python.org/pypi/pytest-twisted>`_: write tests
for `twisted <http://twistedmatrix.com>`_ apps, starting a reactor and
processing deferreds from test functions.
* `pytest-capturelog <http://pypi.python.org/pypi/pytest-capturelog>`_:
to capture and assert about messages from the logging module
* `pytest-xdist <http://pypi.python.org/pypi/pytest-xdist>`_:
to distribute tests to CPUs and remote hosts, to run in boxed
mode which allows to survive segmentation faults, to run in
looponfailing mode, automatically re-running failing tests
on file changes, see also :ref:`xdist`
* `pytest-timeout <http://pypi.python.org/pypi/pytest-timeout>`_:
to timeout tests based on function marks or global definitions.
* `pytest-cache <http://pypi.python.org/pypi/pytest-cache>`_:
to interactively re-run failing tests and help other plugins to
store test run information across invocations.
* `pytest-cov <http://pypi.python.org/pypi/pytest-cov>`_:
coverage reporting, compatible with distributed testing
* `pytest-pep8 <http://pypi.python.org/pypi/pytest-pep8>`_:
a ``--pep8`` option to enable PEP8 compliance checking.
* `oejskit <http://pypi.python.org/pypi/oejskit>`_:
a plugin to run javascript unittests in life browsers
You may discover more plugins through a `pytest- pypi.python.org search`_.
.. _`available installable plugins`:
.. _`pytest- pypi.python.org search`: http://pypi.python.org/pypi?%3Aaction=search&term=pytest-&submit=search
Writing a plugin by looking at examples
------------------------------------------------------
.. _`Distribute`: http://pypi.python.org/pypi/distribute
.. _`setuptools`: http://pypi.python.org/pypi/setuptools
If you want to write a plugin, there are many real-life examples
you can copy from:
* a custom collection example plugin: :ref:`yaml plugin`
* around 20 `builtin plugins`_ which provide py.test's own functionality
* many `external plugins`_ providing additional features
All of these plugins implement the documented `well specified hooks`_
to extend and add functionality.
.. _`setuptools entry points`:
Making your plugin installable by others
-----------------------------------------------
If you want to make your plugin externally available, you
may define a so-called entry point for your distribution so
that ``py.test`` finds your plugin module. Entry points are
a feature that is provided by `setuptools`_ or `Distribute`_.
py.test looks up the ``pytest11`` entrypoint to discover its
plugins and you can thus make your plugin available by definig
it in your setuptools/distribute-based setup-invocation:
.. sourcecode:: python
# sample ./setup.py file
from setuptools import setup
setup(
name="myproject",
packages = ['myproject']
# the following makes a plugin available to py.test
entry_points = {
'pytest11': [
'name_of_plugin = myproject.pluginmodule',
]
},
)
If a package is installed this way, py.test will load
``myproject.pluginmodule`` as a plugin which can define
`well specified hooks`_.
.. _`pluginorder`:
Plugin discovery order at tool startup
--------------------------------------------
py.test loads plugin modules at tool startup in the following way:
* by loading all builtin plugins
* by loading all plugins registered through `setuptools entry points`_.
* by pre-scanning the command line for the ``-p name`` option
and loading the specified plugin before actual command line parsing.
* by loading all :file:`conftest.py` files as inferred by the command line
invocation (test files and all of its *parent* directories).
Note that ``conftest.py`` files from *sub* directories are by default
not loaded at tool startup.
* by recursively loading all plugins specified by the
``pytest_plugins`` variable in ``conftest.py`` files
Requiring/Loading plugins in a test module or conftest file
-------------------------------------------------------------
You can require plugins in a test module or a conftest file like this::
pytest_plugins = "name1", "name2",
When the test module or conftest plugin is loaded the specified plugins
will be loaded as well. You can also use dotted path like this::
pytest_plugins = "myapp.testsupport.myplugin"
which will import the specified module as a py.test plugin.
Accessing another plugin by name
--------------------------------------------
If a plugin wants to collaborate with code from
another plugin it can obtain a reference through
the plugin manager like this:
.. sourcecode:: python
plugin = config.pluginmanager.getplugin("name_of_plugin")
If you want to look at the names of existing plugins, use
the ``--traceconfig`` option.
.. _`findpluginname`:
Finding out which plugins are active
----------------------------------------------------------------------------
If you want to find out which plugins are active in your
environment you can type::
py.test --traceconfig
and will get an extended test header which shows activated plugins
and their names. It will also print local plugins aka
:ref:`conftest.py <conftest>` files when they are loaded.
.. _`cmdunregister`:
Deactivating / unregistering a plugin by name
----------------------------------------------------------------------------
You can prevent plugins from loading or unregister them::
py.test -p no:NAME
This means that any subsequent try to activate/load the named
plugin will it already existing. See :ref:`findpluginname` for
how to obtain the name of a plugin.
.. _`builtin plugins`:
py.test default plugin reference
====================================
You can find the source code for the following plugins
in the `pytest repository <http://bitbucket.org/hpk42/pytest/>`_.
.. autosummary::
_pytest.assertion
_pytest.capture
_pytest.config
_pytest.doctest
_pytest.genscript
_pytest.helpconfig
_pytest.junitxml
_pytest.mark
_pytest.monkeypatch
_pytest.nose
_pytest.pastebin
_pytest.pdb
_pytest.pytester
_pytest.python
_pytest.recwarn
_pytest.resultlog
_pytest.runner
_pytest.main
_pytest.skipping
_pytest.terminal
_pytest.tmpdir
_pytest.unittest
.. _`well specified hooks`:
py.test hook reference
====================================
Hook specification and validation
-----------------------------------------
py.test calls hook functions to implement initialization, running,
test execution and reporting. When py.test loads a plugin it validates
that each hook function conforms to its respective hook specification.
Each hook function name and its argument names need to match a hook
specification. However, a hook function may accept *fewer* parameters
by simply not specifying them. If you mistype argument names or the
hook name itself you get an error showing the available arguments.
Initialization, command line and configuration hooks
--------------------------------------------------------------------
.. currentmodule:: _pytest.hookspec
.. autofunction:: pytest_cmdline_preparse
.. autofunction:: pytest_cmdline_parse
.. autofunction:: pytest_namespace
.. autofunction:: pytest_addoption
.. autofunction:: pytest_cmdline_main
.. autofunction:: pytest_configure
.. autofunction:: pytest_unconfigure
Generic "runtest" hooks
------------------------------
All all runtest related hooks receive a :py:class:`pytest.Item` object.
.. autofunction:: pytest_runtest_protocol
.. autofunction:: pytest_runtest_setup
.. autofunction:: pytest_runtest_call
.. autofunction:: pytest_runtest_teardown
.. autofunction:: pytest_runtest_makereport
For deeper understanding you may look at the default implementation of
these hooks in :py:mod:`_pytest.runner` and maybe also
in :py:mod:`_pytest.pdb` which interacts with :py:mod:`_pytest.capture`
and its input/output capturing in order to immediately drop
into interactive debugging when a test failure occurs.
The :py:mod:`_pytest.terminal` reported specifically uses
the reporting hook to print information about a test run.
Collection hooks
------------------------------
py.test calls the following hooks for collecting files and directories:
.. autofunction:: pytest_ignore_collect
.. autofunction:: pytest_collect_directory
.. autofunction:: pytest_collect_file
For influencing the collection of objects in Python modules
you can use the following hook:
.. autofunction:: pytest_pycollect_makeitem
.. autofunction:: pytest_generate_tests
Reporting hooks
------------------------------
Session related reporting hooks:
.. autofunction: pytest_collectstart
.. autofunction: pytest_itemcollected
.. autofunction: pytest_collectreport
.. autofunction: pytest_deselected
And here is the central hook for reporting about
test execution:
.. autofunction: pytest_runtest_logreport
Reference of objects involved in hooks
===========================================================
.. autoclass:: _pytest.config.Config()
:members:
.. autoclass:: _pytest.config.Parser()
:members:
.. autoclass:: _pytest.main.Node()
:members:
.. autoclass:: _pytest.main.Collector()
:members:
:show-inheritance:
.. autoclass:: _pytest.main.Item()
:members:
:show-inheritance:
.. autoclass:: _pytest.python.Module()
:members:
:show-inheritance:
.. autoclass:: _pytest.python.Class()
:members:
:show-inheritance:
.. autoclass:: _pytest.python.Function()
:members:
:show-inheritance:
.. autoclass:: _pytest.runner.CallInfo()
:members:
.. autoclass:: _pytest.runner.TestReport()
:members:
