.. _paramexamples: ãã©ã¡ã¼ã¿ã¼ãã¹ã ================== .. Parametrizing tests ================================================= .. currentmodule:: _pytest.python py.test ã¯ãç°¡åã«ãã©ã¡ã¼ã¿ã¼ããã¹ãé¢æ°ã¸æ¸¡ãã¾ãããã©ã¡ã¼ã¿ã¼ãã¹ããè¡ãããã®çµã¿è¾¼ã¿ã®ä»çµã¿ã使ã£ããµã³ãã«ãç´¹ä»ãã¾ãã .. py.test allows to easily parametrize test functions. In the following we provide some examples using the builtin mechanisms. .. _parametrizemark: ã·ã³ãã«ãª "ãã³ã¬ã¼ã¿ã¼" ã«ãããã©ã¡ã¼ã¿ã¼ãã¹ã -------------------------------------------------- .. Simple "decorator" parametrization of a test function ---------------------------------------------------------------------------- .. versionadded:: 2.2 .. The builtin ``pytest.mark.parametrize`` decorator directly enables parametrization of arguments for a test function. Here is an example of a test function that wants to compare that processing some input results in expected output:: çµã¿è¾¼ã¿ã® ``pytest.mark.parametrize`` ãã³ã¬ã¼ã¿ã¼ã¯ãç´æ¥ããã¹ãé¢æ°ã®å¼æ°ã¸ãã©ã¡ã¼ã¿ã¼ã渡ãã¾ããå ¥åå¤ãå¦çãã¦ããã®çµæã¨ãã¦æå¾ ãããåºåå¤ãæ¯è¼ããããã¹ãé¢æ°ã®ãµã³ãã«ãç´¹ä»ãã¾ã:: # test_expectation.py ã®å 容 import pytest @pytest.mark.parametrize(("input", "expected"), [ ("3+5", 8), ("2+4", 6), ("6*9", 42), ]) def test_eval(input, expected): assert eval(input) == expected .. we parametrize two arguments of the test function so that the test function is called three times. Let's run it:: ãã¹ãé¢æ°ã3åå¼ã³åºããããã®ãã¹ãé¢æ°ã¸2ã¤ã®å¼æ°ããã©ã¡ã¼ã¿ã¼ã¨ãã¦æ¸¡ãã¾ããå®è¡ãã¦ã¿ã¾ããã:: $ py.test -q collecting ... collected 3 items ..F ================================= FAILURES ================================= ____________________________ test_eval[6*9-42] _____________________________ input = '6*9', expected = 42 @pytest.mark.parametrize(("input", "expected"), [ ("3+5", 8), ("2+4", 6), ("6*9", 42), ]) def test_eval(input, expected): > assert eval(input) == expected E assert 54 == 42 E + where 54 = eval('6*9') test_expectation.py:8: AssertionError 1 failed, 2 passed in 0.01 seconds .. As expected only one pair of input/output values fails the simple test function. æå¾ ããéããå ¥åå¤ï¼åºåå¤ã®çµã¿åããã®1ã¤ã ãããã®åç´ãªãã¹ãé¢æ°ã失æããã¾ãã .. Note that there are various ways how you can mark groups of functions, see :ref:`mark`. é¢æ°ã®ã°ã«ã¼ãããã¼ã¯ããæ¹æ³ã¯æ§ã ãªããæ¹ãããã®ã«æ³¨æãã¦ãã ããã詳細㯠:ref:`mark` ãåç §ãã¦ãã ããã .. Generating parameters combinations, depending on command line ---------------------------------------------------------------------------- ã³ãã³ãã©ã¤ã³ãããã©ã¡ã¼ã¿ã¼ã®çµã¿åãããä½æ ------------------------------------------------ .. regendoc:wipe .. Let's say we want to execute a test with different computation parameters and the parameter range shall be determined by a command line argument. Let's first write a simple (do-nothing) computation test:: å¥ã®ãã©ã¡ã¼ã¿ã¼ã§ãã¹ããå®è¡ãããã¨ãã«ããã®ãã©ã¡ã¼ã¿ã¼ã®ç¯å²ã¯ã³ãã³ãã©ã¤ã³å¼æ°ã«ãã£ã¦æ±ºã¾ããã®ã¨ãã¾ããããæåã®ç°¡å㪠(ä½ãããªã) ãã¹ããæ¸ãã¦ã¿ã¾ã:: # test_compute.py ã®å 容 def test_compute(param1): assert param1 < 4 .. Now we add a test configuration like this:: 次ã®ãããªãã¹ãè¨å®ã追å ãã¾ã:: # conftest.py ã®å 容 def pytest_addoption(parser): parser.addoption("--all", action="store_true", help="run all combinations") def pytest_generate_tests(metafunc): if 'param1' in metafunc.fixturenames: if metafunc.config.option.all: end = 5 else: end = 2 metafunc.parametrize("param1", range(end)) .. This means that we only run 2 tests if we do not pass ``--all``:: ãã㯠``--all`` ãæå®ããªãå ´åã2åã ããã¹ããå®è¡ãã¾ã:: $ py.test -q test_compute.py collecting ... collected 2 items .. 2 passed in 0.01 seconds .. We run only two computations, so we see two dots. let's run the full monty:: 2åã ããã¹ããå®è¡ããã®ã§ããããã2ã¤è¡¨ç¤ºããã¾ããã§ã¯ãå ¨ãã¹ããå®è¡ãã¦ã¿ã¾ããã:: $ py.test -q --all collecting ... collected 5 items ....F ================================= FAILURES ================================= _____________________________ test_compute[4] ______________________________ param1 = 4 def test_compute(param1): > assert param1 < 4 E assert 4 < 4 test_compute.py:3: AssertionError 1 failed, 4 passed in 0.02 seconds .. As expected when running the full range of ``param1`` values we'll get an error on the last one. æå¾ ããéã ``param1`` ã®å ¨ã¦ã®ç¯å²å¤ãå®è¡ããã¨ãæå¾ã®1ã¤ãã¨ã©ã¼ã«ãªãã¾ãã .. A quick port of "testscenarios" ------------------------------------ "testscenarios" ã®ææ©ãç§»è¡ ---------------------------- .. _`test scenarios`: http://bazaar.launchpad.net/~lifeless/testscenarios/trunk/annotate/head%3A/doc/example.py .. Here is a quick port to run tests configured with `test scenarios`_, an add-on from Robert Collins for the standard unittest framework. We only have to work a bit to construct the correct arguments for pytest's :py:func:`Metafunc.parametrize`:: Robert Collins ã«ããæ¨æºã©ã¤ãã©ãªã® unittest ãã¬ã¼ã ã¯ã¼ã¯ã®ã¢ããªã³ã§ãã `test scenarios`_ ã§è¨å®ããããã¹ããå®è¡ããããã«ææ©ã移è¡æ¹æ³ãç´¹ä»ãã¾ããpytest ã® :py:func:`Metafunc.parametrize` ã¸æ¸¡ãæ£ããå¼æ°ãä½æããããã«å°ãã ãã³ã¼ãã£ã³ã°ãå¿ è¦ã§ã:: # test_scenarios.py ã®å 容 def pytest_generate_tests(metafunc): idlist = [] argvalues = [] for scenario in metafunc.cls.scenarios: idlist.append(scenario[0]) items = scenario[1].items() argnames = [x[0] for x in items] argvalues.append(([x[1] for x in items])) metafunc.parametrize(argnames, argvalues, ids=idlist) scenario1 = ('basic', {'attribute': 'value'}) scenario2 = ('advanced', {'attribute': 'value2'}) class TestSampleWithScenarios: scenarios = [scenario1, scenario2] def test_demo(self, attribute): assert isinstance(attribute, str) .. this is a fully self-contained example which you can run with:: ããã¯ããå®è¡ã§ããå®å ¨ãªèªå·±å®çµåãµã³ãã«ã§ã:: $ py.test test_scenarios.py =========================== test session starts ============================ platform linux2 -- Python 2.7.1 -- pytest-2.2.4 collecting ... collected 2 items test_scenarios.py .. ========================= 2 passed in 0.01 seconds ========================= .. If you just collect tests you'll also nicely see 'advanced' and 'basic' as variants for the test function:: ãã ãã¹ãã (å®è¡ããã«) éããã ããªãããã¹ãé¢æ°ã®å¤æ°ã¨ã㦠'advanced' 㨠'basic' ããã¾ã表示ããã¾ã:: $ py.test --collectonly test_scenarios.py =========================== test session starts ============================ platform linux2 -- Python 2.7.1 -- pytest-2.2.4 collecting ... collected 2 items <Module 'test_scenarios.py'> <Class 'TestSampleWithScenarios'> <Instance '()'> <Function 'test_demo[basic]'> <Function 'test_demo[advanced]'> ============================= in 0.00 seconds ============================= .. Deferring the setup of parametrized resources --------------------------------------------------- ãã©ã¡ã¼ã¿ã¼åããããªã½ã¼ã¹ã®é 延ã»ããã¢ãã ---------------------------------------------- .. regendoc:wipe .. The parametrization of test functions happens at collection time. It is a good idea to setup expensive resources like DB connections or subprocess only when the actual test is run. Here is a simple example how you can achieve that, first the actual test requiring a ``db`` object:: ãã¹ãé¢æ°ã¸ã®ãã©ã¡ã¼ã¿ã¼æ¸¡ãã¯ã³ã¬ã¯ã·ã§ã³æã«çºçãã¾ããå®éã«ãã¹ããå®è¡ããã¨ãã®ã¿ãDB ã³ãã¯ã·ã§ã³ããµãããã»ã¹ã¨ãã£ãé«ä¾¡ãªãªã½ã¼ã¹ãã»ããã¢ããããã®ã¯è¯ãèãã§ãããããã£ããã¹ããè¡ãç°¡åãªãµã³ãã«ã次ã«ãªãã¾ããæåã®ãã¹ã㯠``db`` ãªãã¸ã§ã¯ããè¦æ±ãã¾ã:: # test_backends.py ã®å 容 import pytest def test_db_initialized(db): # ããã¼ãã¹ã if db.__class__.__name__ == "DB2": pytest.fail("deliberately failing for demo purposes") .. We can now add a test configuration that generates two invocations of the ``test_db_initialized`` function and also implements a factory that creates a database object for the actual test invocations:: ``test_db_initialized`` é¢æ°ã®2åå®è¡ããããã«ãã¹ãè¨å®ã追å ãã¾ããããã«å®éã®ãã¹ãå®è¡æã«ãã¼ã¿ãã¼ã¹ãªãã¸ã§ã¯ããä½æãããã¡ã¯ããªã¼é¢æ°ãå®è£ ãã¾ã:: # conftest.py ã®å 容 def pytest_generate_tests(metafunc): if 'db' in metafunc.fixturenames: metafunc.parametrize("db", ['d1', 'd2'], indirect=True) class DB1: "one database object" class DB2: "alternative database object" def pytest_funcarg__db(request): if request.param == "d1": return DB1() elif request.param == "d2": return DB2() else: raise ValueError("invalid internal test config") .. Let's first see how it looks like at collection time:: ã³ã¬ã¯ã·ã§ã³æã«å ã»ã©ã®è¨å®ãã©ããªãããæåã«è¦ã¦ã¿ã¾ããã:: $ py.test test_backends.py --collectonly =========================== test session starts ============================ platform linux2 -- Python 2.7.1 -- pytest-2.2.4 collecting ... collected 2 items <Module 'test_backends.py'> <Function 'test_db_initialized[d1]'> <Function 'test_db_initialized[d2]'> ============================= in 0.00 seconds ============================= .. And then when we run the test:: ãããããã¹ããå®è¡ãã¾ã:: $ py.test -q test_backends.py collecting ... collected 2 items .F ================================= FAILURES ================================= _________________________ test_db_initialized[d2] __________________________ db = <conftest.DB2 instance at 0x1d4eb00> def test_db_initialized(db): # ããã¼ãã¹ã if db.__class__.__name__ == "DB2": > pytest.fail("deliberately failing for demo purposes") E Failed: deliberately failing for demo purposes test_backends.py:6: Failed 1 failed, 1 passed in 0.01 seconds .. The first invocation with ``db == "DB1"`` passed while the second with ``db == "DB2"`` failed. Our ``pytest_funcarg__db`` factory has instantiated each of the DB values during the setup phase while the ``pytest_generate_tests`` generated two according calls to the ``test_db_initialized`` during the collection phase. æåã® ``db == "DB1"`` ã«ããå®è¡ãæåããã®ã«å¯¾ãã¦ã2çªç®ã® ``db == "DB2"`` ã¯å¤±æãã¾ããã ``pytest_funcarg__db`` ãã¡ã¯ããªã¼ã¯ãã»ããã¢ãããã§ã¼ãºã®ã¨ãã«ããããã® DB å¤ãã¤ã³ã¹ã¿ã³ã¹åãã¾ãããä¸æ¹ ``pytest_generate_tests`` ã¯ãã³ã¬ã¯ã·ã§ã³ãã§ã¼ãºã®ã¨ãã«2åã® ``test_db_initialized`` å¼ã³åºããçæãã¾ããã .. regendoc:wipe .. Parametrizing test methods through per-class configuration -------------------------------------------------------------- ã¯ã©ã¹è¨å®æ¯ã®ãã¹ãã¡ã½ããã®ãã©ã¡ã¼ã¿ã¼æ¸¡ã ---------------------------------------------- .. _`unittest parameterizer`: http://code.google.com/p/unittest-ext/source/browse/trunk/params.py .. Here is an example ``pytest_generate_function`` function implementing a parametrization scheme similar to Michael Foord's `unittest parameterizer`_ but in a lot less code:: Michael Foord ã® `unittest parameterizer`_ ã¨ããä¼¼ã¦ãã¾ãããããããããã£ã¨å°ãªãã³ã¼ãã§ãã©ã¡ã¼ã¿ã¼ã渡ãä»çµã¿ãå®è£ ãã ``pytest_generate_function`` é¢æ°ã®ãµã³ãã«ãããã¾ã:: # ./test_parametrize.py ã®å 容 import pytest def pytest_generate_tests(metafunc): # ããããã®ãã¹ãé¢æ°æ¯ã«1åå¼ã³åºããã funcarglist = metafunc.cls.params[metafunc.function.__name__] argnames = list(funcarglist[0]) metafunc.parametrize(argnames, [[funcargs[name] for name in argnames] for funcargs in funcarglist]) class TestClass: # ãã¹ãã¡ã½ããã®ããã«è¤æ°ã®å¼æ°ã»ãããæå®ãããã£ã¯ã·ã§ã㪠params = { 'test_equals': [dict(a=1, b=2), dict(a=3, b=3), ], 'test_zerodivision': [dict(a=1, b=0), ], } def test_equals(self, a, b): assert a == b def test_zerodivision(self, a, b): pytest.raises(ZeroDivisionError, "a/b") .. Our test generator looks up a class-level definition which specifies which argument sets to use for each test function. Let's run it:: ãã¹ãã¸ã§ãã¬ã¼ã¿ã¼ã¯ãããããã®ãã¹ãã¡ã½ããã¸ã©ã®å¼æ°ã»ããã渡ãããç¹å®ããã¯ã©ã¹ã¬ãã«ã®å®ç¾©ã調ã¹ã¾ããå®è¡ãã¦ã¿ã¾ããã:: $ py.test -q collecting ... collected 3 items F.. ================================= FAILURES ================================= ________________________ TestClass.test_equals[1-2] ________________________ self = <test_parametrize.TestClass instance at 0x10d2e18>, a = 1, b = 2 def test_equals(self, a, b): > assert a == b E assert 1 == 2 test_parametrize.py:18: AssertionError 1 failed, 2 passed in 0.01 seconds .. Indirect parametrization with multiple resources -------------------------------------------------------------- è¤æ°ãªã½ã¼ã¹ã§ã®éæ¥çãªãã©ã¡ã¼ã¿ã¼æ¸¡ã ---------------------------------------- .. Here is a stripped down real-life example of using parametrized testing for testing serialization, invoking different python interpreters. We define a ``test_basic_objects`` function which is to be run with different sets of arguments for its three arguments: å¥ã ã® Python ã¤ã³ã¿ã¼ããªã¿ã¼ã§å®è¡ããã·ãªã¢ã©ã¤ãºåãæ¤è¨¼ããã®ã«ãã©ã¡ã¼ã¿ã¼ãã¹ãã使ããå®éã®ä¸çã§ã®ãµã³ãã«ã解説ãã¾ãã次ã®3ã¤ã®å¼æ°ãå ¨çµã¿åããã§å®è¡ãã ``test_basic_objects`` é¢æ°ãå®ç¾©ãã¾ãã .. * ``python1``: first python interpreter, run to pickle-dump an object to a file * ``python2``: second interpreter, run to pickle-load an object from a file * ``obj``: object to be dumped/loaded * ``python1`` : 1çªç®ã® Python ã¤ã³ã¿ã¼ããªã¿ã¼ããªãã¸ã§ã¯ãããã¡ã¤ã«ã¸ pickle-dump ããããã«å®è¡ * ``python2`` : 2çªç®ã® Python ã¤ã³ã¿ã¼ããªã¿ã¼ããã¡ã¤ã«ãããªãã¸ã§ã¯ãã pickle-load ããããã«å®è¡ * ``obj`` : ãã³ããããèªã¿è¾¼ãããã®ãªãã¸ã§ã¯ã .. literalinclude:: multipython.py .. Running it results in some skips if we don't have all the python interpreters installed and otherwise runs all combinations (5 interpreters times 5 interpreters times 3 objects to serialize/deserialize):: ããå ¨ã¦ã® Python ã¤ã³ã¿ã¼ããªã¿ã¼ãã¤ã³ã¹ãã¼ã«ããã¦ããªãå ´åãå®è¡ãã¦ãã¹ãããããã¾ããã¤ã³ã¹ãã¼ã«æ¸ã¿ã®å ´åãå ¨ã¦ã®çµã¿åãããå®è¡ããã¾ã (5ã¤ã®ã¤ã³ã¿ã¼ããªã¿ã¼ * 5ã¤ã®ã¤ã³ã¿ã¼ããªã¿ã¼ * 3ã¤ã®ã·ãªã¢ã©ã¤ãºï¼ãã·ãªã¢ã©ã¤ãºãããªãã¸ã§ã¯ã):: . $ py.test -rs -q multipython.py collecting ... collected 75 items ............sss............sss............sss............ssssssssssssssssss ========================= short test summary info ========================== SKIP [27] /home/hpk/p/pytest/doc/example/multipython.py:36: 'python2.8' not found 48 passed, 27 skipped in 1.71 seconds