-*- emacs-wiki -*- Structure of a test Each test consists of a set of directories under testing/. There are directories for klips, pluto, packaging and libraries. Each directory has a list of tests to run is stored in a file called TESTLIST in that directory. e.g. testing/klips/TESTLIST. The TESTLIST This isn't actually a shell script. It just looks like one. (Tragically Hip: "She's not pretty, she just looks that way.") Some tools other than /bin/sh process it. Lines that start with # are comments. # test-kind directory-containing-test expectation [PR#] The first word provides the test type, detailed below. The second word is the name of the test to run. This the directory in which the test case is to be found.. The third word may be one of: blank/good the test is believed to function, report failure bad the test is known to fail, report unexpected success suspended the test should not be run The fourth word may be a number, which is a PR# if the test is failing. Test kinds The test types are: skiptest means run no test. ctltest means run a single system without input/output. klipstest means run a single system with input/output networks umlplutotest means run a pair of systems umlXhost run an arbitrary number of systems mkinsttest a test of the "make install" machinery. kernel_test_patch a test of the "make kernelpatch" machinery. Each test directory has a file in it called testparams.sh. This file sets a number of environment variables to define the parameters of the test. Common parameters TESTNAME the name of the test (repeated for checking purposes) TEST_TYPE the type of the test (repeat of type type above) TESTHOST the name of the UML machine to run for the test, typically "east" or "west" TEST_PURPOSE The purpose of the test is one of: goal The goal purpose is where a test is defined for code that is not yet finished. The test indicates when the goals have in fact been reached. regress This is a test to determine that a previously existing bug has been repaired. This test will initially be created to reproduce the bug in isolation, and then the bug will be fixed. exploit This is a set of packets/programs that causes a vulnerability to be exposed. It is a specific variation of the regress option. TEST_GOAL_ITEM in the case of a goal test, this is a reference to the requirements document TEST_PROB_REPORT in the case of regression test, this the problem report number from GNATS TEST_EXPLOIT_URL in the case of an exploit, this is a URL referencing the paper explaining the origin of the test and the origin of exploit software REF_CONSOLE_OUTPUT a file in the test directory that contains the sanitized console output against which to compare the output of the actual test. REF_CONSOLE_FIXUPS a list of scripts (found in klips/test/fixups) to apply to sanitize the console output of the machine under test. These are typically perl, awk or sed scripts that remove things in the kernel output that change each time the test is run and/or compiled. INIT_SCRIPT a file of commands that is fed into the virtual machine's console in single user mode prior to starting the tests. This file will usually set up any eroute's and SADB entries that are required for the test. Lines beginning with # are skipped. Blank lines are skipped. Otherwise, a shell prompted is waited for each time (consisting of \n#) and then the command is sent. Note that the prompt is waited for before the command and not after, so completion of the last command in the script is not required. This is often used to invoke a program to monitor the system, e.g. ipsec pf_key. RUN_SCRIPT a file of commands that is fed into the virtual machine's console in single user mode, before the packets are sent. On single machine tests, this script doesn't provide any more power than INIT_SCRIPT, but is implemented for consistency's sake. FINAL_SCRIPT a file of commands that is fed into the virtual machine's console in single user mode after the final packet is sent. Similar to INIT_SCRIPT, above. If not specified, then the single command "halt" is sent. If specified, then the script should end with a halt command to nicely shutdown the UML. CONSOLEDIFFDEBUG If set to "true" then the series of console fixups (see REF_CONSOLE_FIXUPS) will be output after it is constructed. (It should be set to "false", or unset otherwise) NETJIGDEBUG If set to "true" then the series of console fixups (see REF_CONSOLE_FIXUPS) will be output after it is constructed. (It should be set to "false", or unset otherwise) NETJIGTESTDEBUG If set to "netjig", then the results of talking to the uml_netjig will be printed to stderr during the test. In addition, the jig will be invoked with --debug, which causes it to log its process ID, and wait 60 seconds before continuing. This can be used if you are trying to debug the uml_netjig program itself. HOSTTESTDEBUG If set to "hosttest", then the results of taling to the consoles of the UMLs will be printed to stderr during the test. NETJIGWAITUSER If set to "waituser", then the scripts will wait forever for user input before they shut the tests down. Use this is if you are debugging through the kernel. PACKETRATE A number, in miliseconds (default is 500ms) at which packets will be replayed by the netjig. - [[KLIPSTestParameters][KLIPStest parameters]] - [[MKInstTestParameters][mkinsttest parameters]] - [[RPMBuildInstallParameters][rpm_build_install_test parameters]] - [[LibTestParameters][libtest parameters]] - [[UMLPlutoTestParameters][umlplutotest parameters]] - [[UMLXHostParameters][umlXhost parameters]] - [[KernelPatchTestParameters][kernel_patch_test paramaters]] - [[ModuleCompileParameters][module_compile paramaters]] - [[UnitTestParameters][Unit Test parameters]]