1 2This directory is for regression test programs. 3 4A regression test program is one that will exercise a particular bit of the 5system to check that we have not reintroduced an old bug. 6 7Tests should be implemented in files with a .t extension. Each .t file 8can contain more than one test, and can be implemented in any scripting 9language -- /bin/sh, Perl... 10 11The test protocol is quite simple. At its most basic, each .t file should, 12when run, print a line in this format: 13 14 1..m 15 16where m is the number of tests that will be run. 17 18Each test should produce a single line of output. This line should start 19with one of 20 21 ok n 22 not ok n 23 24to indicate whether or not the test succeeded. 'n' is the test's number. 25Anything after this on the line (up to the first '#' if present) is 26considered to be the name of the test. Naming tests is optional, but 27encouraged. 28 29A test may be written which is conditional, and may need to be skipped. 30For example, the netinet tests require 'options INET' in the kernel. 31A test may be skipped by printing '# skip Reason for skipping' after the 32test name. For example, 33 34 ok 1 - netinet # skip 'options INET' not compiled in 35 36A test may be flagged as 'todo'. This indicates that you expect the test 37to fail (perhaps because the necessary functionality hasn't been written 38yet). 'todo' tests are expected to fail, so when they start working the 39test framework can alert you to this happy occurrence. Flag these tests 40with a '# TODO' comment after the test name 41 42 not ok 1 - infiniteloop # TODO write test for an infinite loop 43 44This is modelled on the protocol followed by the Test::Harness Perl 45module (and therefore much of the automated testing carried out by the 46Perl community). More documentation can be found at: 47 48 http://search.cpan.org/~petdance/Test-Harness-2.42/lib/Test/Harness.pm 49 50To run the tests and parse their output install the devel/p5-Test-Harness 51port. This includes the prove(1) command which is used to run the tests 52and collate the output. 53 54 prove geom_concat # run all the tests in geom_concat 55 prove -r lib # run all tests in lib/, and subdirectories 56 prove -r -v lib # as above, with verbose output 57 prove -r # run *all* the tests 58 59Tests that are for parts of the base system should go into a directory here 60which is the same as their path relative to src/, for example the uuencode(1) 61utility resides in src/usr.bin/uuencode so its regression test resides in 62src/tools/regression/usr.bin/uuencode. 63 64To avoid the pre-commit check program complaining about the lack of 65CVS keywords in test data files, use a .in suffix for input files and 66a .out suffix for output files. 67 68To execute individual regression tests for binaries that you are 69developing, add their directory in the path before running the tests. 70Example: 71cd /usr/src/tools/regression/usr.bin 72(PATH=/home/user/src/experimental/jot:$PATH ; make SUBDIR=jot) 73 74Please make a subdir per other regression test, and add a brief description to 75this file. 76 77acct Exercise the integer to float conversion used in acct(5) 78geom Some tests and an out-of-kernel simulator for the GEOM code 79nfsmmap Some tests to exercise some tricky cases in NFS and mmap 80p1003_1b Exercise 1003.1B scheduler 81pipe Pipe code regression test 82fsx General filesystem exerciser 83sysvmsg SysV IPC Message Queue Regression Utility 84sysvsem SysV IPC Semaphore Regression Utility 85sysvshm SysV IPC Shared Memory Regression Utility 86gaithrstress General threaded getaddrinfo(3) exerciser 87date Date(1) + format string regression test 88