1$FreeBSD$ 2 3This directory is for regression test programs. 4 5A regression test program is one that will exercise a particular bit of the 6system to check that we have not reintroduced an old bug. 7 8Tests should be implemented in files with a .t extension. Each .t file 9can contain more than one test, and can be implemented in any scripting 10language -- /bin/sh, Perl... 11 12The test protocol is quite simple. At its most basic, each .t file should, 13when run, print a line in this format: 14 15 1..m 16 17where m is the number of tests that will be run. 18 19Each test should produce a single line of output. This line should start 20with one of 21 22 ok n 23 not ok n 24 25to indicate whether or not the test succeeded. 'n' is the test's number. 26Anything after this on the line (up to the first '#' if present) is 27considered to be the name of the test. Naming tests is optional, but 28encouraged. 29 30A test may be written which is conditional, and may need to be skipped. 31For example, the netinet tests require 'options INET' in the kernel. 32A test may be skipped by printing '# skip Reason for skipping' after the 33test name. For example, 34 35 ok 1 - netinet # skip 'options INET' not compiled in 36 37A test may be flagged as 'todo'. This indicates that you expect the test 38to fail (perhaps because the necessary functionality hasn't been written 39yet). 'todo' tests are expected to fail, so when they start working the 40test framework can alert you to this happy occurrence. Flag these tests 41with a '# TODO' comment after the test name 42 43 not ok 1 - infiniteloop # TODO write test for an infinite loop 44 45This is modelled on the protocol followed by the Test::Harness Perl 46module (and therefore much of the automated testing carried out by the 47Perl community). More documentation can be found at: 48 49 http://search.cpan.org/~petdance/Test-Harness-2.42/lib/Test/Harness.pm 50 51To run the tests and parse their output install the devel/p5-Test-Harness 52port. This includes the prove(1) command which is used to run the tests 53and collate the output. 54 55 prove geom_concat # run all the tests in geom_concat 56 prove -r lib # run all tests in lib/, and subdirectories 57 prove -r -v lib # as above, with verbose output 58 prove -r # run *all* the tests 59 60Tests that are for parts of the base system should go into a directory here 61which is the same as their path relative to src/, for example the uuencode(1) 62utility resides in src/usr.bin/uuencode so its regression test resides in 63src/tools/regression/usr.bin/uuencode. 64 65To avoid the pre-commit check program complaining about the lack of 66CVS keywords in test data files, use a .in suffix for input files and 67a .out suffix for output files. 68 69To execute individual regression tests for binaries that you are 70developing, add their directory in the path before running the tests. 71Example: 72cd /usr/src/tools/regression/usr.bin 73(PATH=/home/user/src/experimental/jot:$PATH ; make SUBDIR=jot) 74 75Please make a subdir per other regression test, and add a brief description to 76this file. 77 78acct Exercise the integer to float conversion used in acct(5) 79geom Some tests and an out-of-kernel simulator for the GEOM code 80nfsmmap Some tests to exercise some tricky cases in NFS and mmap 81p1003_1b Exercise 1003.1B scheduler 82pipe Pipe code regression test 83fsx General filesystem exerciser 84sysvmsg SysV IPC Message Queue Regression Utility 85sysvsem SysV IPC Semaphore Regression Utility 86sysvshm SysV IPC Shared Memory Regression Utility 87gaithrstress General threaded getaddrinfo(3) exerciser 88date Date(1) + format string regression test 89