1*3a92d97fSJulio Merino 2*3a92d97fSJulio Merinodnl A library of routines for doing regression tests for userland utilities. 3*3a92d97fSJulio Merino 4*3a92d97fSJulio Merinodnl Start up. We initialise the exit status to 0 (no failure) and change 5*3a92d97fSJulio Merinodnl into the directory specified by our first argument, which is the 6*3a92d97fSJulio Merinodnl directory to run the tests inside. 7*3a92d97fSJulio Merinodefine(`REGRESSION_START', 8*3a92d97fSJulio MerinoTESTDIR=$1 9*3a92d97fSJulio Merinoif [ -z "$TESTDIR" ]; then 10*3a92d97fSJulio Merino TESTDIR=. 11*3a92d97fSJulio Merinofi 12*3a92d97fSJulio Merinocd $TESTDIR 13*3a92d97fSJulio Merino 14*3a92d97fSJulio MerinoSTATUS=0) 15*3a92d97fSJulio Merino 16*3a92d97fSJulio Merinodnl Check $? to see if we passed or failed. The first parameter is the test 17*3a92d97fSJulio Merinodnl which passed or failed. It may be nil. 18*3a92d97fSJulio Merinodefine(`REGRESSION_PASSFAIL', 19*3a92d97fSJulio Merinoif [ $? -eq 0 ]; then 20*3a92d97fSJulio Merino echo "ok - $1 # Test detected no regression. (in $TESTDIR)" 21*3a92d97fSJulio Merinoelse 22*3a92d97fSJulio Merino STATUS=$? 23*3a92d97fSJulio Merino echo "not ok - $1 # Test failed: regression detected. See above. (in $TESTDIR)" 24*3a92d97fSJulio Merinofi) 25*3a92d97fSJulio Merino 26*3a92d97fSJulio Merinodnl An actual test. The first parameter is the test name. The second is the 27*3a92d97fSJulio Merinodnl command/commands to execute for the actual test. Their exit status is 28*3a92d97fSJulio Merinodnl checked. It is assumed that the test will output to stdout, and that the 29*3a92d97fSJulio Merinodnl output to be used to check for regression will be in regress.TESTNAME.out. 30*3a92d97fSJulio Merinodefine(`REGRESSION_TEST', 31*3a92d97fSJulio Merino$2 | diff -u ${SRCDIR:-.}/regress.$1.out - 32*3a92d97fSJulio MerinoREGRESSION_PASSFAIL($1)) 33*3a92d97fSJulio Merino 34*3a92d97fSJulio Merinodnl A freeform regression test. Only exit status is checked. 35*3a92d97fSJulio Merinodefine(`REGRESSION_TEST_FREEFORM', 36*3a92d97fSJulio Merino$2 37*3a92d97fSJulio MerinoREGRESSION_PASSFAIL($1)) 38*3a92d97fSJulio Merino 39*3a92d97fSJulio Merinodnl A regression test like REGRESSION_TEST, except only regress.out is used 40*3a92d97fSJulio Merinodnl for checking output differences. The first argument is the command, the 41*3a92d97fSJulio Merinodnl second argument (which may be empty) is the test name. 42*3a92d97fSJulio Merinodefine(`REGRESSION_TEST_ONE', 43*3a92d97fSJulio Merino$1 | diff -u ${SRCDIR:-.}/regress.out - 44*3a92d97fSJulio MerinoREGRESSION_PASSFAIL($2)) 45*3a92d97fSJulio Merino 46*3a92d97fSJulio Merinodnl A fatal error. This will exit with the given status (first argument) and 47*3a92d97fSJulio Merinodnl print the message (second argument) prefixed with the string "FATAL :" to 48*3a92d97fSJulio Merinodnl the error stream. 49*3a92d97fSJulio Merinodefine(`REGRESSION_FATAL', 50*3a92d97fSJulio Merinoecho "Bail out! $2 (in $TESTDIR)" > /dev/stderr 51*3a92d97fSJulio Merinoexit $1) 52*3a92d97fSJulio Merino 53*3a92d97fSJulio Merinodnl Cleanup. Exit with the status code of the last failure. Should probably 54*3a92d97fSJulio Merinodnl be the number of failed tests, but hey presto, this is what it does. This 55*3a92d97fSJulio Merinodnl could also clean up potential droppings, if some forms of regression tests 56*3a92d97fSJulio Merinodnl end up using mktemp(1) or such. 57*3a92d97fSJulio Merinodefine(`REGRESSION_END', 58*3a92d97fSJulio Merinoexit $STATUS) 59