1# 2# This file and its contents are supplied under the terms of the 3# Common Development and Distribution License ("CDDL"), version 1.0. 4# You may only use this file in accordance with the terms of version 5# 1.0 of the CDDL. 6# 7# A full copy of the text of the CDDL should have accompanied this 8# source. A copy of the CDDL is also available via the Internet at 9# http://www.illumos.org/license/CDDL. 10# 11 12# 13# Copyright (c) 2013 by Delphix. All rights reserved. 14# 15 16illumos Testing README 17 181. A Brief History of usr/src/test 192. How to Run These Tests 203. How to Develop New Tests 214. Porting Tests from Other Frameworks 22 23-------------------------------------------------------------------------------- 24 251. A Brief History of usr/src/test 26 27The tests here come in two varieties - tests written from scratch, and tests 28that have been ported from the Solaris Test Collection. Not all of the STC 29tests have been ported, and a forked repository of those that have been made 30publicly available may currently be found here: 31 32 https://bitbucket.org/illumos/illumos-stc/ 33 34Regardless of origin, all of these tests are executed using the run(1) script 35described in the next section. 36 372. How to Run These Tests 38 39Currently, all the test suites under usr/src/test provide a wrapper script 40around run(1). These wrappers allow environment variables to be set up which 41may (for example) allow the script to specify which disks may be used by a test 42suite, and which must be preserved. Additionally, the wrappers allow options to 43run(1) to be passed through so that a user may specify a custom configuration 44file for a test suite. For specifics on the options available in the framework 45itself, please see the run(1) manpage. 46 473. How to Develop New Tests 48 49New tests should mimic the directory layout of existing tests to the degree 50possible. This includes the following directories: 51 52cmd - Any support binaries or scripts used by the tests in this package. 53doc - READMEs or other support documentation to be delivered with the package. 54runfiles - Configuration files that dictate how the tests are run. 55tests - The tests themselves (see below). 56 57The tests you create will be run, and given a PASS or FAIL status in accordance 58with the exit value returned by the test. A test may also be marked SKIPPED in 59the event that a prerequisite test is marked FAIL, or marked KILLED in the 60event the test times out. Note that there is no way to force a test to be 61marked SKIPPED; this is intentional. If a test must be skipped due to 62insufficient resources for example, then a wrapper script should be provided 63that chooses or creates an appropriate configuration file. The goal of every 64run is that every test is marked PASS. 65 664. Porting Tests from Other Frameworks 67 68STF (Solaris Test Framework) 69 70Porting tests from this framework is relatively straightforward. For the most 71part the tests can be arranged in the new directory structure and added to the 72configuration file. The template for a configuration file can easily be created 73using the -w option to run(1). There are a few other changes that may be 74required: 75 76Some STF tests consume the values of variables from the user's environment. 77These variables must be set before beginning the test run either manually, or 78via a wrapper script. 79 80Groups of tests in STF can automatically source .cfg files specified in the 81'stf_description' file that exists in an STF directory. Ported tests that 82require any of these variables must manually source the .cfg file instead. 83 84The configuration file of a newly ported test must specify the user the test 85should run as, along with the timeout value. In STF, these are also specified 86in the 'stf_description' file. 87