1.\" 2.\" This file and its contents are supplied under the terms of the 3.\" Common Development and Distribution License ("CDDL"), version 1.0. 4.\" You may only use this file in accordance with the terms of version 5.\" 1.0 of the CDDL. 6.\" 7.\" A full copy of the text of the CDDL should have accompanied this 8.\" source. A copy of the CDDL is also available via the Internet at 9.\" http://www.illumos.org/license/CDDL. 10.\" 11.\" 12.\" Copyright (c) 2012 by Delphix. All rights reserved. 13.\" 14.Dd May 24, 2026 15.Dt RUN 1 16.Os 17.Sh NAME 18.Nm run 19.Nd find, execute, and log the results of tests 20.Sh SYNOPSIS 21.Nm 22.Op Fl dgq 23.Op Fl i Ar testdir 24.Op Fl o Ar outputdir 25.Op Fl p Ar script 26.Op Fl P Ar script 27.Op Fl T Ar tags 28.Op Fl t Ar seconds 29.Op Fl u Ar username 30.Op Fl x Ar username 31.Op Fl X Ar username 32.Ar pathname ... 33.Nm 34.Fl w Ar runfile 35.Op Fl gq 36.Op Fl i Ar testdir 37.Op Fl o Ar outputdir 38.Op Fl p Ar script 39.Op Fl P Ar script 40.Op Fl T Ar tags 41.Op Fl t Ar seconds 42.Op Fl u Ar username 43.Op Fl x Ar username 44.Op Fl X Ar username 45.Ar pathname ... 46.Nm 47.Fl c Ar runfile Ns Op , Ns Ar runfile Ns No ... 48.Op Fl dq 49.Nm 50.Fl l Ar logfile 51.Op Fl dgq 52.Op Fl i Ar testdir 53.Op Fl o Ar outputdir 54.Op Fl p Ar script 55.Op Fl P Ar script 56.Op Fl t Ar seconds 57.Op Fl u Ar username 58.Op Fl x Ar username 59.Op Fl X Ar username 60.Nm 61.Op Fl h 62.Sh DESCRIPTION 63The 64.Nm 65command has four basic modes of operation. 66With none of the 67.Fl c , 68.Fl l , 69or 70.Fl w 71options, 72.Nm 73processes the arguments provided on the command line, adding them to the list 74for this run. 75If a specified 76.Ar pathname 77is an executable file, it is added as a test. 78If a specified 79.Ar pathname 80is a directory, the behavior depends upon the 81.Fl g 82option. 83If 84.Fl g 85is specified, the directory is treated as a test group. 86See the section on 87.Sx Test Groups 88below. 89Without the 90.Fl g 91option, 92.Nm 93simply descends into the directory looking for executable files. 94The tests are then executed, and the results are logged. 95.Pp 96With the 97.Fl w 98option, 99.Nm 100finds tests in the manner described above. 101Rather than executing the tests and logging the results, the test 102configuration is stored in a 103.Ar runfile 104which can be used in future invocations, or edited to modify which tests are 105executed and which options are applied. 106Options included on the command line with 107.Fl w 108become defaults in the 109.Ar runfile . 110.Pp 111With the 112.Fl c 113option, 114.Nm 115parses one or more 116.Ar runfile Ns s, 117which can specify a series of tests and test groups to be executed. 118The tests are then executed, and the results are logged. 119.Pp 120With the 121.Fl l 122option, 123.Nm 124reads a log file produced by a previous invocation and re-executes only the 125tests that failed. 126This is useful for iterating on a failing subset without re-running the entire 127test suite. 128.Ss Test Groups 129A test group is comprised of a set of executable files, all of which exist in 130one directory. 131The options specified on the command line or in a 132.Ar runfile 133apply to individual tests in the group. 134The exception is options pertaining to pre and post scripts, which act on all 135tests as a group. 136Rather than running before and after each test, these scripts are run only 137once each at the start and end of the test group. 138.Ss Test Execution 139The specified tests run serially, and are typically assigned results according 140to exit values. 141Tests that exit zero and non-zero are marked 142.Qq PASS 143and 144.Qq FAIL 145respectively. 146When a pre script fails for a test group, only the post script is executed, 147and the remaining tests are marked 148.Qq SKIPPED . 149Any test that exceeds its 150.Ar timeout 151is terminated, and marked 152.Qq KILLED . 153.Pp 154By default, tests are executed with the credentials of the 155.Nm 156script. 157Executing tests with other credentials is done via 158.Xr sudo 8 , 159which must be configured to allow execution without prompting for a password. 160Environment variables from the calling shell are available to individual tests. 161During test execution, the working directory is changed to 162.Ar outputdir . 163.Ss Output Logging 164By default, 165.Nm 166will print one line on standard output at the conclusion of each test 167indicating the test name, result and elapsed time. 168Additionally, for each invocation of 169.Nm , 170a directory is created using the ISO 8601 date format. 171Within this directory is a file named 172.Pa log 173containing all the test output with timestamps, and a directory for each test. 174Within the test directories, there is one file each for standard output, 175standard error and merged output. 176The default location for the 177.Ar outputdir 178is 179.Pa /var/tmp/test_results . 180.Ss Runfiles 181The 182.Ar runfile 183is an ini style configuration file that describes a test run. 184The file has one section named 185.Qq DEFAULT , 186which contains configuration option names and their values in 187.Qq name = value 188format. 189The values in this section apply to all the subsequent sections, unless they 190are also specified there, in which case the default is overridden. 191The remaining section names are the absolute pathnames of files and 192directories, describing tests and test groups respectively. 193The legal option names are: 194.Bl -tag -width "post_user" 195.It Cm outputdir No = Ar pathname 196The name of the directory that holds test logs. 197.It Cm pre No = Ar script 198Run 199.Ar script 200prior to the test or test group. 201.It Cm pre_user No = Ar username 202Execute the pre script as 203.Ar username . 204.It Cm post No = Ar script 205Run 206.Ar script 207after the test or test group. 208.It Cm post_user No = Ar username 209Execute the post script as 210.Ar username . 211.It Cm quiet No = Cm True Ns | Ns Cm False 212If set to True, only the results summary is printed to standard out. 213.It Cm tests No = No [\(aq Ns Ar filename Ns No \(aq No [, ...]] 214Specify a list of 215.Ar filenames 216for this test group. 217Only the basename of the absolute path is required. 218This option is only valid for test groups, and each 219.Ar filename 220must be single quoted. 221.It Cm timeout No = Ar n 222A timeout value of 223.Ar n 224seconds. 225.It Cm user No = Ar username 226Execute the test or test group as 227.Ar username . 228.El 229.Sh OPTIONS 230The following options are available for the 231.Nm 232command. 233.Bl -tag -width Ds 234.It Fl c Ar runfile Ns Op , Ns Ar runfile No ... 235Specify one or more 236.Ar runfile Ns s 237.Pq comma-separated 238defining tests and configuration. 239.It Fl d 240Dry run mode. 241Execute no tests, but print a description of each test that would have been 242run. 243.It Fl g 244Create test groups from any directories found while searching for tests. 245.It Fl h 246Print a usage summary and exit. 247.It Fl i Ar testdir 248Specify the directory from which tests are found when no 249.Ar pathname 250arguments are given. 251.It Fl l Ar logfile 252Read 253.Ar logfile 254from a previous run and re-execute only the tests that failed. 255.It Fl o Ar outputdir 256Specify the directory in which to write test results. 257.It Fl p Ar script 258Run 259.Ar script 260prior to any test or test group. 261.It Fl P Ar script 262Run 263.Ar script 264after any test or test group. 265.It Fl q 266Print only the results summary to the standard output. 267.It Fl t Ar n 268Specify a timeout value of 269.Ar n 270seconds per test. 271.It Fl T Ar tags 272Execute only test groups whose tags match the comma-separated list of 273.Ar tags . 274.It Fl u Ar username 275Execute tests or test groups as 276.Ar username . 277.It Fl w Ar runfile 278Specify the name of the 279.Ar runfile 280to create. 281.It Fl x Ar username 282Execute the pre script as 283.Ar username . 284.It Fl X Ar username 285Execute the post script as 286.Ar username . 287.El 288.Sh EXIT STATUS 289.Bl -tag -width Ds 290.It 0 291All tests passed. 292.It 1 293One or more tests failed or an error occurred. 294.It 2 295One or more tests were killed due to exceeding the timeout. 296.El 297.Sh EXAMPLES 298.Bl -enum 299.It 300Running ad-hoc tests. 301.Pp 302This example demonstrates the simplest invocation of 303.Nm . 304.Bd -literal 305% run my-tests 306Test: /home/jkennedy/my-tests/test-01 [00:02] [PASS] 307Test: /home/jkennedy/my-tests/test-02 [00:04] [PASS] 308Test: /home/jkennedy/my-tests/test-03 [00:01] [PASS] 309 310Results Summary 311PASS 3 312 313Running Time: 00:00:07 314Percent passed: 100.0% 315Log directory: /var/tmp/test_results/20120923T180654 316.Ed 317.It 318Creating a 319.Ar runfile 320for future use. 321.Pp 322This example demonstrates creating a 323.Ar runfile 324with non default options. 325.Bd -literal 326% run -p setup -x root -g -w new-tests.run new-tests 327% cat new-tests.run 328[DEFAULT] 329pre = setup 330post_user = 331quiet = False 332user = 333timeout = 60 334post = 335pre_user = root 336outputdir = /var/tmp/test_results 337 338[/home/jkennedy/new-tests] 339tests = ['test-01', 'test-02', 'test-03'] 340.Ed 341.El 342.Sh SEE ALSO 343.Xr sudo 8 344