1 2CURSES TESTFRAME 3---------------- 4 51. Introduction 6 7The curses library is a complex piece of software and, often, changes 8made to the library may introduce subtle bugs that are hidden by other 9actions so a visual check of the curses output may look correct in 10some circumstances and the bug only show itself after a certain 11sequence of actions. To assist with validating that changes made to 12the curses library have no undesired effects an automated test is 13needed to detect and highlight any changes in the curses application 14output stream. The programmer can then analyse the output changes and 15either correct a bug or update the automated test to accept the new 16output as valid. 17 182. Architecture 19 20The curses testframe consists of two separate programs connected by a 21number of pipes and a pseudo-tty (pty). The programs are called the 22director and the slave. The director reads a configuration file of 23tests to perform, passes these commands to the slave over a pipe and 24reads the pty for any output from the slave. Data from the slave is 25compared against expected output held in a file and any differences 26are highlighted to the tester. The slave is a curses application that 27is forked by the director on start up. It reads commands from the 28director over a pipe, these commands are calls to curses routines 29along with the parameters required for the call. The slave takes the 30parameters and uses them as arguments for the requested curses routine 31call. The return value from the curses routine is passed back to the 32director over another pipe, if the curses routine updates any passed 33by reference arguments then these are also passed back to the director 34for analysis. 35 363. Director 37 38The director has the following optional command line options: 39 40 -v enables verbose output to assist debugging 41 -s slave_path the director will execute slave_path as the slave 42 process. The default is ./slave 43 -t term Sets the TERM environment variable to term when 44 executing the slave. The default is atf 45 46There is one mandatory command line parameter, that is a file name 47that contains the test command file. The test command file holds the 48calls required to exercise a particular curses routine and validate 49both the return codes from the routines and the output from the 50slave. The test language has a small number of commands, they are: 51 52assign: 53 Assign a value to a variable. The syntax is: 54 55 assign var_name value 56 57 Where var_name is the name of the variable. Variable names are 58 an arbitrary sequence of alphanumeric characters, the variable 59 name must start with an alphabetic character. Value is the value 60 to be assigned. The value can either be a numeric or a string 61 type. Variables are created on first use and will be 62 overwritten on each subsequent use. 63 64call, call2, call3, call4: 65 All these are used to call curses routines, the only difference 66 between then is the number of expected return values. Call 67 expects one return value, call2 expects 2, call3 expects 3 and 68 call4 expects four. Any parameters that are passed by reference 69 and updated by the call are treated like returns. So, for 70 example, calling the function getyx() which has three 71 parameters, the window, a pointer to storage for y and a pointer 72 to storage for x would be called like this: 73 74 call3 OK 4 5 getyx $win1 75 76 Which calls getyx, the first (and possibly only) return is the 77 return status of the function call, in this case we expect "OK" 78 indicating that the call succeeded. The next two returns are 79 the values of y and x respectively, the parameter $win1 is a 80 variable that was assigned by a previous call. Any return can 81 be assigned to a variable by including the variable name in a 82 call return list. Variables are referenced in a call parameter 83 list by prefixing the name with a $ character. All returns are 84 validated against the expected values and an error raised if 85 there is a mismatch. The only exception to this is when the 86 return is assigned to a variable. Valid values for the returns 87 list are: 88 89 variable - assign the return to the given variable 90 name. 91 numeric - the value of the return must match the 92 number given. 93 string - an arbitrary sequence of characters 94 enclosed in double quotes. 95 ERR - expect an ERR return 96 OK - expect an OK return 97 NULL - expect a NULL pointer return 98 NON_NULL - expect a pointer that is not NULL valued 99 100 There is one special parameter that can be passed to a call, 101 that is the label STDSCR. This parameter will be substituted by 102 the value of stdscr when the function call is made. 103 104check: 105 Validate the value of a variable. This allows a variable to be 106 checked for an expected return after it has been assigned in a 107 previous call. The syntax is: 108 109 check var_name expected_result 110 111 Where var_name is a variable previously assigned and 112 expected_result is one of the valid return values listed in the 113 above call section. 114 115compare: 116 Compares the output stream from the slave against the contents 117 of a file that contains the expected 118 output. The syntax is: 119 120 compare filename 121 122 Where filename is the name of the file containing the expected 123 output. The file can either be an absolute path or relative 124 path. In the latter case the value of the environment variable 125 CHECK_PATH will be prepended to the argument to provide the path 126 to the file. The contents of this file will be compared byte by 127 byte against the output from the slave, any differences in the 128 output will be flagged. If the director is not in verbose mode 129 then the first mismatch in the byte stream will cause the 130 director to exit. 131 132comparend: 133 Performs the same function as the above compare except that 134 excess output from the slave is not discarded if there is more 135 data from the slave than there is in the check file. This 136 allows chaining of multiple check files. 137 138delay: 139 Defines an inter-character delay to be inserted between 140 characters being fed into the input of the slave. The syntax 141 is: 142 143 delay time 144 145 Where time is the amount of time to delay in milliseconds. 146 147include: 148 Include the contents of another test file, the parser will 149 suspend reading the current file and read commands from the 150 include file until the end of file of the include file is 151 reached at which point it will continue reading the original 152 file. Include files may be nested. The syntax is: 153 154 include filename 155 156 Where filename is the name of the file to include. If the 157 filename is not an absolute path then the contents of the 158 environment variable INCLUDE_PATH are prepended to the file 159 name. 160 161input: 162 Defines a string of characters that will be fed to the slave 163 when a call requires input. Any unused input will be discarded 164 after the call that required the input is called. The syntax 165 is: 166 167 input "string to pass" 168 169noinput: 170 Normally the director will error if an input function is called 171 without input being previously defined, this is to prevent input 172 functions causing the test to hang waiting for input that never 173 comes. If it is known that there is pending input for the slave 174 then the noinput keyword can be used to flag that the input 175 function has data available for it to read. The noinput command 176 only applies to the next function call then behaviour reverts to 177 the default. 178 179The testframe can define different types of strings, the type of string 180depends on the type of enclosing quotes. A null terminated string is 181indicated by enclosing double (") quotes. A byte string, one that is 182not null terminated and may contain the nul character within it is 183indicated by enclosing single (') quotes. A string of chtype 184character which are a combined attribute and character value is 185indicated by enclosing backticks (`), for this type of string pairs of 186bytes between the backticks are converted to an array of chtype, the 187first byte is the attribute and the second is the character. 188 189All strings defined will have a simple set of character substitutions 190performed on them when they are parsed. This allows the tester to 191embed some control characters into the string. Valid substitutions 192are: 193 194 \e escape 195 \n new line 196 \r carriage return 197 \t tab 198 \\ \ character 199 \nnn Where nnn is three octal digits, the character 200 represented by the octal number will be inserted into 201 the string. 202 203Any other invalid conversions will have the \ stripped and the 204subsequent characters inserted into the string. 205 206Integers may be specified by either a plain numeric (e.g. 12345) or by 207hexadecimal notation by prefixing the number with 0x (e.g. 0x3039). 208Internally, no distinction is made between the two formats and they 209can be freely intermixed. 210 211Integers and variables containing integers can have operations 212performed on them. Currently only bitwise ORing numbers together is 213supported. This can be done by separating a list of integers and 214variables with the pipe (|) symbol and enclosing the entire list in 215round brackets "()" like this: 216 217 ( $var1 | 0x0100 | $var2 | 512 ) 218 219Variables and integer constants may be freely intermixed. The result 220of the operation can either be used as an argument for a call or can 221be used as an expected result for a call. 222 223In addition to all the curses calls being supported by the slave, 224there is one more special call called "drain". This call repeatedly 225called getch() until there are no more characters in stdin. The call 226assumes that the curses input is either in no delay or timed input 227mode otherwise the test will time out and fail. This call can be used 228to clear any pending input when testing testing a timed read to 229prevent the input being used in a later test. 230 2314. Slave 232 233The user has no direct interaction with the slave process. The slave 234is forked off by the director communicates to the director over a set 235of pipes and a pseudo-tty connected to its standard i/o file 236descriptors. The slave executes the passed curses calls and passes 237back return values to the director. The slave automatically calls 238initscr() on start up. 239 240 241 242