1 2Comments on the IEEE P1003.2 Draft 12 3 Part 2: Shell and Utilities 4 Section 4.55: sed - Stream editor 5 6Diomidis Spinellis <dds@doc.ic.ac.uk> 7Keith Bostic <bostic@cs.berkeley.edu> 8 9In the following paragraphs, "wrong" usually means "inconsistent with 10historic practice", as most of the following comments refer to 11undocumented inconsistencies between the historical versions of sed and 12the POSIX 1003.2 standard. All the comments are notes taken while 13implementing a POSIX-compatible version of sed, and should not be 14interpreted as official opinions or criticism towards the POSIX committee. 15All uses of "POSIX" refer to section 4.55, Draft 12 of POSIX 1003.2. 16 17 1. 32V and BSD derived implementations of sed strip the text 18 arguments of the a, c and i commands of their initial blanks, 19 i.e. 20 21 #!/bin/sed -f 22 a\ 23 foo\ 24 \ indent\ 25 bar 26 27 produces: 28 29 foo 30 indent 31 bar 32 33 POSIX does not specify this behavior as the System V versions of 34 sed do not do this stripping. The argument against stripping is 35 that it is difficult to write sed scripts that have leading blanks 36 if they are stripped. The argument for stripping is that it is 37 difficult to write readable sed scripts unless indentation is allowed 38 and ignored, and leading whitespace is obtainable by entering a 39 backslash in front of it. This implementation follows the BSD 40 historic practice. 41 42 2. Historical versions of sed required that the w flag be the last 43 flag to an s command as it takes an additional argument. This 44 is obvious, but not specified in POSIX. 45 46 3. Historical versions of sed required that whitespace follow a w 47 flag to an s command. This is not specified in POSIX. This 48 implementation permits whitespace but does not require it. 49 50 4. Historical versions of sed permitted any number of whitespace 51 characters to follow the w command. This is not specified in 52 POSIX. This implementation permits whitespace but does not 53 require it. 54 55 5. The rule for the l command differs from historic practice. Table 56 2-15 includes the various ANSI C escape sequences, including \\ 57 for backslash. Some historical versions of sed displayed two 58 digit octal numbers, too, not three as specified by POSIX. POSIX 59 is a cleanup, and is followed by this implementation. 60 61 6. The POSIX specification for ! does not specify that for a single 62 command the command must not contain an address specification 63 whereas the command list can contain address specifications. The 64 specification for ! implies that "3!/hello/p" works, and it never 65 has, historically. Note, 66 67 3!{ 68 /hello/p 69 } 70 71 does work. 72 73 7. POSIX does not specify what happens with consecutive ! commands 74 (e.g. /foo/!!!p). Historic implementations allow any number of 75 !'s without changing the behaviour. (It seems logical that each 76 one might reverse the behaviour.) This implementation follows 77 historic practice. 78 79 8. Historic versions of sed permitted commands to be separated 80 by semi-colons, e.g. 'sed -ne '1p;2p;3q' printed the first 81 three lines of a file. This is not specified by POSIX. 82 Note, the ; command separator is not allowed for the commands 83 a, c, i, w, r, :, b, t, # and at the end of a w flag in the s 84 command. This implementation follows historic practice and 85 implements the ; separator. 86 87 9. Historic versions of sed terminated the script if EOF was reached 88 during the execution of the 'n' command, i.e.: 89 90 sed -e ' 91 n 92 i\ 93 hello 94 ' </dev/null 95 96 did not produce any output. POSIX does not specify this behavior. 97 This implementation follows historic practice. 98 9910. Deleted. 100 10111. Historical implementations do not output the change text of a c 102 command in the case of an address range whose first line number 103 is greater than the second (e.g. 3,1). POSIX requires that the 104 text be output. Since the historic behavior doesn't seem to have 105 any particular purpose, this implementation follows the POSIX 106 behavior. 107 10812. POSIX does not specify whether address ranges are checked and 109 reset if a command is not executed due to a jump. The following 110 program will behave in different ways depending on whether the 111 'c' command is triggered at the third line, i.e. will the text 112 be output even though line 3 of the input will never logically 113 encounter that command. 114 115 2,4b 116 1,3c\ 117 text 118 119 Historic implementations did not output the text in the above 120 example. Therefore it was believed that a range whose second 121 address was never matched extended to the end of the input. 122 However, the current practice adopted by this implementation, 123 as well as by those from GNU and SUN, is as follows: The text 124 from the 'c' command still isn't output because the second address 125 isn't actually matched; but the range is reset after all if its 126 second address is a line number. In the above example, only the 127 first line of the input will be deleted. 128 12913. Historical implementations allow an output suppressing #n at the 130 beginning of -e arguments as well as in a script file. POSIX 131 does not specify this. This implementation follows historical 132 practice. 133 13414. POSIX does not explicitly specify how sed behaves if no script is 135 specified. Since the sed Synopsis permits this form of the command, 136 and the language in the Description section states that the input 137 is output, it seems reasonable that it behave like the cat(1) 138 command. Historic sed implementations behave differently for "ls | 139 sed", where they produce no output, and "ls | sed -e#", where they 140 behave like cat. This implementation behaves like cat in both cases. 141 14215. The POSIX requirement to open all w files at the beginning makes 143 sed behave nonintuitively when the w commands are preceded by 144 addresses or are within conditional blocks. This implementation 145 follows historic practice and POSIX, by default, and provides the 146 -a option which opens the files only when they are needed. 147 14816. POSIX does not specify how escape sequences other than \n and \D 149 (where D is the delimiter character) are to be treated. This is 150 reasonable, however, it also doesn't state that the backslash is 151 to be discarded from the output regardless. A strict reading of 152 POSIX would be that "echo xyz | sed s/./\a" would display "\ayz". 153 As historic sed implementations always discarded the backslash, 154 this implementation does as well. 155 15617. POSIX specifies that an address can be "empty". This implies 157 that constructs like ",d" or "1,d" and ",5d" are allowed. This 158 is not true for historic implementations or this implementation 159 of sed. 160 16118. The b t and : commands are documented in POSIX to ignore leading 162 white space, but no mention is made of trailing white space. 163 Historic implementations of sed assigned different locations to 164 the labels "x" and "x ". This is not useful, and leads to subtle 165 programming errors, but it is historic practice and changing it 166 could theoretically break working scripts. This implementation 167 follows historic practice. 168 16919. Although POSIX specifies that reading from files that do not exist 170 from within the script must not terminate the script, it does not 171 specify what happens if a write command fails. Historic practice 172 is to fail immediately if the file cannot be opened or written. 173 This implementation follows historic practice. 174 17520. Historic practice is that the \n construct can be used for either 176 string1 or string2 of the y command. This is not specified by 177 POSIX. This implementation follows historic practice. 178 17921. Deleted. 180 18122. Historic implementations of sed ignore the RE delimiter characters 182 within character classes. This is not specified in POSIX. This 183 implementation follows historic practice. 184 18523. Historic implementations handle empty RE's in a special way: the 186 empty RE is interpreted as if it were the last RE encountered, 187 whether in an address or elsewhere. POSIX does not document this 188 behavior. For example the command: 189 190 sed -e /abc/s//XXX/ 191 192 substitutes XXX for the pattern abc. The semantics of "the last 193 RE" can be defined in two different ways: 194 195 1. The last RE encountered when compiling (lexical/static scope). 196 2. The last RE encountered while running (dynamic scope). 197 198 While many historical implementations fail on programs depending 199 on scope differences, the SunOS version exhibited dynamic scope 200 behaviour. This implementation does dynamic scoping, as this seems 201 the most useful and in order to remain consistent with historical 202 practice. 203