For full documentation, see lexdoc(1). This manual entry is intended for use as a quick reference.
-b generate backing-up information to lex.backup. This is a list of scanner states which require backing up and the input characters on which they do so. By adding rules one can remove backing-up states. If all backing-up states are eliminated and -Cf or -CF is used, the generated scanner will run faster.
-c is a do-nothing, deprecated option included for POSIX compliance.
NOTE: in previous releases of flex -c specified table-compression options. This functionality is now given by the -C flag. To ease the the impact of this change, when flex encounters -c, it currently issues a warning message and assumes that -C was desired instead. In the future this "promotion" of -c to -C will go away in the name of full POSIX compliance (unless the POSIX meaning is removed first).-d makes the generated scanner run in debug mode. Whenever a pattern is recognized and the global yy_flex_debug is non-zero (which is the default), the scanner will write to stderr a line of the form:
--accepting rule at line 53 ("the matched text")
The line number refers to the location of the rule in the file
defining the scanner (i.e., the file that was fed to flex). Messages
are also generated when the scanner backs up, accepts the
default rule, reaches the end of its input buffer (or encounters
a NUL; the two look the same as far as the scanner's concerned),
or reaches an end-of-file.
-f specifies fast scanner. No table compression is done and stdio is bypassed. The result is large but fast. This option is equivalent to -Cfr (see below).
-h generates a "help" summary of flex's options to stderr and then exits.
-i instructs flex to generate a case-insensitive scanner. The case of letters given in the flex input patterns will be ignored, and tokens in the input will be matched regardless of case. The matched text given in yytext will have the preserved case (i.e., it will not be folded).
-l turns on maximum compatibility with the original AT&T lex implementation, at a considerable performance cost. This option is incompatible with -+, -f, -F, -Cf, or -CF. See lexdoc(1) for details.
-n is another do-nothing, deprecated option included only for POSIX compliance.
-p generates a performance report to stderr. The report consists of comments regarding features of the flex input file which will cause a loss of performance in the resulting scanner. If you give the flag twice, you will also get comments regarding features that lead to minor performance losses.
-s causes the default rule (that unmatched scanner input is echoed to stdout) to be suppressed. If the scanner encounters input that does not match any of its rules, it aborts with an error.
-t instructs flex to write the scanner it generates to standard output instead of lex.yy.c.
-v specifies that flex should write to stderr a summary of statistics regarding the scanner it generates.
-w suppresses warning messages.
-B instructs flex to generate a batch scanner instead of an interactive scanner (see -I below). See lexdoc(1) for details. Scanners using -Cf or -CF compression options automatically specify this option, too.
-F specifies that the .ul fast scanner table representation should be used (and stdio bypassed). This representation is about as fast as the full table representation (-f), and for some sets of patterns will be considerably smaller (and for others, larger). It cannot be used with the -+ option. See lexdoc(1) for more details.
This option is equivalent to -CFr (see below).-I instructs flex to generate an interactive scanner, that is, a scanner which stops immediately rather than looking ahead if it knows that the currently scanned text cannot be part of a longer rule's match. This is the opposite of batch scanners (see -B above). See lexdoc(1) for details.
Note, -I cannot be used in conjunction with full or fast tables, i.e., the -f, -F, -Cf, or -CF flags. For other table compression options, -I is the default.-L instructs flex not to generate #line directives in lex.yy.c. The default is to generate such directives so error messages in the actions will be correctly located with respect to the original flex input file, and not to the fairly meaningless line numbers of lex.yy.c.
-T makes flex run in trace mode. It will generate a lot of messages to stderr concerning the form of the input and the resultant non-deterministic and deterministic finite automata. This option is mostly for use in maintaining flex.
-V prints the version number to stderr and exits.
-7 instructs flex to generate a 7-bit scanner, which can save considerable table space, especially when using -Cf or -CF (and, at most sites, -7 is on by default for these options. To see if this is the case, use the -v verbose flag and check the flag summary it reports).
-8 instructs flex to generate an 8-bit scanner. This is the default except for the -Cf and -CF compression options, for which the default is site-dependent, and can be checked by inspecting the flag summary generated by the -v option.
-+ specifies that you want flex to generate a C++ scanner class. See the section on Generating C++ Scanners in lexdoc(1) for details.
-C[aefFmr] controls the degree of table compression and scanner optimization.
-Ca trade off larger tables in the generated scanner for faster performance because the elements of the tables are better aligned for memory access and computation. This option can double the size of the tables used by your scanner. -Ce directs flex to construct equivalence classes, i.e., sets of characters which have identical lexical properties. Equivalence classes usually give dramatic reductions in the final table/object file sizes (typically a factor of 2-5) and are pretty cheap performance-wise (one array look-up per character scanned). -Cf specifies that the full scanner tables should be generated - flex should not compress the tables by taking advantages of similar transition functions for different states. -CF specifies that the alternate fast scanner representation (described in lexdoc(1)) should be used. This option cannot be used with -+. -Cm directs flex to construct meta-equivalence classes, which are sets of equivalence classes (or characters, if equivalence classes are not being used) that are commonly used together. Meta-equivalence classes are often a big win when using compressed tables, but they have a moderate performance impact (one or two "if" tests and one array look-up per character scanned). -Cr causes the generated scanner to bypass using stdio for input. In general this option results in a minor performance gain only worthwhile if used in conjunction with -Cf or -CF. It can cause surprising behavior if you use stdio yourself to read from yyin prior to calling the scanner. A lone -C specifies that the scanner tables should be compressed but neither equivalence classes nor meta-equivalence classes should be used. The options -Cf or -CF and -Cm do not make sense together - there is no opportunity for meta-equivalence classes if the table is not being compressed. Otherwise the options may be freely mixed. The default setting is -Cem, which specifies that flex should generate equivalence classes and meta-equivalence classes. This setting provides the highest degree of table compression. You can trade off faster-executing scanners at the cost of larger tables with the following generally being true:
slowest & smallest
-Cem
-Cm
-Ce
-C
-C{f,F}e
-C{f,F}
-C{f,F}a
fastest & largest
-C options are cumulative.
-Pprefix changes the default "yy" prefix used by flex to be prefix instead. See lexdoc(1) for a description of all the global variables and file names that this affects.
-Sskeleton_file overrides the default skeleton file from which flex constructs its scanners. You'll never need this option unless you are doing flex maintenance or development.
x match the character 'x'
. any character except newline
[xyz] a "character class"; in this case, the pattern
matches either an 'x', a 'y', or a 'z'
[abj-oZ] a "character class" with a range in it; matches
an 'a', a 'b', any letter from 'j' through 'o',
or a 'Z'
[^A-Z] a "negated character class", i.e., any character
but those in the class. In this case, any
character EXCEPT an uppercase letter.
[^A-Z\\n] any character EXCEPT an uppercase letter or
a newline
r* zero or more r's, where r is any regular expression
r+ one or more r's
r? zero or one r's (that is, "an optional r")
r{2,5} anywhere from two to five r's
r{2,} two or more r's
r{4} exactly 4 r's
{name} the expansion of the "name" definition
(see above)
"[xyz]\\"foo"
the literal string: [xyz]"foo
\\X if X is an 'a', 'b', 'f', 'n', 'r', 't', or 'v',
then the ANSI-C interpretation of \\x.
Otherwise, a literal 'X' (used to escape
operators such as '*')
\\123 the character with octal value 123
\\x2a the character with hexadecimal value 2a
(r) match an r; parentheses are used to override
precedence (see below)
rs the regular expression r followed by the
regular expression s; called "concatenation"
r|s either an r or an s
r/s an r but only if it is followed by an s. The
s is not part of the matched text. This type
of pattern is called as "trailing context".
^r an r, but only at the beginning of a line
r$ an r, but only at the end of a line. Equivalent
to "r/\\n".
<s>r an r, but only in start condition s (see
below for discussion of start conditions)
<s1,s2,s3>r
same, but in any of start conditions s1,
s2, or s3
<*>r an r in any start condition, even an exclusive one.
<<EOF>> an end-of-file
<s1,s2><<EOF>>
an end-of-file when in start condition s1 or s2
The regular expressions listed above are grouped according to
precedence, from highest precedence at the top to lowest at the bottom.
Those grouped together have equal precedence.
Some notes on patterns:
foo/bar$ foo|(bar$) foo|^bar <sc1>foo<sc2>bar
%{
#undef YY_INPUT
#define YY_INPUT(buf,result,max_size) \\
{ \\
int c = getchar(); \\
result = (c == EOF) ? YY_NULL : (buf[0] = c, 1); \\
}
%}
-ll library with which to link scanners to obtain the default versions of yywrap() and/or main().
lex.yy.c generated scanner (called lexyy.c on some systems).
lex.yy.cc generated C++ scanner class, when using -+.
<FlexLexer.h> header file defining the C++ scanner base class, FlexLexer, and its derived class, yyFlexLexer.
flex.skl skeleton scanner. This file is only used when building flex, not when flex executes.
lex.backup backing-up information for -b flag (called lex.bck on some systems).
lexdoc(1), lex(1), yacc(1), sed(1), awk(1).
M. E. Lesk and E. Schmidt, LEX - Lexical Analyzer Generator
reject_used_but_not_detected undefined or
yymore_used_but_not_detected undefined - These errors can occur at compile time. They indicate that the scanner uses REJECT or yymore() but that flex failed to notice the fact, meaning that flex scanned the first two sections looking for occurrences of these actions and failed to find any, but somehow you snuck some in (via a #include file, for example). Make an explicit reference to the action in your flex input file. (Note that previously flex supported a %used/%unused mechanism for dealing with this problem; this feature is still supported but now deprecated, and will go away soon unless the author hears from people who can argue compellingly that they need it.)
flex scanner jammed - a scanner compiled with -s has encountered an input string which wasn't matched by any of its rules.
warning, rule cannot be matched indicates that the given rule cannot be matched because it follows other rules that will always match the same text as it. See lexdoc(1) for an example.
warning, -s option given but default rule can be matched means that it is possible (perhaps only in a particular start condition) that the default rule (match any single character) is the only one that will match a particular input. Since
scanner input buffer overflowed - a scanner rule matched more text than the available dynamic memory.
token too large, exceeds YYLMAX - your scanner uses %array and one of its rules matched a string longer than the YYLMAX constant (8K bytes by default). You can increase the value by #define'ing YYLMAX in the definitions section of your flex input.
scanner requires -8 flag to use the character 'x' - Your scanner specification includes recognizing the 8-bit character 'x' and you did not specify the -8 flag, and your scanner defaulted to 7-bit because you used the -Cf or -CF table compression options.
flex scanner push-back overflow - you used unput() to push back so much text that the scanner's buffer could not hold both the pushed-back text and the current token in yytext. Ideally the scanner should dynamically resize the buffer in this case, but at present it does not.
input buffer overflow, can't enlarge buffer because scanner uses REJECT - the scanner was working on matching an extremely large token and needed to expand the input buffer. This doesn't work with scanners that use REJECT.
fatal flex scanner internal error--end of buffer missed - This can occur in an scanner which is reentered after a long-jump has jumped out (or over) the scanner's activation frame. Before reentering the scanner, use:
yyrestart( yyin );or use C++ scanner classes (the -+ option), which are fully reentrant.
See lexdoc(1) for additional credits and the address to send comments to.
Some trailing context patterns cannot be properly matched and generate warning messages ("dangerous trailing context"). These are patterns where the ending of the first part of the rule matches the beginning of the second part, such as "zx*/xy*", where the 'x*' matches the 'x' at the beginning of the trailing context. (Note that the POSIX draft states that the text matched by such patterns is undefined.)
For some trailing context rules, parts which are actually fixed-length are not recognized as such, leading to the abovementioned performance loss. In particular, parts using '|' or {n} (such as "foo{3}") are always considered variable-length.
Combining trailing context with the special '|' action can result in fixed trailing context being turned into the more expensive variable trailing context. For example, in the following:
%% abc | xyz/def
Use of unput() or input() invalidates yytext and yyleng, unless the %array directive or the -l option has been used.
Use of unput() to push back more text than was matched can result in the pushed-back text matching a beginning-of-line ('^') rule even though it didn't come at the beginning of the line (though this is rare!).
Pattern-matching of NUL's is substantially slower than matching other characters.
Dynamic resizing of the input buffer is slow, as it entails rescanning all the text matched so far by the current (generally huge) token.
flex does not generate correct #line directives for code internal to the scanner; thus, bugs in flex.skl yield bogus line numbers.
Due to both buffering of input and read-ahead, you cannot intermix calls to <stdio.h> routines, such as, for example, getchar(), with flex rules and expect it to work. Call input() instead.
The total table entries listed by the -v flag excludes the number of table entries needed to determine what rule has been matched. The number of entries is equal to the number of DFA states if the scanner does not use REJECT, and somewhat greater than the number of states if it does.
REJECT cannot be used with the -f or -F options.
The flex internal algorithms need documentation.