Lines Matching full:tokens

25 /// isSameToken - Return true if the two specified tokens start have the same
28   // If two tokens have the same kind and the same identifier info, they are  in isSameToken()
59 /// LexRawTokensFromMainFile - Lets all the raw tokens from the main file into
65   // Create a lexer to lex all the tokens of the main file in raw mode.  Even  in LexRawTokensFromMainFile()
79     // identifier tokens.  in LexRawTokensFromMainFile()
108   // all tokens that are preprocessed from a file other than the main file (e.g.  in RewriteMacrosInInput()
109   // a header).  If we see tokens that are in the preprocessed file but not the  in RewriteMacrosInInput()
110   // lexed file, we have a macro expansion.  If we see tokens in the lexed file  in RewriteMacrosInInput()
112   // tokens, or macro arguments etc.  in RewriteMacrosInInput()
122     // If the raw file hits a preprocessor directive, they will be extra tokens  in RewriteMacrosInInput()
151     // Okay, both tokens are from the same file.  Get their offsets from the  in RewriteMacrosInInput()
166       // Comment out a whole run of tokens instead of bracketing each one with  in RewriteMacrosInInput()
192     // in the output buffer.  Insert the whole run of new tokens at once to get  in RewriteMacrosInInput()