Changeset 162

Show
Ignore:
Timestamp:
06/23/05 07:53:18 (4 years ago)
Author:
sshimko
Message:

-Made whitespace skippable again. Was SPECIAL_TOKEN while experimenting with strings a few months ago. This should save a small amount of time
and an even smaller amount of memory as Tokens will no longer be generated for whitespace.

Files:

Legend:

Unmodified
Added
Removed
Modified
Copied
Moved

  • trunk/src/com/tresys/framework/dictionary/DictionaryParser.jj

    r161 r162  
    176176/* skippable tokens (whitespace) these ARE NOT passed to the parser */ 
    177177/* other whitespace */ 
    178 <DEFAULT, NORES_STATE, IN_NAMEVAL, EP_2_STATE> SPECIAL_TOKEN : { " " | "\t" | "\n" | "\r" | "\f" } 
     178<DEFAULT, NORES_STATE, IN_NAMEVAL, EP_2_STATE> SKIP : { " " | "\t" | "\n" | "\r" | "\f" } 
    179179 
    180180/* #comments */