Could not disagree more about lexer generators. Every device we support comes with 1-2 parsers, and we started out thinking that Ruby's built-in regex support made flex unnecessary. Bzzzt. Lexers are fiddley whether you write them in Flex or Ruby, but the Ruby version is harder to write.<p>We've settled on Ragel for lexers; Craig wrote "ralex", which I don't use or understand but which he insists we're about to open source. So I recommend Ragel. Or Flex. Or anything but just hand-hacking your own tokenizer.