Sexy Lexing with Python

Correct me if I'm wrong, but isn't this a bit like reinventing the wheel when PLY is around? <a href="http://www.dabeaz.com/ply/" rel="nofollow">http://www.dabeaz.com/ply/</a><p>There's another popular library that escapes me, too.

I wrote one of these a while back: <a href="http://knot.appspot.com/e/simple-python-scanner" rel="nofollow">http://knot.appspot.com/e/simple-python-scanner</a><p>For his example, it needs one more line:<p><pre><code> simple = Scan( IDENTIFIER = r'[\w_]\w*', OPERATOR = r'[-+\*=]', DIGIT = r'\d+(\.\d+)?', END_STATEMENT = r';', SPACE = r'\s+', ) simple.ignore('SPACE') simple('foo = 5 * 30; bar = bar - 60;') # generates tokens.</code></pre>