After reading the article just realised why didn't you use mechanize with beautifulsoup library in Python. I feel you could have succeeded in your first attempt within 3-6 months. I did it in 2008 and it worked great for me. Also look at another open source project called scrapy <a href="http://scrapy.org/" rel="nofollow">http://scrapy.org/</a>