I am looking for a web crawler generator, that I could prototype what I would like to crawl using a visual dom explorer type of interface, is there such a thing? anything recommended? Thanks
Not sure if you're looking for a hosted solution (there are less of those) or a library to build upon (google "scraper"/"crawler" and the programming language of your choice).<p>Either way, check out ScraperWiki: <a href="http://scraperwiki.com" rel="nofollow">http://scraperwiki.com</a><p>Hope this helps.