Does it have to do with the VIEWSTATE and STATEHANDLER crap that is supposed to maintain page state between page loads?<p>Reason I ask is that I have been writing screeb scrapers for websites of which several are ASP ones and these scrapers are so damn slow compared to some of the other websites I've dealt with.
In addition to the myriad of reasons a page / site can be slow, with .NET if the site has to do a cold start, aka nobody's accessed it in a while (by default 20 minutes), it can take a significant amount of time to load it all up depending on various factors.<p>If you know you're scraping a .NET site you could do a simple request a few minutes earlier so it shouldn't have to do a cold start.