I think the point you're missing is including a technical one: for a good number of users, getting a tweet history "hits the disk." The limited history is in part due to what's in memory on their servers and easiest to serve.<p>The historic tweet providers don't have this problem-- they (currently) can monetize to offset the cost of the servers to store the data in a necessary way, or by being for analysis only, the fetch-speed (if from disk) isn't as much of a concern.<p>Building a business on Twitter doesn't require using one of the other providers, unless that business is analytics.<p>Rate limits aren't some boogeyman. They're a financial and technological necessity to dissuade abuse and plan for capacity. Serving data isn't free (as in money). Have we had a semi-public API with as much data served in the past? How do you offset that "cost-center" right now?
1. Corporate entities face relentless demand for growth; when creativity fails, there's a scramble for money (control) anywhere they can think of
2. If "Internet time" means anything, certain organizations are already reaching maturity. Just like stars, they consume the easy fuel first, then use the rest.
3. Just as QOS (quality of service) prioritization was never as effective as just growing bandwidth, the organizations that are clamping down (Twitter, craigslist, ...) should understand that they are really entering a new regime, more akin to the allocation of scarce resources in the real world than the limited-only-by-creativity world of the Internet. It has differences, and they might no longer be able to sponge off crowdsourcing, at least not as easily.
As a developer I too a, disappointed by Twitter's move to restrict API access as it has but again, still as a developer, I can understand Twitter's point of view and I'll actually say what they're doing is smart and other sites should follow suit.<p>We all want our API access, we want it now, we want it free, and we want it unrestricted whether we monetize our use of another app's API or not. But unfortunately we can't just feed off Twitter's teet and expect them to just cater to our every want. API access is a privilege, not a right. By monetizing their API as they are they are avoiding the most common mistake that afflicts tech startups - the problem of monetization. They have their "verified" accounts and "promoted tweets" which is a good start and now that the API has become so valuable they are monetizing this. They're a company that offers a free service and making a true profit instead of just taking on VC money and hoping someone will figure out how they'll make money later. They did this at just the right time too. They are now an established player much like Facebook. Now that users are locked in and so many people are getting value from Twitter's quick information sharing and marketing capabilities they made a smart decision with this new monetization strategy.<p>We all should have known the free ride would end eventually. In many cases, apps using the API are actually competing with twitter itself! Surely you can't expect Twitter to allow competing apps like iOS and Android apps to outshine them. So these restrictions force those competing clients to actually innovate still! Yes, access to tweets are limited but apps like Hootsuite and others have found ways to use the API in other ways like scheduling tweets.<p>I'll definitely concede that I think their limits are a bit strict and they are pretty much only useful for hobby projects. At the same time, though, we've become spoiled and entitled and I can definitely understand the restrictions. Reactions to policies like this are to be expected. Everyone always throws a fit when these things happen but we'll live and we have to understand that Twitter needs money to run so they can provide an API at all and if we want to make a buck off Twitter's API we can't expect a free ride. That's like asking the owner of an office building to let you use office space free because... Well... It's there. That said, they need to be smart about this too. Too much restriction will hurt them but too little will mean they aren't generating what they need. But right now I don't think we can judge whether these policies are really that bad because, like I said, any time a service makes these changes everyone starts complaining and a lot of exaggeration goes on.