Such an utter lack of understanding of Twitter's service model I've rarely seen. Message delivery was never Twitter's problem; dealing with persistance (I want my messages when I'm good and ready thanks, not when
the system wants to push them to me - READ: LIKE SMS MULTICAST!) is. Christ.<p>Downvote me all you like, but I'm embarrassed this story made 21 points.
I think this post made by the Twitter team pretty much sums up their woes. (<a href="http://dev.twitter.com/2008/05/twittering-about-architecture.html" rel="nofollow">http://dev.twitter.com/2008/05/twittering-about-architecture...</a>)<p>"Twitter is, fundamentally, a messaging system. Twitter was not architected as a messaging system, however. For expediency's sake, Twitter was built with technologies and practices that are more appropriate to a content management system."
"""It appears that the biggest difference between Twitter and GupShup is 3-tier versus 2-tier. RoR is fantastic for turning out applications quickly, but the way Rails works, the out-of-the-box approach leads to a two-tier architecture (webserver talking directly to database). We all learned back in the 90's that this is an unscalable model, yet it is the model for most Rails applications."""<p>I remember having a discussion with DHH about RoR on IRC a long time ago. Probably somwhere in 2004 when RoR was still very young and not so much known. Coming from a Java/Spring/J2EE background I asked him about abstracting database access in a third tier. He said he had never heard of that and did not know and understand why people were doing that.
I wonder if those techcrunch Twitter stats are up to date or accurate. 3 million messages a day is really nothing in terms of scaling issues. 34 messages a second.<p>Although it also depends how many destinations each of those messages has.
Why do people think Twitter's problems are due to a lack of good technical design?<p>Twitter's real problem is that they've had a pittance of hardware until very recently. It's not my place to go into why this is (and I'm sure the version of the story I know is a bit biased by the teller), but suffice it to say that a lot of Twitter's problems were, until recently, more business-oriented that technology oriented.
@tx claiming that SMS has persistence is wrong, once the SMS is delivered to the customer, and the buffer fills up on the SMSC oldest messages are cleared, a record is written to a file which is then transfered to some other server (That deals with the persistence) so SMS as it stands now is not persistent, since one App handles the messaging and another handles the persistence.
You've gotta remember, Twitter is in a very sticky situation at the moment!<p>One one hand, they know their architecture isn't perfect and will be rebuilding it to cope with more users.<p>But on the other hand - they are a small team which is trying to keep it up in its current state, and cant just decide to close Twitter for a few weeks/months while they work on the new architecture, people will defiantly move to another service, and Twitter is dead.
I laugh when people talk about "sharding". In what way is storing your users in different databases by username any better than calling your disk drives by letter? The major database vendors all tried shared-nothing, and most have rejected it in favour of single images (using partitioning to arrange the data in a form suitable for parallel queries)