Does it scale? Who cares (2011)

Couldn&#x27;t agree with this article more.<p>I built the biggest social network to come out of India from 2006-2009. It was like Twitter but over text messaging. At it&#x27;s peak it had 50M+ users and sent 1B+ text messages in a day.<p>When I started, the app was on a single machine. I didn&#x27;t know a lot about databases and scaling. Didn&#x27;t even know what database indexes are and what are their benefits.<p>Just built the basic product over a weekend and launched. Timeline after that whenever the web server exhausted all the JVM threads trying to serve requests:<p>1. 1 month - 20k users - learnt about indexes and created indexes.<p>2. 3 months - 500k users - Realized MyISAM is a bad fit for mutable tables. Converted the tables to InnoDB. Increased number of JVM threads to tomcat<p>3. 9 months - 5M users - Realized that the default MySQL config is for a desktop and allocates just 64MB RAM to the database. Setup the mysql configs. 2 application servers now.<p>4. 18 months - 15M users - Tuned MySQL even more. Optimized JDBC connector to cache MySQL prepared statements.<p>5. 36 months - 45M users - Split database by having different tables on different machines.<p>I had no idea or previous experience about any of these issues. However I always had enough notice to fix issues. Worked really hard, learnt along the way and was always able to find a way to scale the service.<p>I know of absolutely no service which failed because it couldn&#x27;t scale. First focus on building what people love. If people love your product, they will put up with the growing pains (e.g. Twitter used to be down a lot!).<p>Because of my previous experience, I can now build and launch a highly scalable service at launch. However the reason I do this is that it is faster for me to do it - not because I am building it for scale.<p>Launch as soon as you can. Iterate as fast as you can. Time is the only currency you have which can&#x27;t be earned and only spent. Spend it wisely.<p>Edited: formatting

I care.<p>It&#x27;s easy to brush off scaling concerns as not important, but I&#x27;ve had personal experience where it&#x27;s mattered, and if you want a high profile example, look at twitter.<p>Yes, premature optimization is a bad thing, and so is over engineering; but that&#x27;s easy to say <i>if</i> you have the experience to make the right initial choices that mean you have a meaningful path forward <i>to scale</i> when you do need it.<p>For example, lets say you build a typical business app and push something out quickly that doesn&#x27;t say, log when it fails, or provide an auto-update mechanism, or have any remote access. Now you have it deployed at 50 locations and its &#x27;not working&#x27; for some reason. Not only do you physically have to go out to see whats wrong, you have to organize a reinstall at 50 locations. Bad right? yes. It&#x27;s very bad. (&lt;---- Personal experience)<p>Or, you do a similar ruby or python app when your domain is something that involves bulk processing massive loads of data. It works fine and you have a great &#x27;platform&#x27; until you have 3 users, and then it starts to slow down for everyone; and it turns out, you need a dedicated server for each customer because doing your business logic in a slow language works when you only need to do 10 items a second, not 10000. Bad right? yes. Very. Bad. (&lt;---- Personal experience)<p>It&#x27;s not premature optimization to not pick stupid technology choices for your domain, or ship prototypes.<p>...but sometimes you don&#x27;t have someone on the team with the experience to realize that, and the push from management is to just get it out, and not worry about the details; but trust me, if you have someone who is sticking their neck out and go, hey wait, this isn&#x27;t going to scale...<p>Maybe you should listen to what they have to say, not quote platitudes.<p>Ecommerce is probably one of those things where the domain is well enough known you <i>can</i> get away with it; heck, just throw away all your rubbish and use an off-the-shelf solution if you hit a problem; but I&#x27;m going to suggest that the majority of people aren&#x27;t building that sort of platform, because its largely a solved problem.

Taking a completely blasé approach to efficiency is potentially as dangerous as becoming hyper-focused on it.<p>Not all businesses become roaring successes, and those who achieve moderate success often don&#x27;t get the resources to fix deep-seated performance or architectural issues (either via engineering and&#x2F;or throwing hardware at it.) Eventually these technical woes can completely halt momentum and I&#x27;ve seen it even drown some businesses who just aren&#x27;t able to dig theirselves out of the hole the find themselves in.<p>People always seem to be arguing for extremes, but the most sensible approach for most tends to be somewhere in the middle.

I agree with the general premise of avoiding premature optimizations, but designing systems that scale is important for several reasons:<p>- Startups grow exponentially, if you&#x27;re playing catchup as you&#x27;re growing you are focusing on keeping the lights on and hanging on for the ride. Important for a growing company to focus on vision.<p>- Software that scales in traffic is easier to scale in engineering effort. For example, harder for a 100 engineers to work on a single monolith vs 10 services.<p>- Service infrastructure cost is high on the list of cash burn. Scalable systems are efficient, allow startups to live longer.<p>- If the product you are selling directly correlates to computing power, important to make sure you are selling something that can be done profitably. For example, if you are selling video processing as a service, you absolutely need to validate that you can do this at scale in a profitable manner.<p>I also don&#x27;t agree with the premise that speed of development and scalable systems are always in contention. After a certain point, scalable systems go hand and hand with your ability to execute quickly.

I care because it usually goes like this: Product manager &gt; &quot;Niche sass app {x} will never need to support more than 10-20 users&quot;<p>Two weeks after launch &gt; &quot;We have 10k users and counting, why didn&#x27;t you architect this for scale?&quot;<p>Always assume you underestimated the scope of the project.

I&#x27;m in the unfortunate position where this question actually matters from day 1. I learnt the hard way a few days ago when I hit the bottleneck (about 50-100 concurrent users) and I&#x27;m not sure how to proceed.<p>It&#x27;s a multiplayer drawing site built with Node.js&#x2F;socket.io. I&#x27;m already on the biggest Heroku dyno my budget can allow and it&#x27;s too big of a task to rewrite the back end to support load balancing (and I wouldn&#x27;t know where to start). Bear in mind that this is a side-project I&#x27;m not making any money of.<p>I had a lot of new features planned but now I&#x27;ve put development on hold. It&#x27;s not fun to work on something you can&#x27;t allow to get popular since it would kill it.

Flexibility vs efficiency. Agile vs high momentum.<p>As a rule of thumb, start-ups need to be more agile as they are mostly exploring new territory, trying to create new value or re-scope valueless things into valuable things.<p>Larger companies operate at a scale where minor efficiency improvements can mean millions of dollars and thus require more people to do the same thing, but better. Individualistic thinking on new directions to go is not needed nor appreciated.<p>Of course there are excepttions. The question boils down to whether or not the ladder is on the right walk before charging up it.<p>In rare circumstances you can do both. Either the problem is trivial, or the problem becomes trivial because you have a super expert. 10x programmers who habitually write efficient code without needing to think too much have more bandwidth for things like strategy and direction. The car they drive is both more agile, accelerates faster, has a higher max speed, etc...but even this can&#x27;t move mountains. The problem an individual can solve, no matter the level of genius, is still small in scope conpared to the power of movements and collective action and intention.<p>The most poweful skill is to seed these movements and direct them.<p>Abstractly, this is what VCs look for in founders and also a reason why very smart and technical people feel short-changed that they are not appreciated for their 10x skills. (Making 500k instead of millions&#x2F;billions) They may have 10x skills, but there are whole orders of magnitude they can be blind to.

The reason scale isn&#x27;t so important today is because most DBs can actually scale vertically to really high numbers. The tipping point is high enough that if you have this problem, you probably can also afford to fix it.<p>What matters though is performance and availability. No matter what scale you work at, you can&#x27;t be slow, that will drive people away. You also can&#x27;t be unavailable. This means that you might have to handle traffic spikes.<p>Depending on your offering, you probably also want to be secure and reliable. Losing customer data or leaking it will drive customers away too.<p>So, I&#x27;d mostly agree, in 2016, scale isn&#x27;t a big problem. Better to focus on functionality, performance, security, reliability and availability. These things will impact all your customers, even when you only have one. They&#x27;ll also be much harder to fix.<p>Where scale matters is at big companies. When you already have a lot of customers, you’re first version of any new feature or product must already be at scale. Amazon couldn&#x27;t have launched a non scalable prime now, or echo. Google can&#x27;t launch a non scalable chat service, etc.

Healthcare.gov was a site that failed and suffered due to scaling issues.<p>However, it was an anomaly because unlike a product someone this article is intended towards would be building, that site had an immediate audience of millions of users from the get go.<p>Also, the fact that it took a few weeks to be rewritten to handle the load at which point it became extremely successful, strengthens the original article&#x27;s point.<p>By the time scalability becomes a problem, you will have enough resources to tackle the scalability problem.

No one else bothered by &quot;End-to-end tracking of customers&quot; as the primary concern? Ok then.<p>On the subject of scaling, I think it&#x27;s good to have an idea in your head about a path to scalability. One server, using PHP and MySQL? Ok. Just be aware you might have to load balance either or both the server and DB in the future, and that&#x27;s assuming you&#x27;ve gotten the low hanging fruit of making them faster on their own. But as this thread&#x27;s top comment illustrates, learning that stuff on the fly isn&#x27;t too hard. So maybe it&#x27;s better to make sure you&#x27;re going with technology you sort of know has had big successes elsewhere (like Java, PHP, or MySQL) and even if you&#x27;re not quite sure how you might scale it beyond the defaults you know others have solved that problem and you can learn later if&#x2F;when needed.

He makes the valid point that performance for each individual user, like page load time, does matter. Just that building for an audience size you don&#x27;t yet have is mostly wasted time.<p>Seems reasonable. I wonder, though, if PHP feels like an anchor to the average Facebook developer. I realize they architected around it, but it must have some effect on recruiting, retention, etc. I use PHP myself, and don&#x27;t hate it, but the stigma is there.

The overall premise of the blog is exactly correct; though I would say some areas you probably need to consider more than others.<p>Spending a lot of time figuring out what exact microservice&#x2F;sharding&#x2F;etc strategy you need to serve a zillion visits a day and building it before you&#x27;ve even got customer&#x2F;visitor one is overkill out of the gate. But that shouldn&#x27;t mean you shouldn&#x27;t think about how you&#x27;ll scale over the short term or medium term at all.<p>When I approach scaling, I&#x27;ll tend to spend much more time on the data retention strategy than anything else: databases (or other stores), being stateful, means that it&#x27;s a harder problem to deal with later than earlier as compared to the stateless parts of the system. Even so, I&#x27;m typically not developing the data services for the Unicorn I wish the client will become, I&#x27;m just putting a lot more thought into optimizing the data services I am building so it won&#x27;t hit the breaking point as early as it might if I were designing for functionality alone. I do expect there to be a breaking point and a need to change direction at some point in these early stage designs. But in that short to medium term period, the simpler designs are regularly easier to maintain than the fully &quot;scalable&quot; approaches that might be tried otherwise, and rarely do those companies ever need anything more.

I like the overall idea here. Focus on building something quality first then worry about scaling later. Most servers can handle a decent amount of traffic. Seems like common sense to me. I guess some people can get too hung up on engineering to make their site scale before actually deploying or innovating on the product. Wonder if people have encountered this in the workplace before?

I agree with this article; however, there are considerations that can be made early on, to ensure an easy path for future scalability, that don&#x27;t waste any time or money during the project&#x27;s nascency. For example, if I know I want my app to scale at some point in the future, I may opt to build it in Elixir, or I may choose to use a Riak cluster rather than of MySQL.

Ok but please don&#x27;t do things like using nested array searches with bad runtime when you can use hashmaps instead. I hate seeing code or using programs that are extremely unoptimized.

Once worked at a startup where we expected &#x27;some activity&#x27; in our webshop at product launch, and didn&#x27;t think about scaling for that. Well, some activity turned out to be 450mbit&#x2F;s for five hours, which our unscalable application&#x2F;webshop didn&#x27;t handle very well. It became overloaded in the first minute, and took us more than an hour to get remote access again. It&#x27;s one of those things we did better for our next big event (major sharding, basically replicated the application 32 times of the largest VM instance we could get. It was needed and it survived).

I agree with this article only for launching new products. But if you already have a product which is serving millions of customers, you better worry about scale while you change anything.

While a point the article tries to make (fix your leaky funnel before acquiring users) is true, I disagree with the article. If your application is converting well, scalability problems are not acceptable.<p>I have seen applications that convert very well, but were limited by scalability problems. That meant that the business had to hold on on marketing and user acquisition, missed their financial targets, and that cascaded into breaching contracts. The phrase that nobody wants to hear in that situation is &quot;who cares about scalability&quot;.<p>Now, if you did not have a lot of problems scaling in your particular case, that just means it was not an obstacle for you. e.g: you had good intuition around performance&#x2F;scalability, or the problem was coincidentally a good fit for your technological choices.<p>Unfortunately not everyone has a good intuition about scalability, not everyone is risk averse and not everyone is good at picking a good technology for their use case. So I disagree with this article in the sense that it is not in the best interest of a random reader to not care about scalability.

Having working on an app that we just throwed more hardware at, to the point where the azure subscription cost could be lowered by my whole anually salary, by 3 months work of optimization.<p>I believe performance does matter. We where a 4 person team and could have added a fifth if we had a cheaper design.

I started working at a company 6 months ago, and in that time I&#x27;ve fixed a half dozen performance problems.<p>Some of them didn&#x27;t matter THAT much, but some of them did. My team manages an authentication -service. Some of the calls were averaging hundreds of ms which added terrible overhead to other services&#x27; calls. In other places had CICD builds running tests twice, logs logging twice, inefficient algorithms, single HTTP transactions making dozens of DB calls. You name it.<p>Some of these were simple things that if you stopped to smell the roses you could notice and fix.<p>Let&#x27;s all be boy scouts and make code better while we&#x27;re in it :)

In general: don&#x27;t fix a non-existing problem. You don&#x27;t know beforehand what the problems of the future look like. Fancy technology X of today is technical debt in 10 years. So do invest in solving technical debt along the way in products you keep.

There is a similar argument with regard to making code reusable. I have seen inordinately complex code come from a desire to make it reusable, even if the prospects for it being reused were slim to nonexistent.

Evolution is a beautiful thing.<p>I once worked for a monolith who decided to invent a new way of programming. They built something massive and ready to scale even larger. And then they discovered no one wanted the product.

Fundamentally disagree with the ethos of this article. What about ambition? What happened to that?<p>&gt; You won’t make the cover of Time Magazine and you won’t be ordering a private jet but that Ferrari is a definite possibility, if that’s the thing you are hurting for. (college education for your kids is probably a better idea ;) ).<p>I&#x27;d like to be on the cover of fortune or as Russ Hanneman might say, &quot;I wanna make a fuckton of money all at once&quot;.<p>I don&#x27;t see any problem with being ambitious or wanting that private jet.

A better title would be: scalability is a luxury problem.

Came across a service last week with a free trial and paid plan. How to upgrade? Contact by email! Why spend time and resources on a payment process if you don&#x27;t have any paying customers yet?<p>Obviously not right for everyone, and I&#x27;m not saying it doesn&#x27;t have its own challenges, but the core product deserves the most attention early on.

I do agree, but being at the &quot;we need to scale up asap&quot; stage atm makes it harder to. There&#x27;s a balance to be struck. Maybe about approach of &quot;who cares&quot; with the POC, MVP etc stages, then keep it more and more in mind at each stage after that would be best.

It really depends the type of growth. Organic growth is driven by the quality of product therefore scaling issue comes later. But most venture-backed startup services such as O2O need quickly dominate market by throwing cash to get users so scaling is an issue from day one.

i concur. it&#x27;s fun to dream, but sadly, statistically most of us will never have to worry about scaling! so save yourself the energy and don&#x27;t worry about it.

IMHO until you hit scalability issue don&#x27;t try to spend your time and money there yet

I agree BUT it&#x27;s not that hard to ensure your app scales. It&#x27;s more about using appropriate tools for the job.

Can we have (2011) in the title?

I liked the article and agree with its premise. But as a side question, why do developers use so many parenthetical expressions?<p>Those ideas (like this one, which happens to add nothing) are often either throwaway statements (like this one) or are by themselves complete thoughts that should be a separate sentence. (I see this so often in posts written by devs.)