The Database Administrator is dead

I think someone's spent too long in tech-startup-land and has forgotten that there's a whole bunch of businesses outside of that, some of which don't have access to reliable internet connectivity and need to keep going even if the Internet's down.

I guess I had better tell my colleagues that our jobs are all dead! Wait, it&#x27;s an absolute statement for a headline, so it&#x27;s actually &quot;absolute crap&quot;.<p>&gt; These days, the technology decision maker is the dude with Sublime Text open and a cloud control panel up in Chrome.<p>And when he is successful and gets clients, and a few thousand rows in his database, he realizes that he needs someone to keep that database alive. He needs someone to figure out how to make the cartesian product queries he&#x27;s written into efficient queries.<p>At first, he hires a consultant for a few one-off gigs. However, then he&#x27;s paying someone $200[1] an hour, typically with 8-16 hour engagements. After getting sick of that cost, and still lacking any kind of long term caring about his product, he comes to our team, and hires us to be his DBA, albeit remotely.<p>Business as a DBA is booming. Nobody thinks they need a DBA, but the reality is that you really can&#x27;t afford to not have a DBA. We have customers coming on board with no backups, no high availability plans, no disaster recovery plans, queries that are performing cartesian products (and thus taking minutes against very small datasets), and no monitoring. (And yes, a good portion of users come to us while using the &quot;solution&quot; proposed by the OP (like AWS RDS), for many the same problems.)<p>We set them up with comprehensive backups, automated failover solutions, and 24x7 monitoring. Suddenly, their DB is no longer the primary source of downtime. They&#x27;re no longer loosing customer engagement because their frontend takes seconds to render. They&#x27;re no longer in the position of loosing their entire company because some junior developer accidentally dropped their users table in production.<p>In short, DBAs are a required part of your business, if you&#x27;re using a database. You just haven&#x27;t been burned bad enough by a poor database setup to realize it.<p>[1] Actual hourly rates for a planned engagement. Emergency rates are closer to $450 an hour. Why so much? You can&#x27;t get a DBA from a college, from a technical school, or from any other form of formal education. Most DBAs these days are grown internally from developers or system administrators who decide to (or are forced to) specialize while on the job. There are single-digit thousands of us world wide, and we&#x27;re in high demand.

If you are working on confidential data, it is very likely that your security policy does not allow you to store it &quot;in the cloud&quot;, or to hand it off to a third party without the kinds of contractual obligations that will perplex and confuse anyone trying to sell a best-effort service.<p>Thus your databases will be in-house, thus you will hire one or more DBAs and depend upon their expertise.

The DBA is essential for big companies, not only Facebook or Apple. The author of the post works for a database-as-a-service provider, it&#x27;s just marketing.

The cloud is killing all the low-level admins. Mail administrators? Backup operators?<p>Moving to BYOD will continue to reduce the numbers of administrators as well. Eventually you&#x27;ll have two groups of IT staff, a very small group of high-level engineers who build and implement everything, and then a very large group of low-skill helpdesk type people who reset your accounts and fill in your login information on your device.<p>Amazon&#x27;s Mayday support service shows this already. Soon, a form of this will be on every product you can think of. Office, Windows, every tablet and computer in your office.

TL;DR: Cloud-based DB co-founder says that most of his customers don&#x27;t employ a DB admin.<p>Um, it would only be surprising if they <i>did</i>, that would surely indicate that his company wasn&#x27;t doing a good job.

The DBA may be dead, but he has a lot more life in him than &quot;MongoDB database-as-a-service&quot; is ever likely to have....

I am a database administrator.<p>I somewhat agree with the conclusion which essentially says having your on premise DBA is on the decline.<p>Hiring DBAs has always been tricky for employers anyway. It&#x27;s a position of responsibility (to protect the business&#x27; data) that is hard to know you&#x27;ve hired the right person for, difficult to replace, and difficult to allow to take any vacation&#x2F;leave since there will seldom be more than one.<p>What I don&#x27;t agree with in the conclusion is that by moving off premise it&#x27;s all moving to datastore-as-a-service. Large amounts will also move to remote DBA as a service (consulting firms).

If they define DBA as a guy who sit in a room 9-to-5 for a $100k+ salary, then, yes, there is a recession, you know. But usually (or rather unusually) DBA is an engineer, who could do much more than looking at EXPLAIN output and tune some variables once in a week.<p>I have been Informix DBA for years and I could tell that we were the strongest guys in a team, because in order to do our job we had to understand (abstract out of running system) the data-flows, access pattern (and especially locking issues), actual server&#x27;s topology (disk controllers, channels, hard-drivers) data partitioning (where this or that table-space lives, what&#x27;s else on this volume, this channel, this controller) what are the access patterns for each table, how indexes are utilized, etc.<p>We also have patched, compiled and installed all the required software (have you ever tried to compile Informix support into PHP4? you definitely should.)) and to teach coders how to use it, and then deal with access patterns of silly scripts, etc.<p>The claims that some crap like MongoDB (of all things!) service could replace skilled, productive, (but, yes, quite expensive) professionals is, of course, utter nonsense (what else we could expect from MongoDB?).<p>DBAs and Sysadmins (real ones, not these clowns who use nothing but chef or puppet and doesn&#x27;t know how .&#x2F;configure &amp;&amp; make works) are becoming extinct purely from economical reasons, and all these cloud services, ironically, require even more knowledge to deal with, because all that virtualization crap messes everything up even more (google for redis on EC2 for a change).<p>Sadly, idiots are taking over the world slowly but steadily,)

DBAs have specialized knowledge related to the maintenance and optimization of a particular SQL engine. Developers see the database as a black box where data gets stored, but there is a lot going on under-the-hood to keep the data reliable, consistent, secure, redundant, and available.<p>The last thing you want is a full transaction log during peak hours.

&gt; These days, the technology decision maker is the dude with Sublime Text open and a cloud control panel up in Chrome.<p>My sublime text is open and I have a cloud control panel open on chrome, am I the technology decision maker? Nobody told me that here on the company.<p>Wait... maybe nobody told me because... I&#x27;m the decision maker D=

Who organizes how the data is stored? In some companies, the database is largely ignored until performance and scaling is an issue. At that point a DBA is brought in and the next months or even years are spent fixing the design so the entire product can perform properly and grow.

Persistence ignorance introduced by mainstream ORMs about 5-7 years ago have turned the new generation of programmers blind to the fact database IS part of your application. Records you save don&#x27;t get sucked into a magic black box that just responds to your queries as your app needs grow.<p>DBAs are not going to go anywhere. Sure, you can scale the DB in the cloud quite a bit in and perform well but it&#x27;s not free :) In the cloud, you would be literally paying for your bad design decisions in terms of hard dollars rather than performance issues.<p>Most start-ups or in house apps are fine while the salary(dba) =&gt; cost of cloud. However, that magic condition starts returning false pretty quickly if you are doing any non-trivial data management.

The world is definitely changing for the DBA to which he made a very good points. I feel he graced over a big facet in a single line of the article which had to do with Apple&#x2F;Facebook and large companies.<p>The banking world doesn&#x27;t give their data to others and organization of ATM data, market data, customer account data, etc is only getting more complex and requiring excellent organization and management for terra data analysis to protect cusomters for fraud and worse.

So, a thought experiment.<p>Pick any data storage system you like - MongoDb, Redis, Riak, whatever.<p>Now, you get to place a bet. In the future (let&#x27;s say 25 years) what will still exist: your choice of NoSQL, or the standard database with schema, SQL, access, and so on? If you choose wrong, you die.<p>Now you get to see the future and see if you die. Which are you betting on?

The DBA is not dead! They just look and smell that way!<p>But in all seriousness, if your app uses a database, you are incompetent not to employ an expert to help with the database, whether advising on the query plan of those non-performant queries, or what is the best setup for the current stage of the business and app, they are very useful.