I worked at a self-described "data-driven" company, and the analogy senior leadership liked to make was that the company was like a machine learning algorithm, using data (particularly A/B tests) to do "gradient descent" of the product into its optimal form.<p>My first take-away was that using data to make decisions is tremendously, tremendously powerful. A/B tests, in particular, can help determine causality and drive any key metric you want in the direction you want to. Short-term, it seems to work great.<p>Long-term, it fails. Being purely data-driven without good intuition and long-term bets (that can't be "proven" with data), and the product loses its soul. You can
(and should) invest in metrics that are more indicative of the long-term. And you should use data to help guide and improve your intuition.<p>But data is not a substitute for good judgment, or for a deep understanding of your users and their problems, or of "where the puck is going". It's just a tool. It's a very powerful tool, but if it's your main or only tool, you will lose.
I think "dashboard" is perhaps a better analogy than some even realize. When I hear someone describe their startup as purely data-driven, I picture someone driving a car while focusing exclusively on the dashboard.<p>Data <i>is</i> valuable. Some KPI's can show important, even vital information -- often not directly but through the first or second derivative. If something changes rapidly, you probably want to take notice.<p>Overall though, I think the metrics that we have available are about as helpful to a company as the metrics we can track about our bodies are to our health. Sure, monitor your weight, HRV or even your blood sugar levels and get some insight. Let it help inform your decisions. But only that.
I see too many "data driven companies" that are not data driven, but selectivly use data to do what the executive wants to be done.<p>On top of that I see companies with employees who didn't understand college mathematics but now want to be data driven - E.g. basing major decisions on 10 customer feedback data points. Or who don't know the difference between median and average.
I find his rankings of which companies to invest in hilarious. It's like, take the attributes that you think will make a good solid company - experience in the industry, strong analytical skills, good operational skills. You know, the things we know for a fact are going to make a good solid successful company. Ok, take them, and throw them away. We're not interested in them. We're interested in <i>variance</i>.<p>I think fundamentally this suffers from the problem lots of VCs have right now. Cheap money meant that the correct investment strategy was to just pile money into high risk bets. I think this calculation looks very different without free money.
I connect data-driven to A/B testing. And A/B testing for me is like squeezing the maximum out of an orange, instead of making more oranges.<p>It will give a short term boost, but eventually you will have to look beyond your data and explore new things your customers really want.
Like any good idea that sees wide adoption, being "data-driven" has jumped the shark to cargo-cult methodology in most places.<p>The truth is that data is only powerful if applied judiciously with solid knowledge of statistical fundamentals and careful thinking about causality (which will may not be practically falsifiable!). "Data" can also be misread and applied in big powerpoints with a reckless disregard for reality.<p>Enlightenment is understanding enough about data to use it correctly, but also acknowledging its limitations and that to create a successful business you also need vision and insight about potentials and trends for which accurate data does not and can not exist (except as a post-hoc trailing indicator).
"Data-driven" is a fancier term for "empirical", a method for building knowledge based on observation. The other method is "epistemology", a method for building knowledge by reasoning about things. A successful organisation should not snob the one over the other.
Data-driven progress is largely for optimizing choices, not discovering them. Being data-driven can be just as dangerous intentionally ignoring data. Companies “win” by focusing on what matters, with the right people, at the right time.
I believe everything else being equal data-driven is better.<p>Of course if data-driven means short-term with bad KPIs this version of data-driven will fail. I think a lot of people picture this A/B short-term testing when they think of it.<p>A data-driven approach is still far from objective and relies on choosing good metrics to optimize for.<p>To this end I would recommend the book "How to measure anything" from Douglas W. Hubbard (<a href="https://www.goodreads.com/en/book/show/444653.How_to_Measure_Anything" rel="nofollow">https://www.goodreads.com/en/book/show/444653.How_to_Measure...</a>)
I liked this article and agree with the point - when it comes to innovation, new products, and new markets, instinct and experience could probably help more than data.<p>Using data for decision making is more relevant for more mature processes where the data exists to learn from. Being data driven is especially important for data intensive sectors where the complexity of issues cannot be fathomed just by being brilliant, it's here you need to be able to analyse the intricacies using the data to find possible answers.<p>I just published a blog in response to this article explaining my thoughts on it -
<a href="https://terminusdb.com/blog/some-data-driven-companies-actually-win/" rel="nofollow">https://terminusdb.com/blog/some-data-driven-companies-actua...</a>
We use the term data-driven to describe certain parts of our product wherein we use SQL queries against faithful models of the domain to make deterministic, traceable choices.<p>Certainly works for us, but I don’t think this is how most people understand “data-driven” anymore.
I think it's fairly safe to say that most true data-driven companies sustain their innovation and business over the years. They have little incentive to take many risks and will hold their own throughout time.<p>That's to say that real innovation is being more data-inspired. Taking risks with what is known based on your intuition/opinion.<p>A timeless read on this topic is Clayton Christensen's "Innovators Dilemma".<p>I think the exact same phenomena applies to investing too. You can sustain your growth by making moves purely off data, but your big wins may come from being "data-inspired"<p>This blog somewhat covers these ideas in more detail: <a href="https://amplitude.com/blog/data-driven-data-informed-data-inspired" rel="nofollow">https://amplitude.com/blog/data-driven-data-informed-data-in...</a><p>The irony in this all is that nobody can tell the future and therefore what we know in the current moment is the best thing we've got. Even those with more experience, resources, and knowledge can be disrupted. That's the beauty of the world.
Define "win".<p>Is it about making money? Or is it about about using your money to gain greater control? Or some combination of both?<p>This article assumes its all about the money. But if - as I believe - we are under some hybrid corporate-governance system, where these 2 domains are working together whilst appearing to be separate, then the measure is not just money.
Look at the 13Fs for last quarter, find the funds that are still up, ask them if they made trades based on technicals and data or purely on gut feeling that it’s time to buy lean hog futures again because they’ve been through this before. Careful with the crypto bros they are looking at the sidewalk on 5th ave from their penthouses.<p>Data driven companies tend to use the data to quickly fill in dark spots, collect that income and tap the source dry, and then bureaucracy and nepotism set in and the script is flipped. Data is used to explain away why we lost our shirt and how it’s not our fault.<p>The VC is looking for the moon and trying to argue that more data would somehow make the VC feel less like it’s Las Vegas. Sorry it’s all Vegas unless you want to spread and settle for making few percent by investing in all 5 companies and not trying to guess the pack leader.
Sometimes, but the company will lack soul, purpose and a meaning that it stands for. Sometimes it's better to stand by what you want your company's values to be, even if that means losing some potential customers. As a result, you'll have more passionate and engaged customers who are more likely to stick with you long term.
I think the main issue with being a data-driven company is that it's a known secret. Sure, you can snowball a small edge into a winning position over a few years, but isn't it more likely that your competitors become more data-driven, mitigating your one advantage and enhancing their advantages?
I don't think any company is more data driven than any other. After they're successful, companies all need some minimum of measurement. Some say they're data driven but internally data is usually a mess. The right question is, are there any companies where data is not a mess? I'm guessing not.