Home

32 comments

sumanthvepaabout 1 year ago
As an entrepreneur, my observation is that the vast majority of the experiments I run result in an outcome where the null hypothesis cannot be rejected. That is the case for most social science experiments (which is what marketing experiments really are.) But since I don&#x27;t have to publish anything, I don&#x27;t p-hack my results.<p>If you think that scientific rigour will help you avoid the need for good judgement, you&#x27;re in for a great deal of distress. Edited for typos.
评论 #40344716 未加载
评论 #40346967 未加载
评论 #40344529 未加载
评论 #40345297 未加载
评论 #40346068 未加载
jjk166about 1 year ago
The problem here comes from poor experiment design. Seeing how many sales you get from 100 cold calls has no control, and produces a single datapoint. No matter what number you get, be it 0, 2, or 37, will be useless for predicting future sales.<p>First, you need a sample size that will produce a statistically significant result. Cold calls are expected to have a low success rate and be highly variable. 2 in 100 could very well be a massive success. 0 in 100 wouldn&#x27;t rule out its viability. If instead you had a sample size of 2000, then you&#x27;d get a very good signal to noise ratio.<p>Then, you need controls. Look up statistical design of experiments (DOE), you&#x27;ll find efficient ways for finding how much various different factors affect your results. Basically split up those 2000 calls into groups and vary things as you go, so instead of testing whether a specific cold call technique is working, you can see if cold calls in general work.<p>Finally, understand what success means. A business does not need to rigorously prove that it&#x27;s methods are optimal, it needs only to find an adequate strategy to achieve its objectives. You should know what it costs to make 2000 cold calls, you should know how much revenue is generated from a sale, there should be a specific threshold where the number of sales justifies the cold calls. You should be doing a process capability study that you are sufficiently over this threshold that, given expected variation, you will remain above it most of the time. What is your Cpk? How much does it vary among the different sub-experiments you performed?<p>At the end of the day, you&#x27;re still going to need to make decisions with incomplete information, but don&#x27;t pretend that you&#x27;re making a data driven decision when you&#x27;re not. The worst thing is not an inconclusive experiment, the worst thing is an experiment with an erroneous result you mistake for conclusive.
评论 #40344684 未加载
ulf-77723about 1 year ago
I worked in a SaaS company selling an A&#x2F;B Testing app where I saw a lot of tests and their outcomes from different clients.<p>Of course there are the low hanging fruits, which scored the clear success. Most of the tests although were not a clear success, but just a slightly better conversion rate.<p>Multiple tests applied after each other slowly increased the overall value. If taken the advice not to implement the changes of the small success, the overall amount would not have changed.
评论 #40343081 未加载
评论 #40343514 未加载
throwaway2562about 1 year ago
Clickbait, deceptive title.<p>What the author means is ‘get real about continuous learning’<p>Of course it takes money to run experiments you can actually learn from, and the article is bereft of practical advice about doing this on the cheap.<p>However, I clicked. But you don’t have to.
评论 #40343191 未加载
评论 #40343038 未加载
评论 #40343223 未加载
mkl95about 1 year ago
&gt; So when a startup comes to me with an idea for an experiment, the one thing I tell them is: make sure that there’s a well-defined distinction between success and failure. Don&#x27;t fall in the messy middle.<p>Mediocrity is often enough to put food on the table. The world is full of companies aiming to survive for a few more quarters with their little mediocre product. And once a company embraces that idea there&#x27;s usually no going back.
评论 #40345384 未加载
评论 #40344117 未加载
评论 #40344883 未加载
CM30about 1 year ago
Hmm, while the advice is mostly presented as relevant to startups, I kinda wonder if this is also an issue in other aspects of life and business too.<p>Like say, having a side project or hobby. If you try something and find you both don&#x27;t enjoy it and fail miserably at it, that&#x27;s probably a sign it&#x27;s not the right thing to be doing. If you try and succeed beyond your wildest dreams, then you&#x27;d continue.<p>But if you&#x27;re in the middle and become moderately successful, then what (as the article says)? Do you see that as a sign you&#x27;ve made the right decision and should continue? Or a sign that you&#x27;d be better of trying something else?<p>It&#x27;s all too easy to take it as a sign that &quot;things will work out if I keep going down this path&quot; and plow weeks&#x2F;months&#x2F;years into something that may not pay off in the end, or be best suited to your talents and interests.
gumbyabout 1 year ago
“Mediocre success”, as described by the author, is how great science starts out. You try an experiment, (eventually) get a slight indication that your hypothesis might be valid, and then keep iterating until it seems clear that the idea is real (or not).<p>Startup companies generally <i>shouldn’t</i> do science, but that same interactive process <i>should</i> guide you. Because when you start you never know enough.
评论 #40344264 未加载
RcouF1uZ4gsCabout 1 year ago
&gt; One of the wisest and most important pieces of advice I received as a startup founder was this: “The worst outcome is a mediocre success”.<p>…in a zero interest rate environment where there is a bunch of money sloshing around looking to roll the dice multiple times.
评论 #40342837 未加载
评论 #40342807 未加载
jononomoabout 1 year ago
His point is that when it&#x27;s only halfway working you&#x27;re not sure how much more effort to put in because you&#x27;re not sure how it&#x27;s going to pay off.
评论 #40342751 未加载
评论 #40342696 未加载
pinkmuffinereabout 1 year ago
&gt; make sure that there’s a well-defined distinction between success and failure. Don&#x27;t fall in the messy middle.<p>I agree this is the right thing to do, but it isn’t always possible to do for a given experiment. For instance, with the cold-calling example, how do you ensure that it’s either a smashing success or definite failure a-priori? Is there a principled way to choose the threshold number of conversions? Do you just pick an arbitrary number? What if the result is just below your threshold? With some experiments I think the only way to win is not to play. Happy to be corrected if anyone has a solution
评论 #40342853 未加载
1970-01-01about 1 year ago
Yes, there are nuances to success <i>and</i> failure with business. But, the worst outcome for a startup is being sued for millions by a patent troll and losing the business over it. It&#x27;s the worst outcome due to the inherent pure evil within the system itself.<p><a href="https:&#x2F;&#x2F;techhq.com&#x2F;2023&#x2F;02&#x2F;patent-trolling-latest-victim&#x2F;" rel="nofollow">https:&#x2F;&#x2F;techhq.com&#x2F;2023&#x2F;02&#x2F;patent-trolling-latest-victim&#x2F;</a>
mschuster91about 1 year ago
&gt; Making it worse is that we’re all heavily socialized to aim for mediocre success. Schools, universities, large organizations — they don’t want big swings and big misses; they want safety and consistency. A steady 7 is better than 10s interspersed with 0s. This might work well in structured, predictable environments, but in startup-land it’s anathema.<p>Correction: It&#x27;s anathema <i>for VC-funded startups</i> whose backers need one or two to succeed to unicorn size to recoup their losses from the other 98 attempts. For self-funded or privately&#x2F;bank financed startups, it&#x27;s fine to stay low profile and grow slowly, as long as the investors get their money back and everyone gets paid fairly. You may not end up a billionaire, but hell, even a small trades shop can reasonably be worth a million dollars or two after a few decades.<p>It&#x27;s after all how the many, many &quot;hidden champions&quot; of Germany&#x27;s Mittelstand got to where they are: start small, focus on extremely high quality products&#x2F;services, and dominate entire markets even though no one but experts of their respective fields have even heard of them. And that model working out isn&#x27;t restricted to Germany either... Japanese YKK zippers are the best example there [1].<p>A side note: if you don&#x27;t have VCs or, worse, the vipers of the public stock market breathing down your neck, you have so much more freedom as a founder. You can do what&#x27;s best for the company <i>long term</i>.<p>[1] <a href="https:&#x2F;&#x2F;slate.com&#x2F;business&#x2F;2012&#x2F;04&#x2F;ykk-zippers-why-so-many-designers-use-them.html" rel="nofollow">https:&#x2F;&#x2F;slate.com&#x2F;business&#x2F;2012&#x2F;04&#x2F;ykk-zippers-why-so-many-d...</a>
darkhorse13about 1 year ago
My literal life goal is to have a software startup with moderate success WITHOUT VC funding. One that can sustain my lifestyle, but also keep me busy and happy without going completely overboard.
评论 #40345972 未加载
评论 #40345112 未加载
mattgreenrocksabout 1 year ago
I think people are getting hung up on &quot;mediocre&quot; and &quot;success&quot; here.<p>Author argues for trying to be objective about the viability of business ventures here. It&#x27;s easy to tell yourself lies that doing just one thing a little better on the execution front will magically triple your conversion rate.
ianwalterabout 1 year ago
Everything is a series of battles and campaigns. If the overall war is a mediocre success, you still have plenty to learn from if you&#x27;re digging into those.
__lbracket__about 1 year ago
Competitive sports isn&#x27;t for everybody, regardless of your ability to work hard. If you aren&#x27;t built for it, you&#x27;re going to be a failure or a mediocre success. Entrepreneurship isn&#x27;t for everyone either.<p>Silicon Valley normalized the idea that if you follow the scientific process (The Lean Method, finding Product&#x2F;Market Fit or whatever this garbage is known in the latest terminology), and burn out for 10 years of your life, then you WILL achieve success. Most people are blind to survivorship bias even if they know the phenomenon. On top of it, incompetent, scumbag VCs muddy the waters with their snake oil &quot;wisdom&quot; which bright eyed, impressionable tech bros gulp down with passion.<p>If you achieve a &quot;mediocre&quot; success, which allows you financial independence, then you HAVE achieved success, regardless of what the VCs are pontificating. They&#x27;d scream because their investment didn&#x27;t 1000x. Fuck them and continue living your life. Don&#x27;t burn out for these soul-less MBAs.<p>And say you achieve the outsized &quot;success&quot;. What then? If you go with the history of silicon valley then your so-called success is going to be a shit deal for workers, consumers and the environment.
usernamed7about 1 year ago
The difference between scalable wins and one-offs is knowing why it worked and being able to repeat it. Can you be a human being and connect with other human beings and their needs, and help them move forward?<p>The author is right; you want clear signals, not noisey ones. If you&#x27;re looking for the light but keep seeing shadows you need to turn around.<p>The same is true for people who launch on product hunt, and get 50 signups. Success? If you&#x27;re selling $1K deals, yes. But if it&#x27;s just for $4, or free... do you have PMF?<p>but to be fair, if you can get a small group of people engaged, and you can tease out why&#x2F;how, it usually stands that others will respond too. But not everyone will need it, and those that may need it may not be in market for it.
kjellsbellsabout 1 year ago
The thoughts in this article apply equally well to launching products inside a corporate as well.<p>Sometimes a product team will &quot;buy the business&quot;, eg offer credits or funding to a prospect in order to get their logo. This muddies up the entire experiment. Is the product worth X, or not? Does it solve a problem for customers that is worth X, or not?<p>For b2b sales, selling 1 or 2 can also be exceptionally costly because you are on the hook for long term support: and if you bought the business, you are now underwater from day 1 with a customer who has you over a barrel. Worse, your sales force now think that every deal can be won on cost, and will start to show signs of deal fever.
评论 #40344086 未加载
karaterobotabout 1 year ago
&gt; The worst outcome, the very worst outcome, is to get a small but non-zero number of sales — say 1 or 2. Because now you&#x27;re in a bind. Do you double down or pull the plug? Does cold-calling work or not?<p>That seems like a failure to define success in the first place. That seems like doing an experiment and then deciding how it made you <i>feel</i>. You probably want to declare beforehand what your minimum number of sales would be in order to qualify as a success. I agree that the results can still be ambiguous, but the situation he&#x27;s describing sounds like a poorly designed experiment anyway.
评论 #40345011 未加载
评论 #40343442 未加载
msoadabout 1 year ago
Mediocre Success + VC Investment = failure worse than just failing to start a business. It means wasting many years of your life for a &quot;small win&quot; for a VC.
评论 #40344194 未加载
评论 #40343292 未加载
ufociaabout 1 year ago
The worst outcome by far is SBF and Elizabeth Holmes.
anon291about 1 year ago
By the title, I thought this would be another clickbaity post about how we should all quit our jobs and become entrepreneurs, but in actuality, the post is better described as &#x27;if you&#x27;re going to do that, the worst outcome is to be moderately successful&#x27;. Which I agree with, and it&#x27;s something that really gives me pause when thinking about launching on my own.<p>I mean, my corporate job is realistically pretty cushy and as the sole income earner, my family depends on me. I&#x27;m sure I could do well on my own, but it&#x27;s hard to predict the success of any venture. Failure I actually don&#x27;t mind, because the answer is easy... just get another job. The worst possible outcome is &#x27;success&#x27; where you have a customer base and demand, but it&#x27;s just never going to be enough to make up for that lost income. That would truly be the worst. You can&#x27;t just quit without damaging your reputation; on the other hand, selling would likely be difficult without significant recurring income.
marbanabout 1 year ago
<i>Don&#x27;t fall in the messy middle.</i><p>There&#x27;s a book for that <a href="https:&#x2F;&#x2F;www.themessymiddle.com" rel="nofollow">https:&#x2F;&#x2F;www.themessymiddle.com</a>
helpfulmandrillabout 1 year ago
Mediocre success makes the world go round.
demondemidiabout 1 year ago
What a useless article. Zero insight. Anyone who has run a study knows this. It’s one of the reasons statistics is useful. And also decision making. Before hiring a team of cold callers the person should have had a clear target. Without that kind of leadership of course its confusing.
hmmm-i-wonderabout 1 year ago
A better phrasing could be &#x27;unclear outcomes are the worst outcomes&#x27;, which when stated like that seems self defining. If its not clear its a success or failure, or its marginally a success or failure, then it can be hard (or dangerous) to draw firm conclusions.
mtreis86about 1 year ago
Nyquist frequency applies here too?
flanked-everglabout 1 year ago
normality is seriously underrated and the best way to have a happy life is to pursue normality.
zurferabout 1 year ago
it&#x27;s a good article, but misses the point. you can&#x27;t apply the scientific method early in a startup. You never have enough data and all the factors included change too quickly.<p>what you need is conviction. if it works for 1&#x2F;100 calls, and you believe in the idea continue iterating. Also, observe what your peers are doing, you probably don&#x27;t need&#x2F;want to innovate on sales or business model.
BenFranklin100about 1 year ago
This is a trite column with no practical, actionable advice. The HN comments on the story are more perceptive. Downvoted.
KyleOneillabout 1 year ago
I owned a ML based company with an emphasis on Edge computing for the better part of two years, made $150&#x2F;hr to collect the necessary data alone, and having one client was more than enough income.<p>The danger of mediocre success that the author speaks of is actually the danger of indeterminate market movement being identified as meaningful in a naturally open system, then attempting to draw meaningful conclusions about the data when the most likely cause for, &quot;mediocre success&quot; is you simply haven&#x27;t had enough time in the market. Many of the traits we associate with good business require word of mouth and first hand or second hand experiences with a company&#x27;s output&#x2F;services. Accessibility and availability to your client or consumer is the primary reason businesses fail, and on top of that, most marketing tactics suck valuable equity from the businesses, to the point where people are hired on to determine how much money it will take to gather more clients to cover the costs of the campaign.<p>This is nothing new.<p>Frankly, I have found networking events and bars to be the best generators of clients. I never openly advertise what I do, (except technically in this comment), but often times I instead talk with people and ask what they do for a living, if they enjoy it, etc. I could make the argument that I waste my time at these places, but ultimately the whole reason I even own the company is from being at those places and chatting people up, taking them out of the crazy warped reality we get piped in from mass media.<p>This all said, the article just goes to show how many corporations actually do more harm than good to themselves based on the data they collect and what they believe it correlates to or what the causation may be. These deterministic representations of what reality might actually be for a business owner, or <i>anyone</i> in <i>any situation</i>, is a neo-classical example of Uncertainty Principle imo.<p>To loop back to the top,<p>My concern is the author suggests that a business model is created and built when you don&#x27;t have clients or a working idea of how to directly target them in the first place besides a phone call. The best advertising is client word of mouth,and time in the market. You won&#x27;t have either of those things until you focus on making one client at a time happy, disclosing what you can and cannot do, laying out realistic expectations and deadlines for them or you, having a contract for any services rendered, if they choose a competitor after signing for the same purpose, that sort of thing, and perhaps most importantly having a business centric understanding of if it makes sense to have the upkeep costs of an entire business as opposed to self realizing what those costs can be up front, clausing your contract as a contractor to cover those costs you might write off in the process of the project for the client, and sticking it out as a lone wolf who gets the job done properly, and as defined by the contract ,when needed.<p>Maybe my comments will stop someone from setting up a LLC or S corp. To be honest, I wish I had never done so and I should have just ate the contractor taxes instead. Currently, there&#x27;s a emphasis on volume of production with these corporations, so the cost of paying 10 engineers for a year is much less the cost mentally of trying to do all these things yourself, one client at a time, as a business that has all the same tax benefits or penalties of a larger company with definable cost of goods sold because those people go home at night instead of you the business owner who might pull all nighters or get anxiety over things you can&#x27;t get finished due to unforeseen circumstances, and no team to back you up.<p>In the end, the best way to start a business is to gamble a million Dollars with a need in the market for what you offer, at a price clients are willing to pay, and a level of attention to detail they might not get from a larger company in exchange for building trust with you as opposed to a market leader. You hire a few really good, overqualified engineers and people who are willing to tell you then you are doing a bad job or wrong, and you take their collective advice. Most of the time, you are going to be the one who needs to improve, and most of the time, you aren&#x27;t going to like that feedback as a business owner. It will confuse you, you will want to give up. You will have sleepless nights, you will have nightmares about the most superficial things such that it really wouldn&#x27;t be much different than getting emotionally invested in Dance Moms even though you&#x27;ve seen all the seasons by proxy 4 times already.<p>But, those people have ideas. Those people have visions, and ultimately, the best way to run a business is to have enough money to accept the business might not pull a profit in year one if you&#x27;re going to be nice to your employees.<p>So to all this I suggest:<p>Don&#x27;t start a business unless you really, really have a excellent focus on what you want to do with that financial vehicle because depending on how much money it is we live in an an economy where capital gains are taxed less and require less actual work than actually breaking a sweat or going to a job, assuming you have enough money to invest in the market.<p>I&#x27;ll probably get down voted to hell for that last quip, but when my investment account had larger gains in a month than I made in a month as a business owner, off of only a 10,000 investment, I felt pretty stupid. Obviously I can&#x27;t guarantee that profit&#x2F;loss, but that&#x27;s why you make that money in the market while you can, quit your day job if the market investment pays more, and go find something amazing and new to build that just might change the world.<p>I truly apologize for this.... Block of text. This article just seemed to skip over some really key things that are a reality for many a small business owner.<p>My experience with it was even after charging $200&#x2F;hr for data collection, for the better part of two years, having a single client was all I needed. Looking back, I would have happily taken a larger tax hit as a contractor than go through the trouble of creating an entire company. I reinvested all that money into development kits, educational resources for myself, and now I&#x27;m sitting at home without a job this year, coding and learning whatever I want to code in the hopes that I develop something people appreciate and enjoy using, or finds some wonderful niche purpose worth charging money over.<p>If I cared about money, my business would still have clients but I&#x27;d be much less happier and I&#x27;d have the same number of employees -- myself.<p>Owning a business to me means putting out quality products that do what people ask, runs efficiently, is easily read and understood, and meets customer expectations in a timely and financially sound manner.<p>Hiring another employee isn&#x27;t gonna do that unless I can afford good employees, and I can&#x27;t feel good about what I pay good employees unless I have enough money to pay them what I would like to pay myself. The expectation being that they take the same level of time and care that I would if their task was my own, so I don&#x27;t <i>have</i> to go review all their work all the time.<p>I&#x27;d be happy paying devs 400k&#x2F;1m&#x2F;year+ and having the best team of devs in the world and only taking home 100k as their manager, if it meant we make quality products, and everyone who works there is happy, and some of the stress of being a business owner could be relieved.<p>Sure I could take more money home but ultimately my goal is to make sure we can do what clients need us to do and set realistic expectations and at the end of the day that&#x27;s a high price to pay upfront for services like a company like mine would be, but cheaper than a lifetime of dealing with errors, code problems, you all know what I&#x27;m talking about.<p>Have a good day folks and don&#x27;t be afraid to charge what you are worth for your services. Yes, there are people out there developing products making ridiculous amounts of money, money you could stockpile and turn into your own business someday without worrying about the struggle bus mediocre success, limited working capital concept this author talks about.
jeffreportmill1about 1 year ago
Having a corporate job is usually a &#x27;mediocre success&#x27;. If you strike out on your own with something that involves your passion and you earn a comparable income, you are a stellar success. Don&#x27;t let a nepo baby tell you otherwise.
评论 #40343269 未加载
评论 #40343696 未加载
评论 #40343617 未加载