TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

Net Promoter Score Considered Harmful

209 pointsby arnklintover 7 years ago

23 comments

Dave_TRSover 7 years ago
&gt;We Can’t Reduce User Experience To A Single Number<p>This is the crux of the issue - CEOs do need a simple metric to summarize performance in each of their business units in order to evaluate performance and prioritize company resources. Metrics are only useful for decisions if they are easy to understand, and consistently measured. Just as Earnings is used to measure profitability, and Revenue is used to measure growth, NPS is the best they&#x27;ve come up with so far for customer experience. Of course none of these 3 metrics is perfect, but they still need to be measured and complimented by more detailed reporting. To better understand Earnings you look at changes in the component parts, revenue and cost. To better understand NPS you look at the mix of scores (1s, 2s, 3s, etc) and follow up questions like actual referrals, which differ by business (unlike NPS which can be applied more broadly).<p>The author devotes most of the article pointing out that there are areas where follow up questions would give a richer understanding of customer experience, which of course all NPS proponents would agree with.<p>What he could do a better job of is trying to convince us why executives shouldn&#x27;t even attempt to summarize their company&#x27;s performance in customer experience, just like they summarize lots of other complex activities to report to shareholders. Too many companies focus only on hard metrics like revenue and profits, which is why they find NPS a helpful way to steer the company&#x27;s focus back to the customer.
评论 #16009595 未加载
评论 #16017538 未加载
leroy_masochistover 7 years ago
I work as a consultant to PE companies as a source of cash income while doing other entrepreneurial stuff. Much of this work involves pre-deal commercial diligence and&#x2F;or post-deal portfolio company strategy review. In the context of this type of work, which often involves formal customer &#x2F; partner interviews, I think NPS is actually a very useful metric.<p>For sure, NPS has its limitations. Broadly speaking, I think it&#x27;s a bad metric for something like asking random strangers how much they like their iPhone. But if you&#x27;re trying to figure out the strength of, for example, an ERP software company&#x27;s customer franchise, NPS is really useful, for two reasons:<p>1. When you&#x27;re talking to people who know a product well, and you&#x27;re talking to them in a professional context (e.g., a scheduled 15-minute interview that is part of their work day), they tend to be thoughtful about the 0-10 rating.<p>2. The follow-up question traditionally used in NPS surveys, &quot;What could [company] do to make that score a 10 instead of [number]?&quot; almost never fails to provide valuable feedback. I believe this is because of the way the question is set up: first the respondent is asked to provide a broad indicator of their customer satisfaction, and then, with that number identified, they&#x27;re asked for details as to <i>why</i> they chose that number. I have no background in psychology but I think it&#x27;s pretty intuitive that this question-flow focuses the mind and prompts people to provide insightful constructive recommendations.
评论 #16009289 未加载
svantanaover 7 years ago
&gt;Any normal statistician would just report on the mean of all the scores they collected from respondents<p>What? No!!! Any self-respecting statistician would know that the mean of a quantity where addition is not well-defined is meaningless.
评论 #16008657 未加载
评论 #16008567 未加载
评论 #16010237 未加载
评论 #16010061 未加载
correlationover 7 years ago
NPS is first and foremost a marketing tool for Bain &amp; Co to sell expensive consultants and def not &quot;the one question to ask&quot;.<p>Several marketing scholars have pointed out the psychometrically undesirable properties of the metric and there is conflicting evidence on e.g. the metric&#x27;s predictive validity, e.g. in relation to company success<p>(All in all that old HBR marketing article contains very little details to back up the claim that it is a good and valid metric.)<p>If you can go with behavioural outcomes for measuring success, e.g. purchases, I think that that will always be more powerful than what a user says in a survey.<p>And if you can then causally link (e.g. through experiment) something you can influence (e.g. some dimension of &quot;UX) to that outcome you actually have a quite good decision tool too.<p>NPS does not provide that.
bunderbunderover 7 years ago
I regularly screw these scores up by honestly answering the question I was asked.<p>&gt; On a scale of 0-10, how likely are you to recommend Netflix to a friend or colleague?<p>0<p>&gt; You answered 0 to the last question. Why wouldn&#x27;t you recommend Netflix?<p>It&#x27;s 2018, people. Everyone I know either already uses Netflix, used to use Netflix, or doesn&#x27;t own a computer. If I started going around making a recommendation like that, they would think I&#x27;m a prat.
评论 #16008975 未加载
评论 #16009453 未加载
评论 #16009719 未加载
评论 #16009202 未加载
评论 #16010097 未加载
评论 #16012914 未加载
评论 #16009461 未加载
评论 #16010633 未加载
giovannibajo1over 7 years ago
My feeling about NPS has always been that the very wide pointing system polarizes users towards the two extremes; most people aren’t analytical and they don’t build up a mathematical way to objectively come up with a number between 0 and 10. They would instinctively vote 10 if they’re happy, 0 if they’re angry, and something in the middle if they’re not fully satisfied. The way they map partial satisfaction to a number is unscientific, and this is why NPS only rewards perfect scores (9 or 10).<p>Similar findings were made by Google with the YouTube rating system that was switched from 1 to 5 stars to a simple upvote&#x2F;downvote after realizing that most users are polarizing towards the extremes: <a href="https:&#x2F;&#x2F;techcrunch.com&#x2F;2009&#x2F;09&#x2F;22&#x2F;youtube-comes-to-a-5-star-realization-its-ratings-are-useless&#x2F;" rel="nofollow">https:&#x2F;&#x2F;techcrunch.com&#x2F;2009&#x2F;09&#x2F;22&#x2F;youtube-comes-to-a-5-star-...</a>
评论 #16008278 未加载
评论 #16008310 未加载
notimetorelaxover 7 years ago
There&#x27;s a lot to unpack in this article, but one thing caught my eye. In the &quot;The Ultimate Question 2.0&quot; book the author claims that NPS needs to be considered as a relative measure to competitors. So if we follow this advise, United needs to aim to have the highest score accross all airlines. To give an example, an NPS of -20 is decent if competitors have below -30.
评论 #16008668 未加载
dahartover 7 years ago
&gt; We Can’t Reduce User Experience To A Single Number<p>Growth is a single number, and NPS is measuring growth, not UX.<p>I&#x27;m no big lover of NPS, but this analysis is awful! He claims it&#x27;s bad because he doesn&#x27;t understand it. That&#x27;s not very scientific either.<p>I&#x27;m not sure NPS works well at all, but the idea behind it is obvious. It&#x27;s a growth metric. The goal of NPS is to tell you how many new customers an existing customer will refer to you over the lifetime of their account.<p>This is just like population statistics. NPS is trying to measure your customer birth rate by asking how many customers are (or intend to be) pregnant. It&#x27;s not an accident that there are only two thresholds, and it&#x27;s wrong to conclude that these thresholds indicate a problem with the method.<p>If a customer recommends less than one new customer during the entire time they&#x27;re with you, then you have a replacement rate less than 1, and you&#x27;re losing customers over time. If you have a replacement rate between 1 and 2, and your customer lifetime is long (say, years) then you aren&#x27;t growing fast. If your replacement rate is 2 or higher, and your customer lifetime is short (say, weeks) then you are growing more than 200% month over month virally, without the need for marketing.<p>What the people who designed NPS did, I am sure (meaning I&#x27;m speculating, but giving the strongest possible interpretation), is measure some responses and compare it to the number of actual referrals, then drew the lines where the referral rates cross from negative growth to neutral grow, and from neutral growth to positive growth. That&#x27;s what I would do. And it seems plausible that people who give a score of 6 or less won&#x27;t end up referring anyone, on average.<p>Sadly, the article doesn&#x27;t conclude with any real alternatives for measuring growth. Since NPS is an indirect growth metric, the better answer may be to simply measure your growth directly. That means understanding engagement and activity, not just counting how many accounts exist, but other than that, counting your active customers is a single number that will reliably tell you growth, and can&#x27;t be gamed by your customers -- it can only be gamed by yourself and your team.
评论 #16009642 未加载
评论 #16010113 未加载
simonhampover 7 years ago
If you’re going to ask what someone _will_ do, it’s best to keep it to a simple binary decision and always remember that that’s only representative of a single moment in time&#x2F;the current customer service cycle for that individual.<p>As this article identifies correctly, the more important question is the harder-to-quantify “why?”.<p>Armed with this, the fluctuating and heavily biased score will have more meaning and context. Using that data to improve the service&#x2F;product for the next cycle may do more to realistically reflect sentiment change than any weirdly random maths.<p>Of course, this still ignores what someone actually does do. But it’s possible to track certain cases of referral and actually to encourage trackable referrals using the correct approach. This is a more solid measure (if possible) than any hypothetical gauge of future behaviour - again as the article points out.
bichiliadover 7 years ago
Let me first say that the burden of proof here relies on Fred Reichheld (or whoever else champions NPS now) to defend NPS from the claims in the cited papers.<p>I totally agree that NPS feels like placing faith too much in something magical. However, I don&#x27;t feel totally convinced by this article; when people calculate NPS, do they really ignore any other analysis on the raw input (i.e. all 0&#x27;s vs all 6&#x27;s)? Yes, there are totally exceptional datasets that make the NPS look like it hasn&#x27;t move; has that been a problem for people in the wild?<p>Also, is picking one ecommerce customer out of the data enough to say that there&#x27;s no correlation between NPS and future behavior?<p>I&#x27;ve seen NPS used as a way of keeping a pulse on a community; if it drops sharply, something is clearly wrong in a way that normal monitoring can&#x27;t surface.
评论 #16009666 未加载
mercwearover 7 years ago
NPS is great when used and managed correctly. Unfortunately most companies send out an NPS survey and ONLY look @ the score. The score is a lagging indicator of the company taking time to actually read the comments gathered from the NPS survey and act on them to improve the customer experience.<p>NPS is not harmful, poor leadership and failure to correctly manage a feedback program is.
评论 #16009209 未加载
评论 #16010454 未加载
twissover 7 years ago
I think the NPS question could benefit from some more specific answers, such as:<p>- Yes, I would bring up how much I like [product]<p>- Yes, if the subject of [product category] came up<p>- Yes, if my friend had a problem that [product] could solve<p>- No, because I don&#x27;t care enough about [product category]<p>- No, because I don&#x27;t like [product] enough<p>- No, because I dislike [product]
评论 #16011564 未加载
notimetorelaxover 7 years ago
A long article with a mix of good and some poorly informed arguments, that ends up a sales pitch for a UX workshop.
评论 #16008121 未加载
ianbickingover 7 years ago
I kind of like NPS: it doesn&#x27;t try to make sophisticated distinctions, and distinguishes between enthusiasm and acceptable mediocrity. It&#x27;s oriented towards growth because of that emphasis on enthusiasm. The very low end of the scale doesn&#x27;t really matter, as many of the people who score the product very low have already stopped using the product, so grouping 0 responses with 5 responses seems reasonable to me.<p>But the NPS number is more a number for executives than people working directly on the product. NPS doesn&#x27;t give UX people anywhere near enough information to do their job. None of these top-line numbers have enough information for that.<p>I found myself thinking about this: <a href="http:&#x2F;&#x2F;www.ianbicking.org&#x2F;blog&#x2F;2016&#x2F;04&#x2F;product-journal-data-up-and-data-down.html" rel="nofollow">http:&#x2F;&#x2F;www.ianbicking.org&#x2F;blog&#x2F;2016&#x2F;04&#x2F;product-journal-data-...</a> – there is a real dysfunction that the author is seeing, and is common. Top-down process design means that executives design the work process and give it to their managers, and managers design the process and give it to their reports, and you end up with processes that aren&#x27;t designed to help individuals do their own jobs, instead everyone is supporting someone else&#x27;s job. Executives make broad decisions: is this product succeeding? If we continue our current approach where will that take us? Are the teams performing well? Someone who is doing UX shouldn&#x27;t be worried about these questions, they should be concerned about specific details, because those specific details are what a UX designer can change.
jwatteover 7 years ago
I believe the NPS mechanism of &quot;6 is bad&quot; intends to capture more psychology than the statistical analysis on the article gives it credit for.<p>That being said, I agree that a three point scale would be better in most cases.<p>The focus on the number proves the saying: when a metric turns into a goal, it ceases to be useful. The goal should be &quot;dollars earned.&quot; All the metrics should just assist in improving this metric. Else you&#x27;ll get the bad statistical hacking described. Those hacks are not just for NPS; I think we&#x27;ve all seen them from various companies.<p>Of course, dollars earned also ends up being gamed (see also: Enron) but at least there are usually better safeguards against this in most organizations. And increasing dollars earned are usually a good end state.
mrisoliover 7 years ago
I&#x27;ve seen people who barely understood the NPS formula obsess over their NPS score as well as customers who wrote quizzical responses such as &quot;I love your service it is the best, everything is perfect&quot; and grade an 8, overall I find its an interesting vanity metric.
评论 #16009034 未加载
tjpnzover 7 years ago
I worked for a large Japanese company that settled on NPS as one of their main KPIs. It was spearheaded by their American educated CEO and was implemented across all their products. The data looked good even when the company was hemorrhaging customers. Despite the flaws I can&#x27;t help but wonder if questions were asked about its applicability to Japanese audiences.
daanloover 7 years ago
NPS is a great tool if you want to understand your word of mouth growth. The issues described in the article are issues of Data Interpretation and not of the actual tool. EG &quot;I gave a 0, because I dont know anyone to recommend the product to&quot; is a typical word of mouth problem to Work on
txmx2017over 7 years ago
In customer service, a 10 is good, a 9 is good with room for improvement, and everything below an 8 is bad. The author makes a big deal about the difference between a 0 and 6. But both mean the customer is disatisfied. There’s no real difference.
评论 #16009077 未加载
评论 #16009118 未加载
评论 #16010404 未加载
评论 #16010846 未加载
NelsonMinarover 7 years ago
NPS: like Klout for Customer Satisfaction
评论 #16010468 未加载
tunaover 7 years ago
this field was studied and explored in the 90s by University of Wu-Tang Clan in their seminal paper C.R.E.A.M <a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=PBwAxmrE194" rel="nofollow">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=PBwAxmrE194</a>
tomxorover 7 years ago
...Article considered harmful: Constant JS thread use, avoid.
tangueover 7 years ago
&quot;NPS thinks that a 6 should be equal to a 0.&quot; No : It considers that at 6 you&#x27;re not a promoter. But your score will be 6. And the progress from 0 to 6 will be reflected on your score.
评论 #16008271 未加载
评论 #16008282 未加载