Mistakes startups make when talking to users

I thought this was going to be a post about customer development, where the issue is not generally the formatting of the questions, but asking the wrong questions in the first place, or misinterpreting the answers.<p>Surveys can be a useful tool, but nothing replaces a real conversation. Even asking more open ended questions in an email or survey form can give you so much more information than radio button answers (and unless you're a huge company, it's not too hard to read a few hundred open-ended responses).<p>Along those lines, I'll share a snippet from patio11's Microconf presentation last year (posted on HN yesterday), regarding interpreting the responses you get to a more open-ended conversation:<p>"If you’re solving a problem people actually have, they will say at this point, 'Shut up and take my money.' If someone says, 'That’s kind of interesting, tell me when that exists,' you have not successfully identified a problem that people actually have."<p>You don't get that kind of insight from radio buttons.

While the headline doesn't really match the article, because the article talks about surveys and not customer development, I want to point out one error I see quite frequently in startups or SaaS-businesses when they talk to users.<p>It can be summed up as: Asking the user for some sort of interaction and then failing to respond to the user's action.<p>This can be blog posts where there's an open question at the end with the goal to engage readers and failing to answer to interesting comments.<p>But I see it more frequently when I go for some sort of SaaS-trial, my account expires after a week or two, they ask me for feedback and since I'm not convinced of the product yet, I might give them feedback and state my concerns. That is actually the last chance for them to make me change my mind.<p>And they miss it. They simply fail to reply or to address my concerns that I stated in my feedback.<p>I don't have any numbers but my educated guess is that they lose quite a few customers this way. Some lost me this way and I bet there are others.

If you want to get better at writing surveys - go read "Asking Questions" <a href="http://www.amazon.co.uk/Asking-Questions-Definitive-Questionnaire-Questionnaires/dp/0787970883" rel="nofollow">http://www.amazon.co.uk/Asking-Questions-Definitive-Question...</a><p>If you actually want to look at how to get better at talking to users - rather than writing surveys - I'd recommend getting a copy of Andrew Traver's pocket guide "Interviewing for Research" <a href="http://www.fivesimplesteps.com/products/interviewing-for-research" rel="nofollow">http://www.fivesimplesteps.com/products/interviewing-for-res...</a><p>There's also an upcoming book on interviewing by Steve Portigal <a href="http://rosenfeldmedia.com/books/interviewing-users/" rel="nofollow">http://rosenfeldmedia.com/books/interviewing-users/</a> - googling around his name will find you some useful interviewing slide decks.<p>Indi Young's book Mental Models has some good advice on interviews in one of the chapters, and is also a useful read <a href="http://rosenfeldmedia.com/books/mental-models/" rel="nofollow">http://rosenfeldmedia.com/books/mental-models/</a><p>(Bias disclaimer: I know Andrew, and I do workshops on interviewing ;-)

They forgot the #1 mistake: Too Many Damn Questions.<p>There have been plenty of times when I have willfully taken surveys to help out companies I like then immediately gave up after being shown a survey with 20-60 questions. No.<p>What pisses me off is that they're asking for my time and effort without giving me anything in return. Like they feel entitled to taking 15 minutes of my time.<p>The worst is when they only display 1 question per page and there's a little progress bar at the top. And after 10 questions you only see the progress bar 7% full. At least it's not as bad as the time Starbucks/Chase Duetto Cards hired someone with a studdering speech impediment to call me for a phone survey... That lasted 30 minutes.

For anyone interested in the art and science of surveys/customer interviews, I found this book to be quite helpful: <a href="http://www.amazon.com/Asking-Questions-Definitive-Questionnaire-Questionnaires/dp/0787970883/ref=sr_1_1?ie=UTF8&#38;qid=1366916840&#38;sr=8-1&#38;keywords=asking+questions" rel="nofollow">http://www.amazon.com/Asking-Questions-Definitive-Questionna...</a> It's written for sociology/anthropology students, but the principles are equally applicable in product conversations. It's filled with lots of best practices for designing questions, interview flows, surveys, and other customer tools. There's actually a lot of mistakes that can be made without realizing and this book helps prevent many of them. e.g. switching up types of questions from yes/no, to scales, to free response to prevent answers based on momentum.

The problem with this data is that it comes from Survata, which is one of those god awful spammy survey walls that makes users fill out a survey before they can access the desired content. When your data comes from being hostile and annoying to users, I don't think it can be trusted.

Sometimes unbalanced scales (#4 on this article's list) are appropriate. If you have a five-option list where 95% of your users are answering 4 or 5, then it's time to unbalance the scale so that you can get better information. However, you need to have collected the balanced scale version for long enough to know it's an actual trend.

I still couldn't truthfully answer most of the "improved" questions with anything other than "it depends" or "none of the above", because they still make too many assumptions about people's behavior and choices that simply don't match the diversity of reality.<p>Pay for a streaming video services? Depends, 99% of current services I wouldn't pay for. How often do I get a new mobile phone? Depends, sometimes two in a year, but the current one is already 3 years old. Fuck if I know the "average". How often do I check my mail per hour. Depends on where I am and what the fuck I'm doing, of course.<p>Etcetera, etcetera.

Mistake #11: assuming that an online survey equates to discussions with your users. This is not a knock on survata, but rather talking about what it means to engage with a company's userbase.<p>Survata is one tool, but nothing beats direct, 1-1 customer interaction. Can't remember where I read it, but one CEO of a startup made it his job to handle a decent amount of customer support email. Of course, that only scales so far, but out of the gate -- that's priceless interaction.

Also: Not having an actual conversation with your customers