Launch HN: Prolific (YC S19) – Quickly find high-quality survey participants

48 点作者 psb31将近 6 年前

Hey HN,We’re Katia and Phelim, cofounders of Prolific (<a href="https://www.prolific.co" rel="nofollow">https://www.prolific.co</a>). We help psychological and behavioral researchers quickly find participants they can trust.We built Prolific because Katia had a hard time finding participants for her psychology studies during her PhD. She briefly used Amazon's Mechanical Turk (MTurk), but didn’t like the user experience and couldn’t get the data she wanted (UK participants). The fundamental problem we’re hoping to help with is better access to psychological and behavioral data. This is challenging in many ways: You have to balance the growth of a multi-sided platform, achieve high data quality, align incentives for all stakeholders (researchers, participants, ourselves, society), diversify the participant pool, to name some. We’re first-time founders and we’ve been bootstrapping our startup for the past 5 years during our PhDs.Researchers build their surveys using Google Forms, Qualtrics, SurveyMonkey, Typeform, or another tool; all you need is a survey URL to get started. We verify and monitor participants so you can get data fast (most surveys are completed in <2 hours). Studies range from one-to-one interviews to surveys of thousands of people and you can retarget participants anonymously for follow up studies. You only pay for data you approve. Our business model is to charge a % service charge (typically around 25-35%) on top of rewards researchers pay the participants.We have 70,000+ survey takers in Europe and North America (for distributions of demographic variables see <a href="https://www.prolific.co/demographics" rel="nofollow">https://www.prolific.co/demographics</a>) and 100s of demographic filters (try our audience checker via <a href="https://app.prolific.co/audience-checker" rel="nofollow">https://app.prolific.co/audience-checker</a>). This means we can find many target demographics for you. For example, you can filter for Democrats vs. Republicans, old vs. young people, students vs. professionals, different ethnicities, people with health problems, Brexit voters, and even collect nationally representative samples!Anyone can sign up as a participant and start earning a little extra cash.It's possible to do research using existing platforms like MTurk. Actually, over 50% of behavioral research is now run online, mostly on MTurk. But there are problems with the quality of data you get from existing platforms, and worse, problems with how the people who participate get treated [1]. Our approach addresses these issues. We think the key differences are: It’s data you can trust: We mandate a minimum hourly reward of $6.50, and often rewards are even higher than that. As a result, participants feel respected and treated like valuable contributors, providing high quality data. We comply with data protection regulation and have a range of technical and behavioral checks in place to ensure high quality data [2]. Demographic prescreening is flexible and free: You can easily invite participants for follow-up studies at no extra cost. You can get niche or even nationally representative samples on-demand. Prolific is built by researchers for researchers. We try to distribute studies as evenly as possible across our participant pool through rate limiting, so have less of a problem with “professional survey takers” than MTurk.Our bigger vision is to build tech infrastructure that empowers behavioral research on the internet. The market opportunity is significant because any individuals, businesses, and governments would benefit from better access to rigorous behavioral data when making decisions. For example, what could we do to best curb climate change? What’s the best way to change unhealthy habits? How can we reduce hate crime and political polarization? The stakes are high, and behavioral research can help us find better answers to these kinds of questions.Moreover, although we built Prolific primarily to help academics, we've noticed that businesses have been using the platform for things like market research and idea validation. This is a new market for us that we're excited to explore. We’d love to hear about any ideas, experiences, and feedback you might have. Thank you![1] <a href="https://news.ycombinator.com/item?id=19719197" rel="nofollow">https://news.ycombinator.com/item?id=19719197</a>[2] <a href="https://blog.prolific.co/bots-and-data-quality-on-crowdsourcing-platforms/" rel="nofollow">https://blog.prolific.co/bots-and-data-quality-on-crowdsourc...</a>

10 条评论

notafraudster将近 6 年前

I'm locked in to my current vendor, but I just wanted to share some of my thoughts about some pain points I saw in your sales pitch.My experience is that most academics pay less than $6.50 per hour for an initial point of contact. For instance, I am currently fielding a survey (N > 5,000 per week, cross-sectional rather than panel) and we pay about $2 all-in, including the provider's charge, for a survey that's about 20 minutes. We'd fall afoul of your compensation rate pretty substantially. If we wanted to do some panel work and we needed re-contact, we'd definitely ramp up our payment quickly to help avoid attrition, but for the first contact, no.If we wanted to be paying out your rate, we would almost certainly have to get an additional sponsor partner to piggyback some consumer research on top of our actual treatment. We don't want to do this, it's hard enough dealing with our primary funders. This makes me believe you are mostly targeting commercial / market behavior researchers. That's fine, but the pitch suggests you want academics. For transparency's sake, what is your balance of private and university clients as of today?Second, even working with large sample providers, their pools are often fairly small. We requested 5,000 unique respondents a week for a year and found our sample provider could only guarantee a 1-2 month lockout. Obviously the effective pool you need to guarantee 5,000 * 52 is enormous and so we were expecting to have to negotiate on lockout, but all of this dances around the fact that sample providers are not transparent about the size of their pool and researchers like us are constantly worried about fraud both by sample providers and by respondents. How large is your pool?Finally, this kind of quota sampling relies on our ability to weight the sample to the population. Weighting is totally permissible, but responsible weighting is going to cap the weight at the high end -- no one wants the one black Republican to skew the entire poll because they have a 150 weight on their observation (this isn't me getting needlessly political, this happened with the USC tracking poll last election cycle). In my experience, the hardest thing about quota sampling as opposed to the old RDD phone samples 20 years ago is that it's very difficult to get high education / high SES / high income respondents. High income respondents should be 10% of the population and they simply are nowhere near 10% of the pools that you get from standard recruitment methods. Can you speak a bit about a) how you recruit high income people into your pool; and b) what percentage of your pool would be high income (say HHI > $125k a year or so).Finally, what is your pool attrition rate? If someone takes their first Prolific survey today, what is the probability they will still be taking a survey a year from now? It's nice that you have re-contact as part of your system, but the problem in my experience is not figuring out how to recontact, it's getting people to stay engaged for a long time.Hope you have good answers to these questions, and that if you do, answering them here will help you get positive exposure from other readers.

评论 #20596687 未加载

评论 #20595470 未加载

rococode将近 6 年前

This is definitely a space that needs innovation! How do you plan to handle the case of survey takers being real people but just skimming through the survey?In my experience running research studies, this was the main problem with MTurk. Things like bots and blatantly junk answers were relatively rare and easy enough to detect and filter. What was much harder to deal with was the fairly high volume of users who just want to finish the survey as fast as possible. It's not so bad for short surveys, but anything over 5 minutes starts to have issues with low quality responses.We had to introduce several control questions to check for consistency in answers and measure time to find outliers. But, it was not an ideal setup and there were still many survey takers we suspected were not paying much attention. The breakdown for our studies was something like 60% good quality answers, 35% low quality answers that are hard to distinguish from high quality, and 5% total junk answers. We ended up doing an in-person study where we got much cleaner results, presumably because people pay better attention when they feel someone is watching them.Wages don't seem to do much for this. We were paying a relatively generous $4 per HIT, estimating 30 minutes per HIT when in reality in reality the average time was under 20.So I'm wondering, are you able to share with us how exactly the "unusual data patterns" and "technical and behavioral checks" can help ensure quality in an ecosystem where users are generally motivated to 1) finish surveys as fast as possible, and 2) appear as if they are giving high quality responses when they are not?

评论 #20595209 未加载

uxamanda将近 6 年前

This is really interesting. I've run into an issue finding qualified participants for user research studies many times. The companies focused on that space tend to be very businesses. Will take a look next time I have a survey!A tiny thing I noticed on the site. If you change the estimated time on your "Study cost calculator" and hit tab, you are bounced down to the next section on the screen so you cannot see the results. I believe adding "tabindex=0" to the HTML on the result would solve this. I kept getting confused as to where the result went. :-)Good luck on your project!Edit: typo

评论 #20596702 未加载

mettamage将近 6 年前

It's very cool to see this idea. Most notably because I had it myself! I thought about it between 2014 and 2015. For me your post is a case study as to why I didn't go through with it and you did.What I'm noticing is the followingMy issues:1. Not being able to find a good business model for the markets I identified2. Not being able to identify all markets3. Not testing it out or simply starting it4. Uncertain whether I'd be able to amass a poolMy successes:1. Identifying the same pain point for researchers2. The ability to create the web app (not that I did this, but I was confident back then that I could and in retrospect, I think I was right)It's funny as I'm currently prepping for becoming a strategy consultant (first round at MBB firm, never got a reply by FAANG in Amsterdam) and I'm already much better at the conceptual thinking regarding my issues -- though strategy consultants simply scratch the surface when it comes to identifying markets, so I have a lot to learn there. And since this is just all interview prep, I've got a long way to learn business skills anyway.An unusual tip if you are 'suffering' the same fate as me. Look on YouTube on how to do new market entry consulting cases and you'll get an idea on how to think about it from a high level.I still wonder what good resources are to learn marketing though.

评论 #20597112 未加载

elektor将近 6 年前

Very cool!I signed up and will be looking out for surveys. Looking at the demographics, I noticed the survey panel is similar to that of Hacker News: white English speaking 20-30 year olds many who are in school. I'd love to hear about your efforts to have a more nationally/globally representative sample?

评论 #20595464 未加载

slugalicious将近 6 年前

Congrats on creating a side hustle while completing grad school. This UX is far superior to MTurk, but as others have suggested, the price point is definitely higher. A few questions:1)"You can easily invite participants for follow-up studies at no extra cost." So, I can re-contact them through your platform OR I gain access to their contact info?2) What % of customers are repeat buyers (multiple surveys)?3) What if I want to recruit childless individuals? The current audience-checker doesn't seem to offer zero as an option.4) How automated is this process for you? What % of customers require a touchpoint or multiple touchpoints?

评论 #20596159 未加载

yc_user_2019将近 6 年前

Started using Prolific a few days. Love the fact that it has a clean and simple interface and it pays in British Pounds which then translate to higher CAD/USD amount. The only thing is that there are not enough survey and the site does sign me out sometime after showing a 500 error. Would also love to know if you guys plan to have a mobile app/mobile friendly site at some point. Started using Qmee app on my phone and love it's interface and level of engagement so far. I understand that Prolific and Qmee might be serving different markets but would be nice to see Prolific implement some of it. Would love to give additional feedback if you'd like.

omarhaneef将近 6 年前

We use Survey Monkey. This seems similar/identical. How do you compare to that? Is there a reason to pick you over them? They charge about $1/response.

评论 #20595834 未加载

baby_wipe将近 6 年前

Could I send someone a link to my iOS app and have them record themselves using it for like 10 min? And get a little feedback from them? How much would that cost?

评论 #20595879 未加载

bwb将近 6 年前

Nice work, checking it out now as I've been looking for something like this :)