TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

Mechanical Turk Workers Are Not Anonymous

127 点作者 anandkulkarni大约 12 年前

12 条评论

nmcfarl大约 12 年前
This ID is pretty much the only thing employers get to ID the workers, and thus track the quality of their work. (Which is particularly important on Turk, as anonymity seems to encourage fraud.) Everything else is push: web service calls to contact a worker, or comment on an assignment. That stuff is all intermediated by turk, meaning it’s all anonymized fairly effectively - and quite constrained as to content and format.<p>These days Turk goes so far as to handle your 1099s for you - so you don’t know the names of even the workers that have done more than $600 worth of work for you…<p>Of course most employers in it for the long haul build up reputation databases built on these ids, and many encourage their workers to fill out profiles, direct on the employers website, allowing for more direct contact, and less constrained messaging.<p>Which is why this is a big deal for the employers: names, pictures, locations, and review helpfulness, can all be shoved into the reputation algorithm or used by survey companies for demographics. So it will be harvested.<p>And that makes turk a lot less anonymous. Which seems like a breech of trust to the workers, because although they aren’t guaranteed anonymity in the TOS, that has been the bargain so far.
anandkulkarni大约 12 年前
This was a curious design choice by Amazon.<p>By choosing to use Amazon IDs as worker identifiers, Mechanical Turk gets all of the disadvantages of having anonymous strangers do jobs on the web -- lack of accountability, diminished incentives, malicious workers -- but none of the advantages of protecting workers, since at the end of the day, they're not actually anonymous anyway.<p>This is a non-issue in MobileWorks (and for that matter, oDesk and Freelancer as well). In these systems, everyone's explicitly working under a real name, which improves incentives to perform well, and makes it a whole lot easier to get good results through trust and honesty.
评论 #5334964 未加载
LogicX大约 12 年前
&#62; "For academic researchers: worker IDs may have to be treated as personally identifiable information. For example, publishing worker IDs online in public data sets may be a violation of worker privacy, and counter to the requirements of the researcher’s institutional review board."<p>This is why we created <a href="http://SocialSci.com" rel="nofollow">http://SocialSci.com</a> for use by academic researchers: We have better control over PII outside the Amazon ecosystem, and access to more data we can take our own steps to protect or de-identify, enabling higher quality data collection.<p>In general academic research done on Mechanical Turk has a ton of issues, from response quality to verifiability of responses. Frankly people have no incentive to remain authentic across surveys, and we've seen cases of people just taking on identities or just faking answers, which isn't reliably detectable. The only way we've been able to overcome this is by tracking users' response consistency over time, and keeping them anonymous in the process. It's proved to be a much more reliable method, and at this point based on our experience we can't recommend MTurk for valid academic research.
评论 #5332726 未加载
评论 #5332681 未加载
smackfu大约 12 年前
Are they supposed to be anonymous, or were people just assuming that?<p>Edit: The closest thing I found was that you can't create tasks that involve "collecting personal identifiable information". I would think that is aimed more at preventing tasks that are essentially selling your info for the price.
firloop大约 12 年前
Shocking that something with privacy implications like this wouldn't have been already caught by Amazon. It almost seems like common sense to not reuse (at least, publicly) the same user ID between a user's public page and a users more anonymous MTurk profile but I guess it slipped through the cracks or something.
obituary_latte大约 12 年前
It appears crowdresearch.org has yet to research how to handle a crowd.
评论 #5332203 未加载
评论 #5331679 未加载
runarb大约 12 年前
Cache: <a href="http://webcache.googleusercontent.com/search?q=cache:crowdresearch.org/blog/?p=5177" rel="nofollow">http://webcache.googleusercontent.com/search?q=cache:crowdre...</a>
评论 #5331785 未加载
niggler大约 12 年前
Where in the privacy policy does it state that Mechanical Turk program wouldn't share the ID with people using the service? I don't see any item in the privacy notice <a href="https://www.mturk.com/mturk/privacynotice" rel="nofollow">https://www.mturk.com/mturk/privacynotice</a> or the Participation Agreement <a href="https://www.mturk.com/mturk/conditionsofuse" rel="nofollow">https://www.mturk.com/mturk/conditionsofuse</a>
评论 #5332107 未加载
评论 #5332089 未加载
krilnon大约 12 年前
Static page: <a href="http://groups.csail.mit.edu/uid/crowdresearch/mechanical-turk-workers-are-not-anonymous/" rel="nofollow">http://groups.csail.mit.edu/uid/crowdresearch/mechanical-tur...</a>
lnanek2大约 12 年前
Not particularly a new finding. Heck we've had posts where people had the turks post pictures and details of themselves too, and many times turks have to create a profile on site x as part of their job...
noagendamarket大约 12 年前
Amazon should use bitcoin for mechanical turk.
est大约 12 年前
site's gone... any cache?