Hi, I’m Seth and I just screwed over lots of people on Mechanical Turk. My writing this is an attempt to fix that. I was negligent because I didn’t check the code that was reviewing the jobs (which rejected 70% of maybe 1000 people), and I didn’t check the Turk account’s email address ever, not once over the four months that the job has been running. The email in that account, from hundreds of people who were rejected for honest work, was full of hints that something was wrong, but I only discovered the extent of it last week.
I feel horrible about it. I do a lot of work for participatory workplaces and democracy as a route to responsible business practices. I’m a founder of a worker cooperative (http://bloomingtoncoop.org) and I’m involved in a bunch of organization and projects to raise the profile of workplace democracy (http://nasco.coop http://geo.coop/issue/9). I like to think I’m doing good things for people who work, so imagine my surprise to find that I’m a truly horrible, exploitative boss. Literally hundreds of people put in 5-20 minutes, on the promise of payment, and they got rejected with no explanation, sometimes incorrectly. Here are some testimonials:
I’m not sure how to fix all this. I want to apologize, but any apology is empty until after I’ve actually fixed things. The code is fixed, and I’m working with a friend to figure out who should have gotten paid, and then pay them. Next step after that is to ask Amazon to unreject everyone. And beyond that, what? Damages? (UPDATE: we’ve sent out payments with an apology and a bonus. We still have to ask Amazon about unrejections. I can now offer a good apology: I’m sorry for rejecting your honest work)
Out of 850 jobs, 300 were accepted, 122 were falsely rejected. Most of the remaining 50% didn’t fill the survey out completely. But even though they were possibly fairly rejected, I could still have done more to prevent that: like warning people that they are submitting an incomplete HIT with a few simple lines of PHP. I’ve thought about it a bunch, but your comments will help me figure out what is the right thing to do moving forward. The best ideas have come from other Turkers who got rejected on this badly designed HIT, like attention checks, brief qualifying HITs, and captchas that help eliminate bots and people who are inattentive. More ideas?
If you are interested, I can offer the original questionnaire, and the results from everyone who took it (anonymized).
I’m happy to answer any questions about it.
I’m going to do my best to make up for it, but I don’t blame anyone for mistaking my negligence for maliciousness. If you aren’t feeling right yet, let me know what I can do.
