Comments on: Using the Mechanical Turk to validate petition signatures http://boingboing.net/2012/05/09/using-the-mechanical-turk-to-v.html Brain candy for Happy Mutants Sun, 26 May 2013 02:25:00 +0000 hourly 1 http://wordpress.org/?v=3.4.1 By: Antinous / Moderator http://boingboing.net/2012/05/09/using-the-mechanical-turk-to-v.html#comment-1419553 Antinous / Moderator Wed, 09 May 2012 22:03:00 +0000 http://boingboing.net/?p=159559#comment-1419553 Similarly, voter registration info is publicly available.  Any "candidate" can get a list of e-mail addresses. Similarly, voter registration info is publicly available.  Any “candidate” can get a list of e-mail addresses.

]]> By: travtastic http://boingboing.net/2012/05/09/using-the-mechanical-turk-to-v.html#comment-1419529 travtastic Wed, 09 May 2012 21:36:00 +0000 http://boingboing.net/?p=159559#comment-1419529 <blockquote>It's also ideal for repetitive tasks that you wouldn't want one person to be forced to do 20,000 times ... but are fine to have 20,000 people do once.</blockquote>I wish you had just said that in the first place. I would <i>highly</i> suggest you do your research. You're going to find out pretty quickly that that's not how crowdsourced-labor works, in reality. You're not going to have masses of bored kids and the unemployed chipping away at your problem, you're going to have an outsourcing/insourcing firm in India or China paying employees poverty wages to do it for you.

It’s also ideal for repetitive tasks that you wouldn’t want one person to be forced to do 20,000 times … but are fine to have 20,000 people do once.

I wish you had just said that in the first place. I would highly suggest you do your research. You’re going to find out pretty quickly that that’s not how crowdsourced-labor works, in reality. You’re not going to have masses of bored kids and the unemployed chipping away at your problem, you’re going to have an outsourcing/insourcing firm in India or China paying employees poverty wages to do it for you.

]]> By: Jeff Reifman http://boingboing.net/2012/05/09/using-the-mechanical-turk-to-v.html#comment-1419510 Jeff Reifman Wed, 09 May 2012 21:29:00 +0000 http://boingboing.net/?p=159559#comment-1419510 The automation and on demand nature of Turks is either than training and deploying volunteers ... and in the absence of funding for local workers (or office space)... MTurk is a compelling option. It's also ideal for repetitive tasks that you wouldn't want one person to be forced to do 20,000 times ... but are fine to have 20,000 people do once. The automation and on demand nature of Turks is either than training and deploying volunteers … and in the absence of funding for local workers (or office space)… MTurk is a compelling option. It’s also ideal for repetitive tasks that you wouldn’t want one person to be forced to do 20,000 times … but are fine to have 20,000 people do once.

]]> By: travtastic http://boingboing.net/2012/05/09/using-the-mechanical-turk-to-v.html#comment-1419505 travtastic Wed, 09 May 2012 21:24:00 +0000 http://boingboing.net/?p=159559#comment-1419505 Just that you need specific contributions to pay people locally. If you're going to pay a reasonable wage, then why do you need Turks at all? The only difference is losing direct control over the process. Just that you need specific contributions to pay people locally. If you’re going to pay a reasonable wage, then why do you need Turks at all? The only difference is losing direct control over the process.

]]> By: Jeff Reifman http://boingboing.net/2012/05/09/using-the-mechanical-turk-to-v.html#comment-1419501 Jeff Reifman Wed, 09 May 2012 21:22:00 +0000 http://boingboing.net/?p=159559#comment-1419501 Trav, no one said we would pay less than minimum wage if we do this. Trav, no one said we would pay less than minimum wage if we do this.

]]> By: Jeff Reifman http://boingboing.net/2012/05/09/using-the-mechanical-turk-to-v.html#comment-1419498 Jeff Reifman Wed, 09 May 2012 21:21:00 +0000 http://boingboing.net/?p=159559#comment-1419498 Tim, take a look at http://envisionseattle.org/2012/04/pacific-northwest-emerging-as-a-stronghold-for-a-new-kind-of-activism.html and http://envisionseattle.org/2012/05/supreme-court-approvals-drop-to-25-year-low-65-percent-disapprove-of-citizens-united.html Tim, take a look at http://envisionseattle.org/2012/04/pacific-northwest-emerging-as-a-stronghold-for-a-new-kind-of-activism.html and http://envisionseattle.org/2012/05/supreme-court-approvals-drop-to-25-year-low-65-percent-disapprove-of-citizens-united.html

]]> By: travtastic http://boingboing.net/2012/05/09/using-the-mechanical-turk-to-v.html#comment-1419496 travtastic Wed, 09 May 2012 21:20:00 +0000 http://boingboing.net/?p=159559#comment-1419496 Why would outsourcing jobs to skirt minimum wage laws even be a <b>concept</b> in the first place? You're aware that the tech novelty of the approach doesn't affect the ethics of the approach, right? Why would outsourcing jobs to skirt minimum wage laws even be a concept in the first place? You’re aware that the tech novelty of the approach doesn’t affect the ethics of the approach, right?

]]> By: travtastic http://boingboing.net/2012/05/09/using-the-mechanical-turk-to-v.html#comment-1419493 travtastic Wed, 09 May 2012 21:17:00 +0000 http://boingboing.net/?p=159559#comment-1419493 <i>"We're a progressive organization, so we understand the ethical issues here. But we're going to go ahead and pay people in India $3 an hour to work for us anyway. Please sign our petition!"</i> “We’re a progressive organization, so we understand the ethical issues here. But we’re going to go ahead and pay people in India $3 an hour to work for us anyway. Please sign our petition!”

]]> By: Tim Quinn http://boingboing.net/2012/05/09/using-the-mechanical-turk-to-v.html#comment-1419468 Tim Quinn Wed, 09 May 2012 20:55:00 +0000 http://boingboing.net/?p=159559#comment-1419468 Maybe some civics classes before worrying abut the workers. Will a petition, initiative, what have you, passed or not, have any power over a supreme court decision? No. Maybe some civics classes before worrying abut the workers. Will a petition, initiative, what have you, passed or not, have any power over a supreme court decision? No.

]]> By: Jeff Reifman http://boingboing.net/2012/05/09/using-the-mechanical-turk-to-v.html#comment-1419424 Jeff Reifman Wed, 09 May 2012 20:23:00 +0000 http://boingboing.net/?p=159559#comment-1419424 Sedan, Joe, well said. As I've said, we did this as a proof concept thus far and haven't chosen to implement it broadly.  Some of the steps here may be applied to volunteers as well ... e.g. we can host the signatures, use google doc spreadsheets and have specific volunteers do online tasks. If you'd like to contribute funds for us to hire locally, you can do this by clicking the donation button on the right sidebar of http://i103.org. Sedan, Joe, well said. As I’ve said, we did this as a proof concept thus far and haven’t chosen to implement it broadly. 

Some of the steps here may be applied to volunteers as well … e.g. we can host the signatures, use google doc spreadsheets and have specific volunteers do online tasks.

If you’d like to contribute funds for us to hire locally, you can do this by clicking the donation button on the right sidebar of http://i103.org.

]]> By: zartan http://boingboing.net/2012/05/09/using-the-mechanical-turk-to-v.html#comment-1419235 zartan Wed, 09 May 2012 18:17:00 +0000 http://boingboing.net/?p=159559#comment-1419235 And having some random person corroborate your name against a record of your address will change this how? come to think of it, what do you think the name is being matched to?  Probably some sort of existing record of your address, huh? And having some random person corroborate your name against a record of your address will change this how?

come to think of it, what do you think the name is being matched to?  Probably some sort of existing record of your address, huh?

]]> By: Joe Buck http://boingboing.net/2012/05/09/using-the-mechanical-turk-to-v.html#comment-1419217 Joe Buck Wed, 09 May 2012 18:08:00 +0000 http://boingboing.net/?p=159559#comment-1419217  As an alternative, you could ask for financial contributions that would allow you to hire someone to check signatures and pay that someone a living wage.  But increasingly, even progressive nonprofits pay the staff that does the work little to nothing but pay the director six figures.  As an alternative, you could ask for financial contributions that would allow you to hire someone to check signatures and pay that someone a living wage.  But increasingly, even progressive nonprofits pay the staff that does the work little to nothing but pay the director six figures.

]]> By: Joe Buck http://boingboing.net/2012/05/09/using-the-mechanical-turk-to-v.html#comment-1419218 Joe Buck Wed, 09 May 2012 18:08:00 +0000 http://boingboing.net/?p=159559#comment-1419218  As an alternative, you could ask for financial contributions that would allow you to hire someone to check signatures and pay that someone a living wage.  But increasingly, even progressive nonprofits pay the staff that does the work little to nothing but pay the director six figures.  As an alternative, you could ask for financial contributions that would allow you to hire someone to check signatures and pay that someone a living wage.  But increasingly, even progressive nonprofits pay the staff that does the work little to nothing but pay the director six figures.

]]> By: Joe Buck http://boingboing.net/2012/05/09/using-the-mechanical-turk-to-v.html#comment-1419206 Joe Buck Wed, 09 May 2012 18:05:00 +0000 http://boingboing.net/?p=159559#comment-1419206  Exactly.  Using Mechanical Turk says you're happy to exploit the unemployed and the poor to do piece labor for you for far less than minimum wage.  This is not a progressive thing to do.  Exactly.  Using Mechanical Turk says you’re happy to exploit the unemployed and the poor to do piece labor for you for far less than minimum wage.  This is not a progressive thing to do.

]]> By: theophrastvs http://boingboing.net/2012/05/09/using-the-mechanical-turk-to-v.html#comment-1419180 theophrastvs Wed, 09 May 2012 17:44:00 +0000 http://boingboing.net/?p=159559#comment-1419180  yes...  so quickly and efficiently discover every petition that i've signed since 1993 and cross correlate that with my library check-out record and credit-card info.  you can't, "quickly and efficiently", can you?  and i put it to you that it is still a difficult matter if you're "one of them" (nebulous designation, i know, but we each have our own "them").   do you want to make it easier for your "them"?   even Kafka's castle dwellers bemoaned the amount of paperwork.  yes…  so quickly and efficiently discover every petition that i’ve signed since 1993 and cross correlate that with my library check-out record and credit-card info.  you can’t, “quickly and efficiently”, can you?  and i put it to you that it is still a difficult matter if you’re “one of them” (nebulous designation, i know, but we each have our own “them”).   do you want to make it easier for your “them”?   even Kafka’s castle dwellers bemoaned the amount of paperwork.

]]> By: theophrastvs http://boingboing.net/2012/05/09/using-the-mechanical-turk-to-v.html#comment-1419181 theophrastvs Wed, 09 May 2012 17:44:00 +0000 http://boingboing.net/?p=159559#comment-1419181  yes...  so quickly and efficiently discover every petition that i've signed since 1993 and cross correlate that with my library check-out record and credit-card info.  you can't, "quickly and efficiently", can you?  and i put it to you that it is still a difficult matter if you're "one of them" (nebulous designation, i know, but we each have our own "them").   do you want to make it easier for your "them"?   even Kafka's castle dwellers bemoaned the amount of paperwork.  yes…  so quickly and efficiently discover every petition that i’ve signed since 1993 and cross correlate that with my library check-out record and credit-card info.  you can’t, “quickly and efficiently”, can you?  and i put it to you that it is still a difficult matter if you’re “one of them” (nebulous designation, i know, but we each have our own “them”).   do you want to make it easier for your “them”?   even Kafka’s castle dwellers bemoaned the amount of paperwork.

]]> By: SedanChair http://boingboing.net/2012/05/09/using-the-mechanical-turk-to-v.html#comment-1419159 SedanChair Wed, 09 May 2012 17:34:00 +0000 http://boingboing.net/?p=159559#comment-1419159 <blockquote>Paying anonymous Internet-based Mechanical Turk workers does raise some murky questions related to outsourcing vs. local employment</blockquote>God, I am sick of this kind of mealy-mouthed language. It doesn't "raise murky questions," it <i>represents a very clear and obvious moral and ethical failure in direct contradiction to your ideals</i>.

Paying anonymous Internet-based Mechanical Turk workers does raise some murky questions related to outsourcing vs. local employment

God, I am sick of this kind of mealy-mouthed language. It doesn’t “raise murky questions,” it represents a very clear and obvious moral and ethical failure in direct contradiction to your ideals.

]]> By: zartan http://boingboing.net/2012/05/09/using-the-mechanical-turk-to-v.html#comment-1419144 zartan Wed, 09 May 2012 17:29:00 +0000 http://boingboing.net/?p=159559#comment-1419144 but WHY do you care? your signature on the petition is a matter of public record.   but WHY do you care?

your signature on the petition is a matter of public record.  

]]> By: Jeff Reifman http://boingboing.net/2012/05/09/using-the-mechanical-turk-to-v.html#comment-1419136 Jeff Reifman Wed, 09 May 2012 17:26:00 +0000 http://boingboing.net/?p=159559#comment-1419136 In response to the above comments, all the petitions will be public record in a few months when we turn them in. We're NOT paying any signature gatherers - this is an all volunteer effort. So, it's not easy to just gather more signatures. One alternative we'll consider is asking volunteers to digitize petitions but Turk is a more automated/on demand option. In response to the above comments, all the petitions will be public record in a few months when we turn them in. We’re NOT paying any signature gatherers – this is an all volunteer effort. So, it’s not easy to just gather more signatures. One alternative we’ll consider is asking volunteers to digitize petitions but Turk is a more automated/on demand option.

]]> By: LikesTurtles http://boingboing.net/2012/05/09/using-the-mechanical-turk-to-v.html#comment-1419129 LikesTurtles Wed, 09 May 2012 17:21:00 +0000 http://boingboing.net/?p=159559#comment-1419129 There may also be a significant turn around time for the validation process after the petitions are turned in.  If you usually go 8% over for safety but this time you get 11% bad entries, you need to know as soon as possible so you can get the additional required signatures before the deadline. There may also be a significant turn around time for the validation process after the petitions are turned in.  If you usually go 8% over for safety but this time you get 11% bad entries, you need to know as soon as possible so you can get the additional required signatures before the deadline.

]]> By: LikesTurtles http://boingboing.net/2012/05/09/using-the-mechanical-turk-to-v.html#comment-1419130 LikesTurtles Wed, 09 May 2012 17:21:00 +0000 http://boingboing.net/?p=159559#comment-1419130 There may also be a significant turn around time for the validation process after the petitions are turned in.  If you usually go 8% over for safety but this time you get 11% bad entries, you need to know as soon as possible so you can get the additional required signatures before the deadline. There may also be a significant turn around time for the validation process after the petitions are turned in.  If you usually go 8% over for safety but this time you get 11% bad entries, you need to know as soon as possible so you can get the additional required signatures before the deadline.

]]> By: LikesTurtles http://boingboing.net/2012/05/09/using-the-mechanical-turk-to-v.html#comment-1419125 LikesTurtles Wed, 09 May 2012 17:18:00 +0000 http://boingboing.net/?p=159559#comment-1419125 The petitions are almost always a matter of public record. I guess you can feel better knowing that typically the only ones who will exercise their right to see the petitions are operatives of the political parties rather than mechanical turk users whose eyes are probably wanting to pop out of their head from the mind numbingly boring task they're subjected to. The petitions are almost always a matter of public record. I guess you can feel better knowing that typically the only ones who will exercise their right to see the petitions are operatives of the political parties rather than mechanical turk users whose eyes are probably wanting to pop out of their head from the mind numbingly boring task they’re subjected to.

]]> By: theophrastvs http://boingboing.net/2012/05/09/using-the-mechanical-turk-to-v.html#comment-1419113 theophrastvs Wed, 09 May 2012 17:14:00 +0000 http://boingboing.net/?p=159559#comment-1419113  Many petitions pay the gatherers by the number of signers they get.  Lately there's been a more than few gatherers for whom a substantial number of the pages after the first few are largely fictitious.   To the degree that there have been recent cases where a substantial proportion of all the signatures are made-up and/or copies-of-other-petition drives.  This is the problem this sort of effort may address.  That is, one might have to collect <b>a lot</b> more than "more than you need".  ...against, 'diminishing returns', i think.  Many petitions pay the gatherers by the number of signers they get.  Lately there’s been a more than few gatherers for whom a substantial number of the pages after the first few are largely fictitious.   To the degree that there have been recent cases where a substantial proportion of all the signatures are made-up and/or copies-of-other-petition drives.  This is the problem this sort of effort may address.  That is, one might have to collect a lot more than “more than you need”.  …against, ‘diminishing returns’, i think.

]]> By: autark http://boingboing.net/2012/05/09/using-the-mechanical-turk-to-v.html#comment-1419099 autark Wed, 09 May 2012 17:07:00 +0000 http://boingboing.net/?p=159559#comment-1419099  wrong, I'm not in any phone directory, it's not 1950.  wrong, I’m not in any phone directory, it’s not 1950.

]]> By: theophrastvs http://boingboing.net/2012/05/09/using-the-mechanical-turk-to-v.html#comment-1419100 theophrastvs Wed, 09 May 2012 17:07:00 +0000 http://boingboing.net/?p=159559#comment-1419100  I cares.   er... 'care'.  I cares.   er… ‘care’.

]]> By: zartan http://boingboing.net/2012/05/09/using-the-mechanical-turk-to-v.html#comment-1419093 zartan Wed, 09 May 2012 16:58:00 +0000 http://boingboing.net/?p=159559#comment-1419093 Who cares?  Your name and address are already freely available in the phone directory etc.  The reflexive anxiety about identity theft and privacy is often overblown. Who cares?  Your name and address are already freely available in the phone directory etc.  The reflexive anxiety about identity theft and privacy is often overblown.

]]> By: SamSam http://boingboing.net/2012/05/09/using-the-mechanical-turk-to-v.html#comment-1419066 SamSam Wed, 09 May 2012 16:39:00 +0000 http://boingboing.net/?p=159559#comment-1419066 Aren't all the signatures going to need to be re-validated after they're turned in? I don't imagine the Mechanical Turk system will allow anyone to notarize the results. So if they're going to be re-validated anyway, why not do as Joe Buck says and just get n% more signatures? Possible reason off the top of my head: n% varies wildly across different ballot initiatives, and the extra cost to gather 30% more signatures instead of 10% more signatures is such that it's worth spending money to check that maybe you're fine after just 10%. Aren’t all the signatures going to need to be re-validated after they’re turned in? I don’t imagine the Mechanical Turk system will allow anyone to notarize the results.

So if they’re going to be re-validated anyway, why not do as Joe Buck says and just get n% more signatures?

Possible reason off the top of my head: n% varies wildly across different ballot initiatives, and the extra cost to gather 30% more signatures instead of 10% more signatures is such that it’s worth spending money to check that maybe you’re fine after just 10%.

]]> By: Joe Buck http://boingboing.net/2012/05/09/using-the-mechanical-turk-to-v.html#comment-1419040 Joe Buck Wed, 09 May 2012 16:27:00 +0000 http://boingboing.net/?p=159559#comment-1419040 Why bother with this?  The usual approach signature gatherers take is to collect substantially more signatures than they need, so they still have enough after invalid signatures are disqualified. How many extra do you typically need?  Just ask organizations who've done signature-gathering in your area before.  It's less expensive and less privacy-intrusive to just get more signatures, and Mechanical Turk is piece labor that pays people sub-minimum wage anyway. Why bother with this?  The usual approach signature gatherers take is to collect substantially more signatures than they need, so they still have enough after invalid signatures are disqualified. How many extra do you typically need?  Just ask organizations who’ve done signature-gathering in your area before.  It’s less expensive and less privacy-intrusive to just get more signatures, and Mechanical Turk is piece labor that pays people sub-minimum wage anyway.

]]> By: autark http://boingboing.net/2012/05/09/using-the-mechanical-turk-to-v.html#comment-1419029 autark Wed, 09 May 2012 16:22:00 +0000 http://boingboing.net/?p=159559#comment-1419029 it raises more than just questions about local employment... what about privacy for the people signing the petition? often those petitions include address info to verify your eligibility.  can you ensure data privacy when hiring somebody anonymously on the internet to verify identity? knowing this would make me wary of signing any petitions. it raises more than just questions about local employment… what about privacy for the people signing the petition? often those petitions include address info to verify your eligibility.  can you ensure data privacy when hiring somebody anonymously on the internet to verify identity?

knowing this would make me wary of signing any petitions.

]]> By: autark http://boingboing.net/2012/05/09/using-the-mechanical-turk-to-v.html#comment-1419030 autark Wed, 09 May 2012 16:22:00 +0000 http://boingboing.net/?p=159559#comment-1419030 it raises more than just questions about local employment... what about privacy for the people signing the petition? often those petitions include address info to verify your eligibility.  can you ensure data privacy when hiring somebody anonymously on the internet to verify identity? knowing this would make me wary of signing any petitions. it raises more than just questions about local employment… what about privacy for the people signing the petition? often those petitions include address info to verify your eligibility.  can you ensure data privacy when hiring somebody anonymously on the internet to verify identity?

knowing this would make me wary of signing any petitions.

]]>