Using the Mechanical Turk to validate petition signatures

Jeff sez:

To qualify our initiative for the ballot in Seattle, we need the signatures of more than 20,600 registered city voters. This means we're going to have approximately 2,000 - 3,000 pages of handwritten petition forms.
We wrote two Mechanical Turk tasks to digitally capture the names, addresses and emails of petition signers and then to have workers verify what percentage of the signers are actually registered to vote in Seattle (as only these voters count towards our ballot qualification requirements).
Paying anonymous Internet-based Mechanical Turk workers does raise some murky questions related to outsourcing vs. local employment. And, since our Initiative is aimed at re-establishing democracy and social justice, we are sensitive to this. Thus far, we've just experimented with the Turk system. We're not sure yet if we'll use it in whole. And, if we do, we're going to be very thoughtful about how much we pay for each task.

Jeff's project is "initiative 103":

Initiative 103 will change the law in Seattle to:
* Ban corporate spending on elections, reversing Citizens United
* Ban corporate lobbying except in public forums
* Strip Corporate Personhood and judge-made corporate "Constitutional" rights

Using Mechanical Turk to Digitize Handwritten Petition Forms and Validate Voter Registrations

I write books. My latest is a YA science fiction novel called Homeland (it's the sequel to Little Brother). More books: Rapture of the Nerds (a novel, with Charlie Stross); With a Little Help (short stories); and The Great Big Beautiful Tomorrow (novella and nonfic). I speak all over the place and I tweet and tumble, too.

MORE: mechanical turk • politics • web theory

More at Boing Boing

Eurovision 2013: An American in London

The technology that links taxonomy and Star Trek

autark

it raises more than just questions about local employment… what about privacy for the people signing the petition? often those petitions include address info to verify your eligibility. can you ensure data privacy when hiring somebody anonymously on the internet to verify identity?

knowing this would make me wary of signing any petitions.
autark

it raises more than just questions about local employment… what about privacy for the people signing the petition? often those petitions include address info to verify your eligibility. can you ensure data privacy when hiring somebody anonymously on the internet to verify identity?

knowing this would make me wary of signing any petitions.
- zartan
  
  Who cares? Your name and address are already freely available in the phone directory etc. The reflexive anxiety about identity theft and privacy is often overblown.
  - autark
    
    wrong, I’m not in any phone directory, it’s not 1950.
  - theophrastvs
    
    I cares. er… ‘care’.
  - zartan
    
    but WHY do you care?
    
    your signature on the petition is a matter of public record.
    - theophrastvs
      
      yes… so quickly and efficiently discover every petition that i’ve signed since 1993 and cross correlate that with my library check-out record and credit-card info. you can’t, “quickly and efficiently”, can you? and i put it to you that it is still a difficult matter if you’re “one of them” (nebulous designation, i know, but we each have our own “them”). do you want to make it easier for your “them”? even Kafka’s castle dwellers bemoaned the amount of paperwork.
    - theophrastvs
      
      yes… so quickly and efficiently discover every petition that i’ve signed since 1993 and cross correlate that with my library check-out record and credit-card info. you can’t, “quickly and efficiently”, can you? and i put it to you that it is still a difficult matter if you’re “one of them” (nebulous designation, i know, but we each have our own “them”). do you want to make it easier for your “them”? even Kafka’s castle dwellers bemoaned the amount of paperwork.
      - zartan
        
        And having some random person corroborate your name against a record of your address will change this how?
        
        come to think of it, what do you think the name is being matched to? Probably some sort of existing record of your address, huh?
- LikesTurtles
  
  The petitions are almost always a matter of public record. I guess you can feel better knowing that typically the only ones who will exercise their right to see the petitions are operatives of the political parties rather than mechanical turk users whose eyes are probably wanting to pop out of their head from the mind numbingly boring task they’re subjected to.
  - Antinous / Moderator
    
    Similarly, voter registration info is publicly available. Any “candidate” can get a list of e-mail addresses.
Joe Buck

Why bother with this? The usual approach signature gatherers take is to collect substantially more signatures than they need, so they still have enough after invalid signatures are disqualified. How many extra do you typically need? Just ask organizations who’ve done signature-gathering in your area before. It’s less expensive and less privacy-intrusive to just get more signatures, and Mechanical Turk is piece labor that pays people sub-minimum wage anyway.
- theophrastvs
  
  Many petitions pay the gatherers by the number of signers they get. Lately there’s been a more than few gatherers for whom a substantial number of the pages after the first few are largely fictitious. To the degree that there have been recent cases where a substantial proportion of all the signatures are made-up and/or copies-of-other-petition drives. This is the problem this sort of effort may address. That is, one might have to collect a lot more than “more than you need”. …against, ‘diminishing returns’, i think.
SamSam

Aren’t all the signatures going to need to be re-validated after they’re turned in? I don’t imagine the Mechanical Turk system will allow anyone to notarize the results.

So if they’re going to be re-validated anyway, why not do as Joe Buck says and just get n% more signatures?

Possible reason off the top of my head: n% varies wildly across different ballot initiatives, and the extra cost to gather 30% more signatures instead of 10% more signatures is such that it’s worth spending money to check that maybe you’re fine after just 10%.
- LikesTurtles
  
  There may also be a significant turn around time for the validation process after the petitions are turned in. If you usually go 8% over for safety but this time you get 11% bad entries, you need to know as soon as possible so you can get the additional required signatures before the deadline.
- LikesTurtles
  
  There may also be a significant turn around time for the validation process after the petitions are turned in. If you usually go 8% over for safety but this time you get 11% bad entries, you need to know as soon as possible so you can get the additional required signatures before the deadline.
Jeff Reifman

In response to the above comments, all the petitions will be public record in a few months when we turn them in. We’re NOT paying any signature gatherers – this is an all volunteer effort. So, it’s not easy to just gather more signatures. One alternative we’ll consider is asking volunteers to digitize petitions but Turk is a more automated/on demand option.
http://imcravingpresidency.tumblr.com/ SedanChair

Paying anonymous Internet-based Mechanical Turk workers does raise some murky questions related to outsourcing vs. local employment

God, I am sick of this kind of mealy-mouthed language. It doesn’t “raise murky questions,” it represents a very clear and obvious moral and ethical failure in direct contradiction to your ideals.
- Joe Buck
  
  Exactly. Using Mechanical Turk says you’re happy to exploit the unemployed and the poor to do piece labor for you for far less than minimum wage. This is not a progressive thing to do.
  - Joe Buck
    
    As an alternative, you could ask for financial contributions that would allow you to hire someone to check signatures and pay that someone a living wage. But increasingly, even progressive nonprofits pay the staff that does the work little to nothing but pay the director six figures.
  - Joe Buck
    
    As an alternative, you could ask for financial contributions that would allow you to hire someone to check signatures and pay that someone a living wage. But increasingly, even progressive nonprofits pay the staff that does the work little to nothing but pay the director six figures.
- travtastic
  
  “We’re a progressive organization, so we understand the ethical issues here. But we’re going to go ahead and pay people in India $3 an hour to work for us anyway. Please sign our petition!”
http://www.facebook.com/jreifman Jeff Reifman

Sedan, Joe, well said. As I’ve said, we did this as a proof concept thus far and haven’t chosen to implement it broadly.

Some of the steps here may be applied to volunteers as well … e.g. we can host the signatures, use google doc spreadsheets and have specific volunteers do online tasks.

If you’d like to contribute funds for us to hire locally, you can do this by clicking the donation button on the right sidebar of http://i103.org.
- travtastic
  
  Why would outsourcing jobs to skirt minimum wage laws even be a concept in the first place? You’re aware that the tech novelty of the approach doesn’t affect the ethics of the approach, right?
  - http://www.facebook.com/jreifman Jeff Reifman
    
    Trav, no one said we would pay less than minimum wage if we do this.
    - travtastic
      
      Just that you need specific contributions to pay people locally. If you’re going to pay a reasonable wage, then why do you need Turks at all? The only difference is losing direct control over the process.
      - http://www.facebook.com/jreifman Jeff Reifman
        
        The automation and on demand nature of Turks is either than training and deploying volunteers … and in the absence of funding for local workers (or office space)… MTurk is a compelling option. It’s also ideal for repetitive tasks that you wouldn’t want one person to be forced to do 20,000 times … but are fine to have 20,000 people do once.
      - travtastic
        
        It’s also ideal for repetitive tasks that you wouldn’t want one person to be forced to do 20,000 times … but are fine to have 20,000 people do once.
        
        I wish you had just said that in the first place. I would highly suggest you do your research. You’re going to find out pretty quickly that that’s not how crowdsourced-labor works, in reality. You’re not going to have masses of bored kids and the unemployed chipping away at your problem, you’re going to have an outsourcing/insourcing firm in India or China paying employees poverty wages to do it for you.
http://www.facebook.com/people/Tim-Quinn/528943941 Tim Quinn

Maybe some civics classes before worrying abut the workers. Will a petition, initiative, what have you, passed or not, have any power over a supreme court decision? No.
http://www.facebook.com/jreifman Jeff Reifman

Tim, take a look at http://envisionseattle.org/2012/04/pacific-northwest-emerging-as-a-stronghold-for-a-new-kind-of-activism.html and http://envisionseattle.org/2012/05/supreme-court-approvals-drop-to-25-year-low-65-percent-disapprove-of-citizens-united.html