Managing Mechanical Turk problems

Dolores Labs is a startup that helps companies solve their problems using Amazon's Mechanical Turk service (a piecework service that allows millions of casual laborers to contribute to "distributable judgment problems" -- like analyzing a large set of photos to find the faces in them, or classifying documents. The brief list of projects they've conducted to date is a fascinating glimpse into the kinds of problems that are tractable with "mechanical turks."
Document Classification: We helped Scribd, an online publisher, classify documents.

Sentiment: We labeled sentiment (buy/hold/sell) for stocks from posts on message boards.

Price Extraction: We extracted prices from popular shopping sites, as well as an open question to find the lowest price on a given product.

Search Relevance: We judged the relevance of a webpage for a search query.

Link (via Data Mining)

See also: Mechanical Turked color names


  1. Interesting. The colorwheel cloud view on the Dolores Labs blog is amazing.

    I went to take a look at the kind of tasks available on the Mechanical Turk website, and while some of them are the kind of tasks discussed above, there also seem to be a number of “Rephrase this sentence in your own words” tasks, and “Write comments and reviews on this review service” tasks. The applications for spammers and ‘turfers of various sorts seem pretty obvious.

    Or am I missing some non-spam-related reason for paying people to rephrase sentences such as “On your Cruisetastic vacation, you’ll enjoy such amenities as delicious drinks, outstanding entertainment, and awesome towel frogs”?

Comments are closed.