On Twitter's engineering blog, a fascinating description of how Twitter uses a blend of machine intelligence and Mechanical Turk tasks to figure out, in real time, what is going on in the world:
Before we delve into the details, here's an overview of how the system works.
First, we monitor for which search queries are currently popular.
Behind the scenes: we run a Storm topology that tracks statistics on search queries.
For example, the query [Big Bird] may suddenly see a spike in searches from the US.
As soon as we discover a new popular search query, we send it to our human evaluators, who are asked a variety of questions about the query.
Behind the scenes: when the Storm topology detects that a query has reached sufficient popularity, it connects to a Thrift API that dispatches the query to Amazon's Mechanical Turk service, and then polls Mechanical Turk for a response.
For example: as soon as we notice "Big Bird" spiking, we may ask judges on Mechanical Turk to categorize the query, or provide other information (e.g., whether there are likely to be interesting pictures of the query, or whether the query is about a person or an event) that helps us serve relevant Tweets and ads.
Finally, after a response from an evaluator is received, we push the information to our backend systems, so that the next time a user searches for a query, our machine learning models will make use of the additional information. For example, suppose our evaluators tell us that [Big Bird] is related to politics; the next time someone performs this search, we know to surface ads by @barackobama or @mittromney, not ads about Dora the Explorer.
In the wake of the Permanent Court of Arbitration ruling that China had been stealing islands in the South China, the Xi Jinping administration’s propaganda machine went into overdrive to whip up patriotic sentiment in China, with a massive wave of anti-American and anti-Japanese sentiment.
Encrypted Media Extensions (EME), part of a DRM system that’s being standardized at the World Wide Web Consortium (W3C), marks the first instance in which a W3C standard will fall under laws like the DMCA, which let companies threaten security researchers with criminal and civil liability just for disclosing the defects in these products.
The day that the Permanent Court of Arbitration ruled that China had been stealing islands in the South China Sea, the Chinese Communist Party Youth League shared this viral video of young Chinese patriots saying “South Sea arbitration, who cares?”
Looks like all of your potential employers are hiring candidates with programming skills (which you don’t have). With all of the languages out there today, it’s tough to know where to start.With the Complete Front-End to Back-End Coding Bundle, you can beef your resume up in all the right places, no confusion necessary. This package of […]
Those of us who love music wish we could listen to it 24/7. But it’s impossible when we’re trying to converse with our friends, or when are swimming in the local pool.That is, until now. The KOAR Bone Conduction Bluetooth Headset, now 48% off, has changed the audio game.Made with lightweight titanium memory metal, this headset boasts patented bone conduction technology to transport sound […]
It’s one thing to enjoy dinner at home and a nice glass of Cabernet Sauvignon with your best friend, Netflix, but it’s another thing entirely to make that meal from scratch and get that wine delivered right to your doorstep.But what if we told you there’s a way to make this possible? To keep your social life, […]