On Twitter's engineering blog, a fascinating description of how Twitter uses a blend of machine intelligence and Mechanical Turk tasks to figure out, in real time, what is going on in the world:
Before we delve into the details, here's an overview of how the system works.
First, we monitor for which search queries are currently popular.
Behind the scenes: we run a Storm topology that tracks statistics on search queries.
For example, the query [Big Bird] may suddenly see a spike in searches from the US.
As soon as we discover a new popular search query, we send it to our human evaluators, who are asked a variety of questions about the query.
Behind the scenes: when the Storm topology detects that a query has reached sufficient popularity, it connects to a Thrift API that dispatches the query to Amazon's Mechanical Turk service, and then polls Mechanical Turk for a response.
For example: as soon as we notice "Big Bird" spiking, we may ask judges on Mechanical Turk to categorize the query, or provide other information (e.g., whether there are likely to be interesting pictures of the query, or whether the query is about a person or an event) that helps us serve relevant Tweets and ads.
Finally, after a response from an evaluator is received, we push the information to our backend systems, so that the next time a user searches for a query, our machine learning models will make use of the additional information. For example, suppose our evaluators tell us that [Big Bird] is related to politics; the next time someone performs this search, we know to surface ads by @barackobama or @mittromney, not ads about Dora the Explorer.
The Ecuadoran Embassy in London has confirmed Wikileaks’ accusation that it terminated Julian Assange’s access to its wifi network because it disapproved of Assange and Wikileaks’ “intervention in the affairs of other states” by publishing material pertaining to the impending US election.
The UK government says it wants to stop people under 18 from looking at pornography, and so it’s going to make all the porn sites operating in Britain collect some kind of age-verification in order to make this happen, on pain of being blocked by the UK’s Great Firewall.
This Python Mega Course will help you learn to code by teaching you to build 10 real-world apps that each highlight a unique use of Python.Job prospects for coders are still growing steadily—and with Python being one of the most popular coding languages out there today, it’s important for job seekers to demonstrate a widespread understanding of the […]
The Atmos R2 may be bigger than the brand’s previously-released vapes, but we argue that in this case it’s definitely a good thing. A bigger heating chamber means more room for packing it full. And the bigger battery means longer, more fulfilling vape sessions. In fact, you can use the Atmos R2 for up to about 25 […]
These days, there is huge demand for ethical hackers. Companies pay these professionals to identify and remedy security holes in their networks before malicious hackers find and exploit them. What’s great about this is that if you love hacking or think you may love hacking, you can do it for a living and not as […]