On Twitter's engineering blog, a fascinating description of how Twitter uses a blend of machine intelligence and Mechanical Turk tasks to figure out, in real time, what is going on in the world:
Before we delve into the details, here's an overview of how the system works.
First, we monitor for which search queries are currently popular.
Behind the scenes: we run a Storm topology that tracks statistics on search queries.
For example, the query [Big Bird] may suddenly see a spike in searches from the US.
As soon as we discover a new popular search query, we send it to our human evaluators, who are asked a variety of questions about the query.
Behind the scenes: when the Storm topology detects that a query has reached sufficient popularity, it connects to a Thrift API that dispatches the query to Amazon's Mechanical Turk service, and then polls Mechanical Turk for a response.
For example: as soon as we notice "Big Bird" spiking, we may ask judges on Mechanical Turk to categorize the query, or provide other information (e.g., whether there are likely to be interesting pictures of the query, or whether the query is about a person or an event) that helps us serve relevant Tweets and ads.
Finally, after a response from an evaluator is received, we push the information to our backend systems, so that the next time a user searches for a query, our machine learning models will make use of the additional information. For example, suppose our evaluators tell us that [Big Bird] is related to politics; the next time someone performs this search, we know to surface ads by @barackobama or @mittromney, not ads about Dora the Explorer.
The ACLU is suing to repeal parts of the Computer Fraud and Abuse Act (CFAA), a 1980s-vintage hacking law that makes it a felony to “exceed authorization” on a remote computer, and which companies and the US government have used to prosecute researchers who violated websites’ terms of service.
June’s Decentralized Web Summit at San Francisco’s Internet Archive was a ground-breaking, three-day combination of workshops, lectures, demos and a hackathon, all aimed at figuring out how to restore the decentralized character of the early internet — and keep it that way.
Maciej Cegłowski (previously) keynoted the Society for the Advancement of Socio-Economics conference with a characteristically brilliant speech about the “moral economy of tech” — that is, the way that treating social problems like software problems allows techies to absolve themselves of the moral consequences of their actions and the harms that result.
The realm of web development is constantly evolving. New platforms, languages, and processes materialize all the time, so staying on top of all that innovation is a tall order.Whether you’re brushing up on new tricks, starting from scratch, or just looking to make your own website a little jazzier, Rob Percival’s new Complete Web Developer Course 2.0 (now […]
Folks used to rely on alarms to protect their home – and before that, the family dog. Now, anyone looking to guard their homes can choose from some high-tech options, including the Amaryllo iCamPRO FHD Home Security Camera (now just $219 in the Boing Boing Store).In fact, this 2015 CES “Best of Innovation” award-winner boasts so many features, it’s […]
If you want a quality vaping experience, it’s usually going to cost you. Vaporizers that deliver a fast, controlled burn will set you back up to $300, which is why the FEZ Vaporizer (now just $99) is an absolute steal.The FEZ dry herb pen does everything that more expensive models handle at a reduced price. It heats up […]