Ben Lorica, O'Reilly's chief data scientist, has posted slides and notes from his talk at last December's Strata Data Conference in Singapore, "We need to build machine learning tools to augment machine learning engineers."
Lorica describes a new job emerging in IT departments: "machine learning engineers," whose job is to adapt machine learning models for production environments. These new engineers run the risk of embedding algorithmic bias into their systems, which unfairly discriminate, create liability, and reduces the quality of the recommendations the systems produce.
He presents a set of technical and procedural steps to take to minimize these risks, with links to the relevant papers and code. It's really required reading for anyone implementing a machine learning system in a production environment.
Another example has to do with error: once we are satisfied with a certain error rate, aren’t we done and ready to deploy our model to production? Consider a scenario where you have a machine learning model used in health care: in the course of model building, your training data for millenials (in red) is quite large compared to the number of labeled examples from senior citizens (in blue). Since accuracy tends to be correlated with the size of your training set, chances are the error rate for senior citizens will be higher than for millenials.
For situations like this, a group of researchers introduced a concept, called "equal opportunity", that can help alleviate disproportionate error rates and ensure the “true positive rate” for the two groups are similar. See their paper and accompanying interactive visualization.
We need to build machine learning tools to augment machine learning engineers [Ben Lorica/O'Reilly]
(via 4 Short Links)
The Googler Uprising was a string of employee actions within Google over a series of issues related to ethics and business practices, starting with the company's AI project for US military drones, then its secretive work on a censored/surveilling search tool for use in China; then the $80m payout to Android founder Andy Rubin after […]
The Bank of England has unveiled its new £50 notes, which had been earmarked to honour a distinguished British scientist, and which will feature Alan Turing, the WWII hero who discovered many of the foundational insights to both modern computing and cryptography, and whose work with the codebreakers of Bletchley Park are widely believed to […]
I used to be on the program committee for the O'Reilly Emerging Technology conferences; one year we decided to make the theme "magic" -- all the ways that new technologies were doing things that baffled us and blew us away.
Looking to upgrade that old laptop or tablet? Holiday sales may be way on the horizon, but there’s an even better way to get like-new tech for a steal. Here are 10 of our favorite deals on personal computers and gaming gear – refurbished, renewed or brand new. Acer Touchscreen 11′ Chromebook 16GB (Certified Refurbished) […]
Vape technology has been around long enough that vapers are starting to get picky about their gear. Luckily, so are we. From disposable models to cutting-edge touchscreen atomizers, there’s a vaporizer in this roundup to suit every taste. Hera 2 – World’s Most Advanced Dual-Use Vaporizer Choose between dry herb or oil extraction modes – […]
With enough practice and commitment, anyone can be a visual artist. But without the right instruction, that time spent honing your skills could seem like an eternity. If you really want to see where your talent can take you, you need sound fundamentals – and no matter what discipline or genre you lean toward, the […]