Ben Lorica, O'Reilly's chief data scientist, has posted slides and notes from his talk at last December's Strata Data Conference in Singapore, "We need to build machine learning tools to augment machine learning engineers."
Lorica describes a new job emerging in IT departments: "machine learning engineers," whose job is to adapt machine learning models for production environments. These new engineers run the risk of embedding algorithmic bias into their systems, which unfairly discriminate, create liability, and reduces the quality of the recommendations the systems produce.
He presents a set of technical and procedural steps to take to minimize these risks, with links to the relevant papers and code. It's really required reading for anyone implementing a machine learning system in a production environment.
Another example has to do with error: once we are satisfied with a certain error rate, aren’t we done and ready to deploy our model to production? Consider a scenario where you have a machine learning model used in health care: in the course of model building, your training data for millenials (in red) is quite large compared to the number of labeled examples from senior citizens (in blue). Since accuracy tends to be correlated with the size of your training set, chances are the error rate for senior citizens will be higher than for millenials.
For situations like this, a group of researchers introduced a concept, called "equal opportunity", that can help alleviate disproportionate error rates and ensure the “true positive rate” for the two groups are similar. See their paper and accompanying interactive visualization.
We need to build machine learning tools to augment machine learning engineers [Ben Lorica/O'Reilly]
(via 4 Short Links)
Larry Tesler, the Xerox PARC computer scientist who coined the terms cut, copy, and paste, has died. Born in 1945 in New York, Tesler went on to study computer science at Stanford University, and after graduation he dabbled in artificial intelligence research (long before it became a deeply concerning tool) and became involved in the […]
Writing in Wired, Boing Boing contributor Clive Thompson discusses the rise and rise of "Edge AI" startups that sell lightweight machine-learning classifiers that run on low-powered chips and don't talk to the cloud, meaning that they are privacy respecting and energy efficient.
Yesterday's column by John Naughton in the Observer revisited Nathan Myhrvold's 1997 prediction that when Moore's Law runs out -- that is, when processors stop doubling in speed every 18 months through an unbroken string of fundamental breakthroughs -- that programmers would have to return to the old disciplines of writing incredibly efficient code whose […]
If you remember your Norse mythology (or just watched Marvel’s Thor movies), you’re probably familiar with Heimdal, the god whose ever-watchful eye was entrusted with protecting the home of the gods in Asgard. Back on Earth, Heimdal Thor is also the name of a security package from Heimdal Security, that’s actually dedicated to much the […]
Everyone’s got their nose in a phone these days, and that doesn’t seem like it’s going to change anytime soon. With the increase in mobile device and e-commerce reliance comes increased need for developers who can build the apps we’re all so glued to. In fact, employment of devs is expected to grow up to […]
Whether you love cooking at home or you swore this was going to be the year you curbed your DoorDash addiction, you know you can’t get the job done well without the proper tools on hand. For all your recipe and meal prep needs, this 3-piece Sukasu Osami Chef’s Knife set will do you right […]