AlphaGo Zero shows that corporate R&D has starved basic research in favor of safe bets and tinkering at the margins

An NBER paper from 2015 tracks the decline in corporate spending on basic science in R&D, which has become a practical, application-focused line-item in corporate budgets, creating a safe, predictable, and slow cycle of innovation.

Deepmind's AlphaGo Zero — which taught itself to play a remarkable game of Go in just 72 hours — is an ironic poster child for this phenomenon. AlphaGo is part of a long-term shift in AI research from generating machine comprehension to "machine learning" that is just a fancy form of statistical analysis, a brute-force approach that relies on ingesting lots of human decisions and making statistical observations that can be used as predictions about the future.

At the same time, universities have also become more short term and more focused on practicalities rather than basic science.

Perhaps it is naive to simply exhort companies to spend more on fundamental research — but somebody has to. One interesting approach is for governments to fund "innovation prizes" for breakthroughs. Such prizes mobilise public funds and public goals while deploying the agility and diversity of private sector approaches. But such prizes only work in certain situations.

Professional sport has made fashionable the practice of "marginal gains" — rapid optimisation in search of the tiniest edge. It turns out that corporate research took the same turn decades ago. There is nothing wrong with marginal improvements, but they must not be allowed to crowd out more speculative research. Science is a deeper, messier practice than sport. We must continue to devote time, space and money to bigger, riskier leaps.

What AlphaGo Zero teaches us about what's going wrong with innovation
[Tim Harford]

(Image: Michael Branson Smith)