GPT-2 is a language model that was trained on 40GB of text scraped from websites that Reddit linked to and that had a Karma score of at least two. As the developers at OpenAI describe it, GPT-2 is "a large-scale unsupervised language model which generates coherent paragraphs of text, achieves state-of-the-art performance on many language modeling benchmarks, and performs rudimentary reading comprehension, machine translation, question answering, and summarization—all without task-specific training." Because the model is probabilistic, it returns a different response every time you enter the same input.
OpenAI decided not to release the 40GB-trained model, due to "concerns about malicious applications of the technology" but it released a 345MB-trained model which you can install as a Python program and run from a command line. (The installation instructions are in the DEVELOPERS.md file.) I installed it and was blown away by the human-quality outputs it gave to my text prompts. Here's an example - I prompted it with the first paragraph of Kafka's The Metamorphosis. And this is just with the tiny 345MB model. OpenAI published a story that the 40G GPT-2 wrote about unicorns, which shows how well the model performs.
In this Computerphile video, Rob Miles of the University of Nottingham explains how GPT-2 works.
In Q2 2018, Chinese investors sank $2.87b into AI startups; in Q2 2019, it was $140.7m.
I used to be on the program committee for the O'Reilly Emerging Technology conferences; one year we decided to make the theme "magic" -- all the ways that new technologies were doing things that baffled us and blew us away.
In the latest Adafruit video (previously) the proprietors, Limor "ladyada" Friend and Phil Torrone, explain the basics of machine learning, with particular emphasis on the difference between computing a model (hard) and implementing the model (easy and simple enough to run on relatively low-powered hardware), and then they install and run Tensorflow Light on a […]
Whether you’re using them for next-level selfies or steady tracking shots, gimbals are a must for anyone who wants to maximize the potential of these powerful smartphone cameras we’re all carrying around. But those smartphones are also supposed to be portable, and let’s face it: Gimbals tend to offset that advantage. Weighing in at just […]
It’s too hot for yard sales, but hey: The internet is here for you. Here are the top ten deals on some of the Boing Boing Store’s best gear, just in time for summer. It’s everything from grills to security cameras to MacBook Pros, and they might be as low as they’re ever going to […]
When it comes to getting stats and ideas across quickly, there’s still nothing like a good slide presentation. But the critical word here is “good” – not 20 slides all thrown together with the same stock PowerPoint template. Whether it’s a crucial pitch for a new business or a quarterly report, Slideshop can be a […]