GPT-2 is a language model that was trained on 40GB of text scraped from websites that Reddit linked to and that had a Karma score of at least two. As the developers at OpenAI describe it, GPT-2 is "a large-scale unsupervised language model which generates coherent paragraphs of text, achieves state-of-the-art performance on many language modeling benchmarks, and performs rudimentary reading comprehension, machine translation, question answering, and summarization—all without task-specific training." Because the model is probabilistic, it returns a different response every time you enter the same input.
OpenAI decided not to release the 40GB-trained model, due to "concerns about malicious applications of the technology" but it released a 345MB-trained model which you can install as a Python program and run from a command line. (The installation instructions are in the DEVELOPERS.md file.) I installed it and was blown away by the human-quality outputs it gave to my text prompts. Here's an example - I prompted it with the first paragraph of Kafka's The Metamorphosis. And this is just with the tiny 345MB model. OpenAI published a story that the 40G GPT-2 wrote about unicorns, which shows how well the model performs.
In this Computerphile video, Rob Miles of the University of Nottingham explains how GPT-2 works.
If a website called Do Not Draw a Penis senses that you are trying to sketch a penis it will erase it. Otherwise, it attempts to describe what you are drawing. It’s pretty easy to sneak a penis in by having it after you draw something it identifies.
Thomas Dimson used the GPT-2 language model to make a website that creates a new word every time you refresh the page. Some of them are really good — I could imagine them entering the lexicon.
All you need to know before taking Google’s Machine Learning Foundations course is “a little bit of Python.” In this first episode, you’ll learn what machine learning is and how it works. Image: YouTube
Even though it feels like Amazon is a singular retail juggernaut crushing everybody else, you might be surprised to learn that half of Amazon’s $280 billion in revenue last year came from third-party sellers. According to numbers compiled by JungleScout, 86 percent of Amazon’s Fulfilled by Amazon (FBA) sellers were profitable last year, more than […]
Amidst all the deadly serious concern and fallout from our global battle against COVID-19, you’ve likely been forced to confront more than a few moments that you never expected to face. And you likely never felt sillier during this scary time than when you were racing all over town hoping desperately that some store had […]
“It’s better to have it and not need it than to need it and not have it.” – “Lonesome Dove,” Larry McMurtry If the past few months have taught us anything, it’s that we need to be prepared now more than ever. Emergency situations can happen quickly and there’s no telling when you may need […]