Researchers from MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL) trained a neural network to recognize materials (e.g., metal grate, plants, concrete sidewalk) being hit with a drumstick, and synthesize sounds to accompany the actions. It did well enough to fool humans into thinking the sounds were real.
From the abstract:
Objects make distinctive sounds when they are hit or scratched. These sounds reveal aspects of an object's material properties, as well as the actions that produced them. In this paper, we propose the task of predicting what sound an object makes when struck as a way of studying physical interactions within a visual scene. We present an algorithm that synthesizes sound from silent videos of people hitting and scratching objects with a drumstick. This algorithm uses a recurrent neural network to predict sound features from videos and then produces a waveform from these features with an example-based synthesis procedure. We show that the sounds predicted by our model are realistic enough to fool participants in a "real or fake" psychophysical experiment, and that they convey significant information about material properties and physical interactions.
As AI improves, the mystery of consciousness interests more programmers and physicists.
Neural nets are starting to wake up. These pickup lines, generated by a neural net maintained by research scientist Janelle Shane are much more interesting than standard pickup lines. Are you a 4loce? Because you’re so hot! I want to get my heart with you. You are so beautiful that you know what I mean. […]
In her spare time, University of California, San Diego engineer Janelle Shane trained a neural network to generate recipes for new dishes. Informed by its reading of existing recipes, the neural network did improve over time yet it’s clearly not quite ready for Iron Chef. Here are two recipes from her Tumblr, Postcards from the […]
Bamboo has lots of uses beyond just being panda food. Things like bikes, roads, scaffolding, and musical instruments are made from the fast-growing grass. But unless you are participating in a tropical-themed LARP, you probably wouldn’t want a shirt made from bamboo stalks. So why do bamboo bed sheets make any sense? Because yarn extracted from […]
If you want to work in tech, but don’t have any desire to code web apps to help businesses sell things to other business, you might want to consider a career in cybersecurity. Judging from the apparent complete infiltration of Russian hackers in American cyberspace, it seems fair to speculate that there’s a major shortage of […]
All moms are different. But all moms like getting flowers on Mother’s Day, and that’s a fact (not, however a fact we can document in any fashion.) Instead of getting chewed out for forgetting to call her on the second Sunday of May, you can take care of it ahead of time with Teleflora’s flower […]