Ian Charnas has figured out a way to override Nintendo's controller to throw punches in real life and have them work inside the video game, "Punch-Out!" Because, of course he has!
My latest project is a real-life version of Nintendo Punch-Out (boxing game) where I've hacked the game so you control it with actual punches instead of a controller. It's super fun! It gives you the feeling of actually being inside the video game!
The software relies heavily on Google's TensorFlow, which is an AI platform they've developed, and on MoveNet, which is a TensorFlow model that tracks your body (AI researchers call this "pose estimation").
The hardest part of the project was hacking the game to slow the opponents. You see, punching into the air is much slower than pressing a button on a controlpad. So, the opponents were moving too fast, relatively, and it wasn't a fair fight. I had to reverse engineer the source code (a process called "disassembly") and slow each opponent down one by one. The result is a very fun and playable game that you can try yourself by visiting RealLifePunchout.com
