Researchers from MIT and Meta have demonstrated a computer vision system that analyzes shadows to create 3D models of scenes, including areas that are hidden from view.
The name of the AI-powered system—PlatoNeRF—was inspired by Plato's allegory of the cave where prisoners can perceive the outside world only by looking at shadows on the cave walls.
The approach merges machine learning and lidar (light detection and ranging)—technology found in many new smart phones— to create more accurate 3D models than other AI methods, especially in tricky lighting conditions. According to the researchers, better 3D models could improve the safety of autonomous vehicles and also help mobile robots locate objects in messier environments. In the VR and AR realm, the PlatoNeRF technique could make it easier to build models of real places because it gathers data from a single camera position rather than requiring a user to move throughout a space taking measurements.
"The work shows how clever algorithms can enable extraordinary capabilities when combined with ordinary sensors — including the lidar systems that many of us now carry in our pocket," comments University of Toronto computer scientist who was not involved in the research.
Previously:
• Optimizing a peanut butter and banana sandwich using machine learning and computer vision
• High-speed computer vision system beats you at Rock, Paper, Scissors every time
