"Diminished reality" software removes objects from video in real time

Via the BB Submitterator, Boing Boing reader "galoot" says:

Researchers at Technical University of Ilmenau in Germany have developed "diminished reality" software that can delete an object from live, full-motion video. The software first reduces the resolution of the object, removes the image, and improves the result (similar to using a smudge tool in Photoshop), then incrementally increases the resolution, improving the result, until the original resolution is restored. It repeats that for each frame of the video in real time, delivering the final image in 40 ms.
More here.


  1. To quote myself from another website on this topic:

    Finally, we can restore my childhood memories and eviscerate Jar-Jar from the last batch of Star Wars movies.

  2. This would be great if you had a game show where you let convicted criminals try to escape past a gauntlet of celebrity bounty hunters.

  3. “We will probably have to revert to a method common in the Middle Ages, which is eyewitness testimony. And there is probably something healthy in that.” — Kathleen Hall Jamieson

  4. I’d love to see a demo of what the algorithm does when the removed object was not on a relatively uniform background. Put that cell phone on the edge of the sink and show how it handles that data set. There’s some interesting artifacting in the asphalt fill when they remove the drain from the street. How does performance degrade when the surrounding data is more irregular?

    1. In the YouTube video linked from the original article, there’s a shot of them using it to ‘erase’ a storm drain cover from an asphalt/gravel street. The visual parallax artifacts (left behind by the ‘clone’ step producing differing results from frame to frame) are very visible. I am certain that with some small effort, those parallax artifacts would be significantly reduced.

      1. Most likely, yes. Processing capacity will no doubt continue to increase, and the analysis and processing routines of this will improve with time as well.

  5. This is effectively the same as the Macguffin that drives the plot of the novel, Whole Wide World, published by Paul McAuley from about 10 years ago. I wonder if he knows…

  6. Old news. They could already do that back in the days of the Zapruder film when they erased the guys on the Grassy Knoll.

  7. Looks ‘shopped. You can tell because of the pixels and because in the reflection of the object in the bathroom shot isn’t erased. Too bad… an object-identification version of this would be much cooler. As is, I wonder if deletion quality degrades when the angle is sufficiently different… or if deletion stops altogether if the object leaves the field of view momentarily!

  8. Are you saying, “It’s time to start running?”

    Get me the Department of Justice, entertainment division.

    Your honor, who are you going to believe, this video or the lying eyes of this flawed human?

  9. I want a “mirror” that is just a big video monitor that randomly picks out a person and erases them from the image. Fun at parties! Accuse your friends of being a vampire. (Stake dinner not included.)

  10. I’m still confused about the reducing-then-increasing-image-quality part. How does that work without a net loss of information? They aren’t claiming to pull a CSI move where they get a crisp image detailed out of a pixellated one, are they?

    1. They remove the desired object by reducing the res of the whole image in a /copy/, locating the object, and subtracting those pixels from the original – and again with a slightly less reduced resolution copy, and again with a slightly less reduced copy … it effectively subtracts the item from the original and blends the result into the surrounding background.

  11. Just this morning I was explaining to my father that we will never know for sure what happened to Connally and Kennedy (as we walked past Dealey Plaza and the Sixth Floor Museum) because technology has advanced to the point that any evidence for any given event could be convincingly forged given sufficient time and resources, and that “sufficient” is now within the reach of the budget of a small business with a week to work.

    Rather than eyewitness testimony (which is the bane of justice), admission of video and photo evidence will be subject to independent certification of the device recording the image(s) and chain of custody of the evidence.

  12. This is precisely what I wanted to see in combination with the unlogo post from a few days ago. I don’t want to see logos covered, I want them gone …

  13. My sister plays these video games in bars where there are two similar photographs and you have to identify the differences.

    Looks like they won’t be still photographs anymore. Or even necessarily made ahead of time!

  14. Next up: Software to tell you if your live feed has been altered.
    Followed by: Software to restore deleted objects.

  15. I like how it removed the box of bandaids, but not their reflection in the mirror. Could make cool special effects in a movie if used well…

Comments are closed.