Recover Sound Only from Images Without Microphone

Watch on YouTube

First, if you don’t know the Veritasium Youtube Channel you really should check out some Videos from Derek Muller.

TL;DW

In that video Dr. Abe Davis gets interviewed about his research on restoring sound from images, without any microphone.

He captured the vibration information in Objects through video. The movement in objects is as tiny as about a micron, smaller than a pixel. So the algorithm must also understand the characteristics of the image.

With that algorithm and some post-processing (cleaning) he is able to restore sound just from a video.

Note: The original article included three inline images showing the process: clipping a video region, examining pixel movement, and the recovered sound waveform. These images were stored at technology/images/2021.107-image-sound-*.jpg.