Pokemon GO and Interactive Dynamic Video:<p><a href="https://www.youtube.com/watch?v=9f1fCCb3hVg" rel="nofollow">https://www.youtube.com/watch?v=9f1fCCb3hVg</a>
When I first saw this (extracting audio from visual vibrations): <a href="http://news.mit.edu/2014/algorithm-recovers-speech-from-vibrations-0804" rel="nofollow">http://news.mit.edu/2014/algorithm-recovers-speech-from-vibr...</a> it seemed incredible that the signal the cameras could capture was strong enough. This work is even cooler, even more non obvious and seems closer to wizardry.
What exactly are the implications of this for construction engineering, which must already have some way of measuring vibrations?<p>I.e. does this measure something previously unmeasurable, or is it just that the visualization provides an extra channel for interpreting the data?<p>Or is it that it would make this kind of tool available to anyone with a digital camera?
Here's TED a talk about the technology <a href="http://www.ted.com/talks/abe_davis_new_video_technology_that_reveals_an_object_s_hidden_properties" rel="nofollow">http://www.ted.com/talks/abe_davis_new_video_technology_that...</a>
The implications of this technology are staggering. He doesn't even talk about online use-cases, but imagine (just as an example) if YouTube added this algorithm to all it's video library, and started allowing you to interact with videos in very different ways. Crazy stuff.
The consumer use cases are interesting but the propagandist use cases are terrifying.
Along with this (Real-time Face Capture and Reenactment):
<a href="https://www.youtube.com/watch?v=ohmajJTcpNk" rel="nofollow">https://www.youtube.com/watch?v=ohmajJTcpNk</a>
Signal Processing was that subject I found difficult and I found no reason to be interested in audio filtering, so I didn't find an incentive to work through that difficulty. Then I see applications of signal processing, like this, that goes beyond simple audio filtering, that makes me want to learn it again. Honestly, this looks like wizardry! It is no surprise, MIT is on the bleeding edge of signal processing and it mainly is due to Dr. Oppenheim . He wrote the textbook on DSP and runs the DSP Group at MIT.
Papers of their findings can be found at <a href="http://www.interactivedynamicvideo.com/publications.html" rel="nofollow">http://www.interactivedynamicvideo.com/publications.html</a>
I wonder of you could use this on car commercials and thereby deduce the actual comparative quality of construction. Assuming that better built cars would vibrate less at speed. It would be interesting to understand where the vibrations occur.<p>A question for the authors. All the video is with vibrating but otherwise stationary objects. Must the object be stationary for these tools to work?
This does look fascinating. The video is a bit showbusiness-like (for a lack of better expression).<p>A few questions if the author is on HN:
I wonder how long it takes to analyse a 5 second video? Also, it seems the algorithm only works with static images after the initial video analysis, or am I wrong? Also, how long does it take to render the "virtual" state of the object?
Wow, more incredible image processing stuff from MIT. What's really cool is how it shows how much hidden information is out there there in world to be captured within a normal camera. We just need to be aware of it and analyze it properly. For me, it just re-injects a bit of wonder into the everyday world we live in.
This might make the VRML included in MPEG4 useful. Could possibly use it to pass along the physical constraints and other data needed to make it interactive. ref: <a href="https://en.wikipedia.org/wiki/MPEG-4_Part_11" rel="nofollow">https://en.wikipedia.org/wiki/MPEG-4_Part_11</a>
I would love to see what this could do when combined with light field technology <a href="https://www.youtube.com/watch?v=xNJZHFZEkYQ" rel="nofollow">https://www.youtube.com/watch?v=xNJZHFZEkYQ</a>
So if we can use this algorithm for Structural Health Monitoring (like a bridge shown in the video) - what's the major obstacle to replace all destructive testing in mechanical engineering with this kind of algorithm?
Dedicated host name for a single algorithm, TED talk, high-production value video with professional animations, narrator talking excitedly, more kitschy examples of applications than description of the algorithm itself.<p>Yup, MIT.