Real time image animation in opencv using first order model

I'm a huge fan of this kind of practice, where the code for a paper is all located in a single public repository with build instructions, along with directions for how to cite it. Obviously, it's a little tough to do with some more data-intensive sources (besides GH hosting limits, no one really wants to download 100G of data if they're just trying to clone a repository), but this kind of thing sets a high standard for reproducibility of published results.

I&#x27;m working with same model, but in a real-time pipeline developed with GStreamer, Rust and PyTorch:<p><a href="https:&#x2F;&#x2F;twitter.com&#x2F;rozgo&#x2F;status&#x2F;1255961525187235842" rel="nofollow">https:&#x2F;&#x2F;twitter.com&#x2F;rozgo&#x2F;status&#x2F;1255961525187235842</a><p>Live motion transfer test with crappy webcam:<p><a href="https:&#x2F;&#x2F;youtu.be&#x2F;QVRpstP5Qws" rel="nofollow">https:&#x2F;&#x2F;youtu.be&#x2F;QVRpstP5Qws</a>

Very cool, reminds me of Avatarify, which is also based upon the First Order Model work:<p><a href="https:&#x2F;&#x2F;github.com&#x2F;alievk&#x2F;avatarify" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;alievk&#x2F;avatarify</a>

Pretty cool. Reminds me of <a href="https:&#x2F;&#x2F;github.com&#x2F;yemount&#x2F;pose-animator" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;yemount&#x2F;pose-animator</a><p>I would use it if there was a JavaScript port.

How can it generate teeth that look like they fit the picture ???

Is no one else deeply afraid of this future?

Looks like the file mentioned in this step<p>&gt; gdown --id 1wCzJP1XJNB04vEORZvPjNz6drkXm5AUK<p>Is no longer accessible (too many downloads in too short a time)<p>Edit: For anyone else with the same problem, the file in question is &quot;vox-cpk.pth.tar&quot; which can be found in various places on the internet.

The google colab version is not really real-time, is that correct? It loads pre-recorded video. I guess that is because it is not easy to add realtime feed from camera into browser notebook or what are the limitations there?

The paper &amp; final models don&#x27;t to justice for detailed outputs though, but this is still a great model for datasets with no annotations per se.

does anyone know if using this tool to generate a music video of famous pictures singing a song would violate any copyrights? it seems like a fun exercise.

very neat! You can crop and convert to mp4 using ffmpeg: ffmpeg -i test.avi -filter:v &quot;crop=250:250:260:0&quot; out.mp4

one of the authors is at snap. inquiring minds want to know: will this soon be available in snap camera?

Really cool, but I hoped to see C++ code for OpenCV, not python