Animate Anyone: Image-to-video synthesis for character animation

Just wow. This is the first time I&#x27;ve seen AI generate convincing human movement. (And the folds of fabric in dresses seem to move realistically too!)<p>Of course, the actual movement skeleton is coming from presumably real motion-capture, but still.<p>I&#x27;m curious what the current state is of <i>generating</i> the movement skeletons, which obviously has a ton of relevance to video games. Where&#x27;s the progress in models where you can type &quot;a burly man stands with erect posture, then crouches down and sneaks five steps forward, then freezes in fear&quot; and output an appropriate movement skeleton?

I&#x27;ll mention Corridor Crew&#x27;s <i>Rock, Paper, Scissors</i> [0] as the previous state of the art in terms of character animation&#x2F;style transfer&#x2F;etc. using AI tooling.<p>I imagine this will make the barrier to entry for animated stuff very, very low. You literally only need a character sheet.<p>Also, the creep factor for AI girlfriends has ratcheted up another notch.<p>[0] <a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=7QAGEvt-btI" rel="nofollow noreferrer">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=7QAGEvt-btI</a>

That’s quite astonishing. In just a few years this might even be generalized to work for characters other than conventionally attractive young women.

Why would you publish your findings on Github of all places, but not release any code? I think this trend is really weird.

I&#x27;m just waiting for the tool or toolchain where I can take a manga that I like that doesn&#x27;t have an anime, and get a season or two out to watch when I feel like it rather than wait for it to get an official release.<p>Bonus points if I can let the tool ingest season 1 or an OVA of said material where a season 2 is never going to come (looking at you &quot;No Game, No Life&quot;)

Just imagine in a few years there will be a site like YouTube aka videogpt where all the videos are created on the fly, like chatgpt does for text.<p>From step by step repairing my electronics to learning about science it will all be customised according to my learning level and focus on things I want to know more. And hopefully without that a word from our sponsors section.

Just imagine when this merges with 3D modeling and VR.<p>The VR pr0n, the video games with dynamic AI characters. Dead actors and historical figures resurrected into movies and education.<p>I&#x27;m not so scared about my future nursing home now.

Pretty huge breakthrough. Hopefully we&#x27;ll be able to access this soon. Between this, SVD, and others, 2024 is going to be the year of AI Video.

The choice of test images here feels super inappropriate. Surely there&#x27;s a more diverse and standardized dataset for benchmarking this task than what was chosen here.<p>I quote similar criticism from Dianne P. O&#x27;Leary:<p>&gt; Suggestive pictures used in lectures on image processing ... convey the message that the lecturer caters to the males only. For example, it is amazing that the &quot;Lena&quot; pin-up image is still used as an example in courses and published as a test image in journals today.

This looks impossibly good. I think these samples are cherry-picked and I also think the system is essentially overfit on these datsets and would not generalize to anything even slightly different. I want to see their failure cases! Lack of failure cases is a red flag.<p>Still, though, it could be useful in its current form, and making a more general system might mostly be a matter of collecting appropriate training data. Impressive work, just needs a more realistic presentation.

This is absolutely insane. The DreamPose output they compare themselves to is less than one year old.<p>It&#x27;s funny to go back to the first avocado chair or deep dream images that wowed me just a couple years ago.<p>I can&#x27;t help but feel lost in the pace of tech.

why is everything from this space so horny. I&#x27;m not sure if I like it or not. Obviously it&#x27;s a bit problematic but on the other hand i welcome when people are honest about their intentions like this.

I&#x27;m as interested as anyone in the methods behind this and find deep learning just continues to amaze and I really enjoy working in the AI space. Having said that, seeing that we now essentially have the capability to synthesize videos of people doing things they haven&#x27;t done and saying things they haven&#x27;t said, I really do worry about the potential for abuse of this technology once it&#x27;s really perfected. What does it mean for society when can no longer enter recordings into evidence?

I love how Chinese image synthesis studies fixate on anime. I&#x27;m sure others do to, but they&#x27;re particularly weeby.

What&#x27;s the trigger for all the diffusion video projects this week? Are they all at the same conference, or did one trigger everyone to rush to publish? I&#x27;m grateful but curious. Animating still pictures from my kids&#x27; books is my goal and it seems close to plausible just from this week&#x27;s advances.

Apologies for the lazy question... But with so many incredible models being released, how are you all keeping up? Are you mostly trying them all out on the web or are you installing each of them locally?

Very impressive quality.

Is this considered as absusing the GitHub CDN? The project is not open source, butis using GitHub for hosting so many videos.

I predict that Meta will release another open source version of this, given their advanced research in pose estimation.

Page is crashing my mobile chrome tab…

How do you generate the movement data?

How did they make the girls smile as she dances in the last ones?

Why even have a GitHub?

This is why actors are striking.

Wow this is uncanny

Great, this looks perfect as a way to sell more fast fashion, the 2nd worst polluting industry in the world. So many better applications of this technology and instead they showcase dancing boring body stereotypes.