Video to video with Stable Diffusion

I was going to get a 4090 this year, but I just don&#x27;t think 24gb VRAM is going to bee enough in the short term future(for AI related stuff).<p>Ended up getting a 3060 for 1&#x2F;4 of the cost, and I&#x27;m planning on using&#x2F;paying for Colab until some 6090 comes out with 128gb vram.<p>Something that kind of bothers me that I don&#x27;t understand. Why is there such an obsession about having small computers&#x2F;servers? I don&#x27;t care if I have to put my computer in the basement because its the size of a closet or two. Maybe there is an EE issue, like too much current or EMF if you make a large computer.<p>I&#x27;d usually use colab, for 99% of the jobs... but I admittedly want to do some nsfw stuff with me and the wife with AI Art.

Not going to bother with this until the temporal cohesion issue is solved. The results are cool, but the variety in frames makes it look like a very specific and distracting art style instead of true animation.

It&#x27;s insane the speed of progress. That being said, the most smooth example in that tutorial (Temporal Kit) required a lot of outside processing.<p>I wonder if AUTOMATIC1111 ui will start handling workflow scripts, a.l.a blender workflow boxes.

Once the temporal fixes are applied, the residual morphing and flickering actually quite appeals to me. Reminds me of the animation style in A Scanner Darkly.

Does anyone else find it weird that a huge amount of AI video&#x2F;imagery in tutorials etc seem to be sexy anime schoolgirls?<p>I&#x27;m a bloke, I&#x27;ve never been that particularly bothered by like, Lara Croft, Lena, muscular guys in action games etc, I get it, I want to look at idealized&#x2F;attractive things as much as the next person, but something about this makes me super uncomfortable.

Same aesthetics as A-ha&#x27;s &quot;Take on me&quot;. Even with the flaws, I can see how this can simply be used with that exact aesthetic in mind, without the need for any improvement.

Could people please add newbie resources to this post?

So all that&#x27;s happening is people are getting increasingly good at hiding violations of spatiotemporal invariants?<p>This is treating the symptom and ignoring the disease.

Does anyone “in the know” have approximations on cost to run this on collab or similar? Maybe a very rough Dollars per minute of video?

&gt; Something that kind of bothers me that I don&#x27;t understand. Why is there such an obsession about having small computers&#x2F;servers? I don&#x27;t care if I have to put my computer in the basement because its the size of a closet or two. Maybe there is an EE issue, like too much current or EMF if you make a large computer.<p>I’m not using my GPU for heavy machine learning tasks at home. I have a Ncase M1 with a ryzen 3950x (16core&#x2F;32 thread) and a 3090 and a few nvme m2 drives. It’s water cooled but for me the video card is the only pcie card I have, and water cooling keeps the temps reasonable even in the small space. It was a little tricky to build in but why take up a bunch of unnecessary space?

Does someone know how long rendering these videos takes? Can it be done in real-time?

I&#x27;ve only been using stable diffusion in a very basic way, give some text, get some images back. Seeing what others produce with it is amazing and also just feels beyond me, I&#x27;m simply not familiar with all these other extensions and scripts. That said I appreciate a tutorial like this which isn&#x27;t glossing over steps, so it&#x27;s worth a try.

That first video is fabulous. It reminds me of a standard trope in generative art : You generate a whole bunch of prospective images and then pick the nicest ones.<p>Except in this case we get to see all the prospects (The variations in chestplate, wings etc). And in such a cool style.

The end result look like a a cartoon filter applied frame by frame. Or, each frame is basically going through stable diffusion in photo mode

It looks to me there&#x27;s a tradeoff between how much SD changes the input, and how temporally coherent it is.

Isn&#x27;t runway vid2vid superior in the aspect of coherency? This flickering is horrible.

Did they finally fix the annoying flickering?

The Temporal Kit write up was amazing.