Stretch iPhone to its limit: 2GiB Stable Diffusion model runs locally on device

here&#x27;s a direct app store link, if anyone wants to try the iPhone app immediately: <a href="https:&#x2F;&#x2F;apps.apple.com&#x2F;us&#x2F;app&#x2F;draw-things-ai-generation&#x2F;id6444050820" rel="nofollow">https:&#x2F;&#x2F;apps.apple.com&#x2F;us&#x2F;app&#x2F;draw-things-ai-generation&#x2F;id64...</a><p>congratulations to liuliu on the launch!

This is absolutely incredible. It takes about 45 seconds to generate a whole image on my iPhone SE3 - which is about as fast as my M1 Pro macbook was doing it with the original version!

Extreme respect to the developer for not including the &quot;industry standard&quot; clientside tracking&#x2F;analytics&#x2F;phone-home in this app. The fact that this runs locally on-device and <i>doesn&#x27;t send any information to anyone anywhere</i> about what you&#x27;re doing on your local device is wonderful.<p>All apps used to be like this, and now the ones that actually respect user privacy are a rare and glorious exception. Thank you!

This is cool and looks like a great way to drain my phone battery :)<p>I just used the prompt &quot;A person looking at their phone in amazement&quot; and got a good picture.<p>Beware that on startup the app downloads almost 2 gig of data.

This is super cool. I just tried the default prompt on my iPhone 13 with the image size set to 768x512 and using the 3D Model (Redshift v1) and it just crashed the whole phone and restarted. Just like when I get BSOD&#x27;s at work on my Windows GPU desktop :)

This is amazing! Finally a use case for using all that compute power on the phone

This is incredible work and a tremendous achievement. Bravo and thanks for sharing.

This is some impressive work.<p>You might like to look at the work HuggingFace has been doing (on non-iOS versions). They can run it in under 1GB RAM:<p>&gt; If is also possible to chain it with attention slicing for minimal memory consumption, running it in as little as &lt; 800mb of GPU vRAM<p><a href="https:&#x2F;&#x2F;huggingface.co&#x2F;docs&#x2F;diffusers&#x2F;optimization&#x2F;fp16#offloading-to-cpu-with-accelerate-for-memory-savings" rel="nofollow">https:&#x2F;&#x2F;huggingface.co&#x2F;docs&#x2F;diffusers&#x2F;optimization&#x2F;fp16#offl...</a>

It works extremely fast on an iPad Pro M1 (kind of expected, but it&#x27;s _impressive_) although the app is built as &quot;iPhone only&quot;, and strangely enough the iPad is cropping the upscaled iPhone app so the lower bar of image controls don&#x27;t show at all, which is a pity

Hahah now I can use my phone as a hand warmer this winter. It&#x27;s incredible that this is an app!

Could in-memory compression used to bring down the RAM requirements?<p>There are some performance compressors like Blosch tuned for this:<p><a href="https:&#x2F;&#x2F;www.blosc.org&#x2F;pages&#x2F;blosc-in-depth&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.blosc.org&#x2F;pages&#x2F;blosc-in-depth&#x2F;</a><p>“Faster than memcpy” is the slogan.

Has anyone found anything similar (self-hosted or local on device) - but for text generation?

Now I see <i>why</i> that coffee cup icon is on the &quot;generate images&quot; button... fingers burning after a few images.<p>Haha, awesome app!

Awesome! Although I wish to see the intermediate denoised image instead of the progress bar. Just a suggestion.

I just tried it works beautifully! I suggest defaulting to lower resolution 384 X 384 since it will speed up.

Unstable trying with 10 iterations on 348x348 on iPad 9th Gen <a href="https:&#x2F;&#x2F;support.apple.com&#x2F;kb&#x2F;SP849?locale=en_US" rel="nofollow">https:&#x2F;&#x2F;support.apple.com&#x2F;kb&#x2F;SP849?locale=en_US</a>. Looks cool tho!

Congrats on the release!<p>I gave this and other available applications a try and I don’t understand what people see in ai image generation.<p>A simple prompt generated a person with 3 nostrils, 7 squished fingers, deformities everywhere I look, it just mashes a bunch of photographs together and generates abominations.<p>Pay close attention to generated models and you will find details which are simply wrong.<p>What is the use case that I’m missing?

liuliu has always been a fucking god

This is awesome. It has killed my battery over the last couple of hours, because I couldn&#x27;t stop generating new images.

I wonder if any of these tricks would be applicable to make a version of Stable Diffusion which could run on the Steam Deck.

Any tips on how you “ If the face is cropped, now I know how to use the inpainting model to fill it in. If the inpainting model doesn’t do its job, you can always use a paint brush to paint it over and do an image-to-image generation again focused in that area.” using the app?

on my iPhone 13, generates 384x384 images in under a minute.<p>discovered they have stable diffusion 1.4,1.5, waifu diffusion(Anime), redshhift(3d model) and other models.<p>iPhone becomes warm after couple of runs and starts draining battery, so do it while connected to charger.

Default prompt gives me an arm, or feels like a crop of a full photo (iPhone 11, iOS 16.1) <a href="https:&#x2F;&#x2F;files.catbox.moe&#x2F;ivy15m.PNG" rel="nofollow">https:&#x2F;&#x2F;files.catbox.moe&#x2F;ivy15m.PNG</a>

Even though I understood very little of that it was still wild fun reading it. I&#x27;m glad such wizards exist, because I and most people I know certainly don&#x27;t qualify.

On a related note, has anyone been able to utilize Apple silicon GPUs? Running CPU-only is incredibly slow (and sad, since i&#x27;ve got these Apple accelerators idle!)

Is there any option to set so that every image is automatically saved, and not to camera roll, but to the local app folder (same folder that contains this apps data)

This looks so cool! Unfortunately, my 16.1 11pro is crashing before an image can be generated, let me know if I can do anything to debug&#x2F;test this.

Thanks, Ive been looking forward to something like this coming out. Runs great on my iPad but a UI optimised for the form factor would be nice.

I cant wait till this is possible with 24fps and do live camera view modifications. That woud be insane during meetings :) I guess 5 more years.

Really cool, but for some reason, it fails to share the images using Airdrop for me. I have to save the images to photos then airdrop from there

I have tested the app on my old iPhone XS Max, it takes less than 3 mins for an image.<p>And also can choose a model, steps, scale, and sampler!<p>Thank you for your great work!

small add. display multiple examples while it&#x27;s downloading the model. Its a long time to stare at an astronaut on a horse in space

Fantastic! Can it be made available for iPad?

Did you try using Core ML for inference?

The quality of comments here is absolutely abysmal given the deeply technical nature of the article.

I wonder how long before the appstore policies are updated to require models be embedded in apps.

Is this something that might make it&#x27;s way to tensor&#x2F;pixel or unique to Apple silicon?

the developer is about to have a MASSIVE hosting bill<p>the download restarts from 0% if the app is sent to the background, as there does not seem to be a download manager. This is especially problematic for the large 1.5gb file.

Hmmm, my ipad keeps crashing and redownloading the sd_v1.4_f16.ckpt file.

This is really impressive !

Aww man, no ipad version? Tsk what did I get this 16gb of ram for?

i have an ipad. the regular non air&#x2F;pro&#x2F;m1 one and the app installed and when i run it, it says &quot;could be device incompatibility&quot; and subsequently crashes

Crashes every time... am I doing something wrong? iPhone 11 Pro.

Kudos. I hope liuliu has a good deal on egress traffic... :-)

Uh..... Android is there? I mean android version?

I&#x27;m downloading it now. This should be fun.

Makes a nice hand warmer on a cool evening.

Works great on my iPhone 14 Pro Max

This will sell Iphone to me.

works really well thank you!

Brilliant!

Preserve your privacy, send us your phone number!

Kinda ironic that this links to a site that is almost unreadable on an iPhone (14 pro max if you’re wondering).

This is amazing. I&#x27;m kind of surprised that it doesn&#x27;t have an NSFW image blocker. I want to be able to generate NSFW images but it probably should have one enabled by default.

Update: Draw Things uses “One-time photo selection” which according to Settings &gt; Privacy &amp; Security &gt; Photos “Even if your photos were recently shown to you to select from, the app did not have access to your photo library.” Still, I didn’t realize apps could save to Photos without explicitly asking permission.<p>I don’t recall giving “Draw Things” permissions to access my photo library, yet the app is able to save to my photo library without prompting and able to read existing images.<p>I may have misunderstood what permissions apps should ask for when saving to the photo library.

Meh… 2gb download only to try “image of unicorn pooping” to get the lamest results ever. If I can’t amuse my 5 yr old with this AI it’s nonsense.<p>It transcribed as “image of unicorn poo ping” in tags :(