MobileDiffusion: Rapid text-to-image generation on-device

&gt; With superior efficiency in terms of latency and size, MobileDiffusion has the potential to be a very friendly option for mobile deployments given its capability to enable a rapid image generation experience while typing text prompts. And we will ensure any application of this technology will be in-line with Google’s responsible AI practices.<p>So I&#x27;m interpreting this that it won&#x27;t ever get released.

some points that stood out to me:<p>1. they made a lot of careful tweaks to the unet network architecture - it seems like they ran many different ablations here (&quot;In total, our endeavor consumes approximately 512 TPUs spanning 30 days&quot;).<p>2. the model distillation is based on previous UFOGen work from the same team <a href="https:&#x2F;&#x2F;arxiv.org&#x2F;abs&#x2F;2311.09257" rel="nofollow">https:&#x2F;&#x2F;arxiv.org&#x2F;abs&#x2F;2311.09257</a> (hence the UFO graphic in the diffusion-gan diagram)<p>3. they train their own 8-channel latent encoder &#x2F; decoder (&quot;VAE&quot;) from scratch (similar to Meta&#x27;s Emu paper) instead of using the SD VAEs like many other papers do<p>4. they use an internal dataset of 150m image&#x2F;text pairs (roughly the size of laion-highres)<p>5. they also reran SD training from scratch on this dataset to get their baseline performance

Kind of funny that they show the iphone 15 pro and the Samsung S24 in the comparison chart, but not their own phone the google pixel 8. (I know it will perform worse than both phones)

Google has fallen so far. Both Inception and Mobilenet were released openly and changed the entire AI world.<p>Nowadays we just get blog posts about results that were supposedly achieved, an accompanying paper that can’t be reproduced (because of Google’s magical “private datasets”), and some screencaps of a cool application of the tech that is virtually guaranteed to never make it to product.

I don&#x27;t suppose there&#x27;s any way to actually get this?

Are like people at Google Research not embarrassed that none of this stuff ever makes it to real life?<p>Google AI internally needs a huge culture change, stop acting like academics making things for academics and start working like developers making products for customers.<p>I&#x27;d say in 10 years we&#x27;ll be looking back and seeing the wasted potential but actually you can look back around 10 years and already see the wasted potential of all the things Google demoed or papered and never shipped.

Original paper: <a href="https:&#x2F;&#x2F;arxiv.org&#x2F;abs&#x2F;2311.16567" rel="nofollow">https:&#x2F;&#x2F;arxiv.org&#x2F;abs&#x2F;2311.16567</a>

<a href="https:&#x2F;&#x2F;arxiv.org&#x2F;pdf&#x2F;2311.16567.pdf" rel="nofollow">https:&#x2F;&#x2F;arxiv.org&#x2F;pdf&#x2F;2311.16567.pdf</a>

So what could be some use cases for this apart from as a toy or for faking photos&#x2F;art?

At first it seems nice. Then I realise it is by Google.<p>This will never see the light of day.

Google may very well be first to create AGI but it will be wrapped in so many “safety” layers that it would effectively be lobotomised. Let’s just hope that a Google AGI never gets to watch A clockwork orange.

I&#x27;m tired of this crap

I never upvote any Google&#x27;s A.I. research articles as most of the time it is: look what we have done, but we will never release anything.<p>OpenAi gets a lot of criticism for being closed, but at least I can play with their api most of the time.<p>What&#x27;s the point of this if we will never be able to use this?