AI and the Future of Pixel Art

As someone who draws, there are some obvious aspects where AI generation just works perfectly:<p>- generate details and texture, which photobashing was already used for; but that mostly solve the licensing problem for it<p>- generate random inspiration boards, for which image search was used (again, mostly solve the licensing problem)<p>- generate derivative stuff, e.g. typical game portraits or props<p>In practice, only the third point is generally discussed, because it lowers tremendously the entry barrier to generate images for people without any skills. It&#x27;s like if you could pick up screenshots from other content, clean them up, and you&#x27;re free to use it.<p>Whereas it essentially does not work for:<p>- cartoony generation. It relies too much on line consistency, visual clarity and abstraction<p>- concept design --not the flashy 10 minutes speedpaint type, but where you have to combine ideas in meaningful ways. In particular hard-surface design which required good 3D thinking and consistency of the whole.<p>These fundamental flaws are omnipresent, but can be hidden by certain styles where things are implied by color blobs, hidden by stylized brushstrokes, or simply an overflow of details (something Midjourney is very good at).<p>All in all, it feels like AI is a danger for people at the bottom of the profession hierarchy, but will elevate people at the top, whose work cannot be replaced. In other words, people who are more akin to be considered &quot;artisans&quot; rather than artists, who will take a prompt and simply clean it.<p>In particular, drawing has something like the 20&#x2F;80 rule, where all the creative input is in the first 20% and the rest is &#x27;rendering&#x27;, a very mechanical task which you can mostly do with your brain turned off. As Yumenoley put it, &quot;it was a mistake to let the AI do the interesting part&quot;.

There are some fundamental pieces missing in AI art generation that when solved will completely change the game. Forever. I think most fundamentally is the capability for AI to have some kind of memory or maybe more technically a way for the AI to be capable of do style &quot;character style transfer&quot; more effectively. I think this is probably possible, it&#x27;s akin to making deep fakes but for drawn&#x2F;photography art.<p>With this tool suddenly the effort of making a series of compositions that are coherent will dramatically change the game, specially in the videogame industry. I think there are a lot of programmers out there capable of making great games but might be lacking the resources to fully complete their visions due to having to needing assets for their games.<p>It seems to me like the approach stable diffusion has taken has dramatically increased interest and utility of these tools. So I&#x27;m hoping they follow similar lines for other types of AI. Every week I&#x27;m reading of a new novel use for these generators that I hadn&#x27;t really considered before.

As a hobby game programmer stable diffusion looks exciting, the idea of quickly generating some assets for a game jam is very appealing. But as a hobby musician it strikes me hard. I want to write, compose and play my own music, so I would understand someone feeling the same with regards to hand drawing arts. This whole stable diffusion is exciting on one side, but on the other makes me wonder what we will be left with once this technology reaches higher capabilities?

With the image generators coming out I’ve been scrambling to understanding their place, I’ve been looking to the chess world as a model of the future.<p>In chess, people would still rather see people play chess than a robot. The top chess players in the world started as a brute force obsession about the game. Those would go on to teach the next generation. They advent of computers allowed for historically statically advantaged moves. ML came along and disrupted even further.<p>Now many of the top chess players consult the ML chess oracle.<p>I see the same thing happening in a lot of areas: grammar, image generation, text replies.<p>I see a world where humans are celebrated for their humanness while machines assist.

I honestly thought pixel art style games were made by creating 3D assets and then rendering them in low resolutions as sprite sheets these days. Or using the 3D models in the game with a &#x27;pixelate&#x27; shader. The idea that people still hand draw sprites slightly blows my mind.

&gt; 200x200 is relatively large for pixel art, but if a single pixel makes this much of a difference it should probably be larger.<p>Huh? Half the point of pixel art is that single pixels make a difference. That art is way above the threshold where one pixel can make too much of a difference.

For the last time, the issue with &quot;AI&quot; is not that it exists as a tool, all of the issue that anyone should have is in how the data used to fit is obtained. No one would have any issue if you took the time to draw thousands of images then trained on them and lived off of that, just don&#x27;t steal data from others, hand-wave as &quot;free use&quot; and carry off into the sunset on data you didn&#x27;t generate.<p>The <i>tools</i> behind AI are fine and have honestly existed for decades. If anyone is up against AI because it isn&#x27;t authentic or something, that&#x27;s a fools errand really because people, artists themselves and developers, will find them useful. The problem is and always will be how the data you fit on and how you obtained it.

The article linked to a twitter search: <a href="https:&#x2F;&#x2F;twitter.com&#x2F;search?q=KaliYuga%20pixel%20diffusion&amp;src=typed_query" rel="nofollow">https:&#x2F;&#x2F;twitter.com&#x2F;search?q=KaliYuga%20pixel%20diffusion&amp;sr...</a><p>Here&#x27;s another option: <a href="https:&#x2F;&#x2F;old.reddit.com&#x2F;r&#x2F;StableDiffusion&#x2F;search?q=pixel+model&amp;restrict_sr=on" rel="nofollow">https:&#x2F;&#x2F;old.reddit.com&#x2F;r&#x2F;StableDiffusion&#x2F;search?q=pixel+mode...</a><p>For example:<p>* Pixel art sprite sheets: <a href="https:&#x2F;&#x2F;old.reddit.com&#x2F;r&#x2F;StableDiffusion&#x2F;comments&#x2F;y54isd&#x2F;couldnt_generate_pixel_art_with_sd_so_i_trained_a&#x2F;" rel="nofollow">https:&#x2F;&#x2F;old.reddit.com&#x2F;r&#x2F;StableDiffusion&#x2F;comments&#x2F;y54isd&#x2F;cou...</a><p>* pixel-art-v1: <a href="https:&#x2F;&#x2F;old.reddit.com&#x2F;r&#x2F;StableDiffusion&#x2F;comments&#x2F;yj1kbi&#x2F;ive_trained_a_new_model_to_output_pixel_art&#x2F;" rel="nofollow">https:&#x2F;&#x2F;old.reddit.com&#x2F;r&#x2F;StableDiffusion&#x2F;comments&#x2F;yj1kbi&#x2F;ive...</a>

Using these generators as tools to find inspiration seems like the best case scenario. I think people are assigning too little probability to the potential future scenario that current systems are about as good as we&#x27;re going to get with current ML methods, though some refinements will marginally improve them. Without a major breakthrough, I don&#x27;t see any AI systems replacing professional artists on video games, e.g., unless it&#x27;s a very low effort, low quality game.

Looking forward to the day when I can generate 4 direction sprite sheets, walk cycle, attacks and equipment.

I&#x27;ve already lost track of all the different apps that let you play with Stable Diffusion et al.<p>Do you have any recommendations for (web)apps that allow you to generate images from prompts in good resolutions? How about ones for img2img?

My wife is an artist and has been using some of these systems to inspire her own work with clients - she&#x27;s said it&#x27;s both given her creative new ideas, and improved her efficiency by about 30%.

Creative director is one of the many hats I wear these days. I am using components of AI in nearly all of my visual assets. The future is already here.