Segmenting comic book frames

Next AI challenge: try to infer the intended panel reading sequence, and the flow of speech bubbles&#x2F;narrative.<p>Would potentially be a useful augmentation to a digital comic book reader, refocusing from panel to panel in sequence. Not to mention making comic book content more accessible.

A lot of youtube AI manga recap channels have popped up in the last year. I assumed they have some open source tool that does the panel segmentation for them.

I always thought a fun side gig (or even volunteer opportunity) would be defining panel areas for digital comics. Both because I&#x27;d get to read a lot of comics, and because a lot of comics I read are frustratingly bad at it, and it makes it much harder to enjoy them. Well, there goes AI taking our jerbs.

Awesome stuff! We&#x27;re also working on comic segmentation @ <a href="https:&#x2F;&#x2F;toona.io" rel="nofollow">https:&#x2F;&#x2F;toona.io</a> and other stuff for motion comic generation. The synthetic dataset approaches are really interesting, I&#x27;m curious if you could use an algorithm like <a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Flood_fill" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Flood_fill</a> to aid in segmentation (especially for manga).

There are some graphic novels where segments are a very loose concept.<p>Cerebus&#x27;s Reads volume is basically a book with illustrations, for example.<p>Wonder what SAM would do in such cases...

Segmenting panels is a graphic design element of storytelling that&#x27;s part of the artist&#x27;s job. Doing it programmatically ignores its actual nature in storytelling - establishing the amount and direction of the story beats, helping the reader understand the right reading order, and establishing the important elements on the page (Which beat is the most important, etc).<p>It&#x27;s an interesting tech but giving such an important creative job to a computer instead of an artist is a bad idea for any comic artist who cares about their work.

I have been wanting to do this exact thing! Super excited to look in to this later.

There’s a high quality, free comic reader app that doesn’t collect any data that implements this feature quite nicely: <a href="https:&#x2F;&#x2F;apps.apple.com&#x2F;us&#x2F;app&#x2F;smart-comic-reader&#x2F;id1511175212" rel="nofollow">https:&#x2F;&#x2F;apps.apple.com&#x2F;us&#x2F;app&#x2F;smart-comic-reader&#x2F;id151117521...</a>

&gt; ... it is often easier to see how to improve the dataset than to design new heuristics. Once you do that, you almost have a guarantee that the Neural Network machinery will get you the results.<p>Money quote. Applicable to so many areas of ML&#x2F;AI.

Next up, AI that adds feet to Rob Liefeld&#x27;s 90&#x27;s artwork. :-) (Note: I still love his work)

I&#x27;d love to see an algorithm try and segment this: <a href="https:&#x2F;&#x2F;dresdencodak.com&#x2F;2009&#x2F;07&#x2F;12&#x2F;fabulous-prizes&#x2F;" rel="nofollow">https:&#x2F;&#x2F;dresdencodak.com&#x2F;2009&#x2F;07&#x2F;12&#x2F;fabulous-prizes&#x2F;</a>

on the topic of AI and comic books, since ChatGPT was trained on Wikipedia and thousands of other properties with complete records of comic book lore, why does it get so many relatively basic comic book questions completely wrong? For example, I&#x27;ve asked several times to Jack GPT how did Psylocke temporarily gain the ability to move through shadows in the past? It was a side effect of drinking the Crimson Dawn elixir, which saved her life after she was nearly killed by Sabertooth. All of this information is readily available online. But ChatGPT completely makes up hallucinated explanations for this every time I ask it. Why is that?