This reminds me of a paper from 2005 by Daniel Sýkora et al. [1] which tries something very similar, with the specific use case of animation video. The authors describe it best in the abstract I think:<p>> Video Codec for Classical Cartoon Animations with Hardware Accelerated Playback<p>> We introduce a novel approach to video compression which is suitable for traditional outline-based cartoon animations. In this case the dynamic foreground consists of several homogeneous regions and the background is static textural image. For this drawing style we show how to recover hybrid representation where the background is stored as a single bitmap and the foreground as a sequence of vector images.<p>The idea of using prior knowledge about the nature of the content to decide on an encoding scheme makes intuitive sense to me, though I'm not a codec person so I don't know how feasible it would be to make these ideas into the hardware-accelerated codecs we know from other methods.<p>Of course, these methods would make the <i>most</i> sense when used directly by the animation studios during export, not as an afterthought. But I'll take what we can get.<p>By the way, the corpus of Sýkora's works [2] is really really impressive in my opinion. He gave a talk in my institute while I was researching methods around neural style, and his take on parametric models, paired with the quality (and speed!) of his results, really left a mark, if not to say they made me seriously question wtf I was doing there. His work is strictly tailored to a
professional animation / video production setting, so it seems <i>extremely</i> applicable compared to the toy-like nature of neural style methods. That is not to say he doesn't know about those. His team's recent papers actually fruitfully combine the two.<p>[1]: <a href="https://link.springer.com/chapter/10.1007/11595755_6" rel="nofollow">https://link.springer.com/chapter/10.1007/11595755_6</a><p>[2]: <a href="https://dcgi.fel.cvut.cz/home/sykorad/" rel="nofollow">https://dcgi.fel.cvut.cz/home/sykorad/</a>