This is cool, but the frame-to-frame variance is distracting. I really want to see this reimplemented with temporal constraints a-la this paper:<p><a href="https://www.youtube.com/watch?v=Khuj4ASldmU" rel="nofollow">https://www.youtube.com/watch?v=Khuj4ASldmU</a>
I remember watching an interview with the creators of South Park in which they described the transition from animating using cardboard cutouts to a system with CorelDraw and other pieces of software which helped speed up the process. The bulk of the efficiency improvement came from carefully defining all the frequently used objects (characters, houses) once with movable components, and reusing those objects in the per-episode animation pipeline.<p>I can easily imagine an animation system like the one presented here enabling another massive improvement in animation efficiency. In the same way animation software allowed South Park to reuse pre-drawn objects, a deep learning system could enable south park to carefully define the entire drawing style just once, then generate complete episodes based on simple story boards and animation directives. Fortunately, South Park already has a significant amount of training data available, specifically every South Park episode yet produced.
"It means nothing to me. I have no opinion about it, and I don't care."<p>On the first moon landing, quoted in The New York Times, (1969-07-21).<p><a href="https://en.wikiquote.org/wiki/Pablo_Picasso" rel="nofollow">https://en.wikiquote.org/wiki/Pablo_Picasso</a><p>Curious about his feelings regarding this work. (I find it beautiful.)
I have two opinions: 1) I don't think cubism transfers well into a motion picture format, 2) I think these experiments, as they are currently, attempt to merge two styles and end up with neither, and nothing novel in its place; there is little Kubrick or Picasso in the final piece.<p>I think it's superficial and doesn't do either source justice.
I remember when The Scanner Darkly came out there was a lot of talk about how they achieved the style of the film. Some of it was automated, but a lot still had to be hand done. I wonder if using deep learning systems we could achieve the same effect that film had with nearly zero human interaction.<p>For those that haven't seen the movie, here's the trailer: <a href="https://www.youtube.com/watch?v=TY5PpGQ2OWY" rel="nofollow">https://www.youtube.com/watch?v=TY5PpGQ2OWY</a>
Possibly the finest painting software currently available is Synthetik's Studio Artist (<a href="http://synthetik.com/" rel="nofollow">http://synthetik.com/</a>). Compared to Adobe's powerhouse software, it's relatively unknown, but that doesn't make it any less innovative.<p>It uses an algorithmic 'paint synthesizer' to generate brushes (with hundreds of presets) and auto-paint canvases, and is designed for animation (rotoscoping) as well as static artwork. The output can be reminiscent of the style of the movie 'A Scanner Darkly', but the software is hugely flexible. Here are a couple of rather amazing examples: <a href="http://studioartist.ning.com/video/auto-rotoscoped-dancers" rel="nofollow">http://studioartist.ning.com/video/auto-rotoscoped-dancers</a> and <a href="http://studioartist.ning.com/video/dance-styles-animation-reel" rel="nofollow">http://studioartist.ning.com/video/dance-styles-animation-re...</a><p>Also, unlike most other 'painterly' software, the graphics are resolution independent - meaning that they can be scaled up to any size without loss of detail.
There is something that escapes me regarding this very cool neural style transfer technique. One would expect it to need <i>at least three</i> starting images: the one to transform, the one used as a source for the style, and a non-styled version of the source. This last one should give the network hints on how to transform the unstyled version in the styled one. For example, what does a straight line end up being in the style? Or how is a colour gradient represented? Missing this, it seems that the neural network should be able to recognize objects in the styled picture, and derive the transformation applied based on a previous knowledge of how they would normally look like. But of course the NN is not advanced enough to do that.
Can someone explain me roughly how does this work?
It certainly has a wow factor, but once you pass the initial impression, it's interesting that the brain starts recognizing the content (motion of characters and objects) separately from the visual style, and even starts applying negative cubism filter so that we won't actually see the visual style anymore. (In other words, the brain treats those applied style as noise.)<p>It could be a way to exploit the mismatch of content and style as certain form of expression; but it may be more interesting if we can modify the temporal structure as well.
Like someone said about this on /r/programming:<p>>Pretty tight that computers can drop acid now.<p>Anyway, here's a direct link to the video for mobile users: <a href="https://vimeo.com/169187915" rel="nofollow">https://vimeo.com/169187915</a>
The big changes frame-to-frame certainly add to the "trippiness" but I'd love to see this where the value function (or whatever it's called for ML) prioritizes reducing the frame-to-frame diff so that I could actually watch the full length movie like this.
I am much more of an artist than a technology person and the rendering inconsistency the author refers to is one of the coolest aspects of the video. This is the kind of happy accident that gives work originality and makes it more than a slavish copy. Reminds me of Link Wray putting a pencil through the speaker cone of his amplifier.
I kind of want someone to do the same thing with a "NON-neural network" Picasso filter, like the ones in Photoshop and similar image editing programs. I want to compare how much the neural network's understanding of Picasso's style adds to the work (I imagine it's a lot, because this looks incredible).
"Poetry is what gets lost in translation",
"Art is what gets lost in machine learning".<p>I think it's interesting that it's possible to create basically filters from existing images, but then applying those filters to large amounts of images (like in this movie) quickly loses the novelty effect and is just as boring as any photoshop or gimp filter became in the 90s after seeing it 3 times.<p>When I look at Picassos actual pictures, I am astonished and amazed with every new one I get to see. With these pictures, I get more and more bored with every additional image.
Neural style transfer is extremely fun to play with.<p>If you have a system with a recent-ish graphics card (I'm doing fine with my GTX 970), put a linux on it and check out the many GitHub projects that implement this stuff (some of the tools will only work on linux).<p>It's a great way to start learning about deep learning and GPU based computation , which are starting to look like very good things to have on your resume.<p>Plus, you get to make cool shit like this that you can actually show to your friends. I'm getting more interested in the text generation stuff as well too - I'd love to make a Trump speech generator :-)
Can someone knowledgeable estimate, how far are we from rendering this in 60 frames per second? Can't wait to try it as a post-processing layer in game development.
Not trying to over-state my qualifications to make the following claim, but I'm pretty sure Kubrick would have hated this. And, as such, have it destroyed.
I wonder how long it takes to render each frame.<p>Eventually with fast enough GPUs you could render a video game in this style, now that I would like to see.
His <a href="http://bhautikj.tumblr.com/tagged/drumpf" rel="nofollow">http://bhautikj.tumblr.com/tagged/drumpf</a> is much better.
Donald Drumpf as sausage
So basically you take someone else's work. Run it with some content (someone else's work also), post it, and wow innovation.<p>Actually in the last year myriads of similar things were created, and this is simply boring.<p>This is as interesting as a random tumblr reblog. May be curious, but lacks any sense of achievement or originality.