Differentiable Programming for Image Processing and Deep Learning in Halide [pdf]

Very glad to see that Halide keeps being developed, despite remaining a bit of a niche language (for now).<p>Anything that makes squeezing out more performance from our computers while writing more elegant code at the same time deserves more attention.<p>I wonder if any of the techniques used in Halide can be (somewhat) generalized to domains outside of image processing?<p>EDIT: Ah, I see that the paper basically answers my questions, as it sees applications in ML contexts. Also, theat new demosaicking algorithm looks great, wonder if will make it&#x27;s way to DarkTable any time soon?<p>The page of the paper[0] also has presentation slides[1] which (to me at least) feel a bit more accessible. Hope the accompanying presentation will be on YouTube soon.<p>[0] <a href="http:&#x2F;&#x2F;gradient.halide.ai&#x2F;" rel="nofollow">http:&#x2F;&#x2F;gradient.halide.ai&#x2F;</a><p>[1] <a href="https:&#x2F;&#x2F;people.csail.mit.edu&#x2F;tzumao&#x2F;gradient_halide&#x2F;gradient_halide_slides.pdf" rel="nofollow">https:&#x2F;&#x2F;people.csail.mit.edu&#x2F;tzumao&#x2F;gradient_halide&#x2F;gradient...</a>

Note that TVM (<a href="https:&#x2F;&#x2F;github.com&#x2F;dmlc&#x2F;tvm" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;dmlc&#x2F;tvm</a>) is a similar system (the IR is a fork of the Halide IR, but substantially improved), and is much more suited to deep learning, with tools like:<p><pre><code> - TOPI, a library of optimized routines for common deep learning operations - Importers for TensorFlow, ONNX (PyTorch&#x2F;Caffe2), Keras, MXNet, DarkNet and other frameworks. - AutoTVM for automatically finding fast schedules for arbitrary devices, much faster than random search or the automatic schedules generated by e.g. Halide. - Various mobile GPU runtimes (OpenGL, OpenCL, Vulkan, etc), compared to Halide - Large community contributing optimized runtimes (e.g. Intel + Amazon contributing CPU improvements, improved schedules&#x2F;declarations, etc) </code></pre> Highly recommend checking it out.