Accurate Image Alignment and Registration Using OpenCV

Image alignment allows for some fun image manipulations like these. I think one of the coolest novel applications of image alignment in recent years must be multi-frame super-resolution. For example, the Pixel phones use it to improve low-light and zoomed in photos[0].<p>[0] <a href="https:&#x2F;&#x2F;sites.google.com&#x2F;view&#x2F;handheld-super-res&#x2F;" rel="nofollow">https:&#x2F;&#x2F;sites.google.com&#x2F;view&#x2F;handheld-super-res&#x2F;</a>

I currently working on an optical respiratory monitor and one of the challenges was image alignment&#x2F;registration for an optical and a thermal image with wildly different resolutions, fovs.<p>We ended up doing this by manually calibrating the images in matlab using a custom script and logging out a transformation matrix that we could then multiply the optical image by to get matching pixels in the thermal image.<p>Really fun project, but definitely a tedious thing to do, especially when its only a small part of the overall project. The project in the link also looks very cool, just not quite right for us as we probably wouldnt consistently have enough landmarks to map.

Note that this is exactly what panorama software like Hugin [0] does - it even comes with a convenient command-line tool for this simple use case (aligning a stack of images that are mostly overlapping): <a href="https:&#x2F;&#x2F;wiki.panotools.org&#x2F;Align_image_stack" rel="nofollow">https:&#x2F;&#x2F;wiki.panotools.org&#x2F;Align_image_stack</a><p>[0] <a href="http:&#x2F;&#x2F;hugin.sourceforge.net&#x2F;" rel="nofollow">http:&#x2F;&#x2F;hugin.sourceforge.net&#x2F;</a>

Turboreg is also good: Paper: <a href="http:&#x2F;&#x2F;bigwww.epfl.ch&#x2F;publications&#x2F;thevenaz9801.html" rel="nofollow">http:&#x2F;&#x2F;bigwww.epfl.ch&#x2F;publications&#x2F;thevenaz9801.html</a><p>Python implementation: <a href="https:&#x2F;&#x2F;pypi.org&#x2F;project&#x2F;pystackreg&#x2F;" rel="nofollow">https:&#x2F;&#x2F;pypi.org&#x2F;project&#x2F;pystackreg&#x2F;</a>

The biggest issue in real life use cases is lens distortion - these alignment algorithms usually don&#x27;t correct barrel, pin-cushion or more complex lens distortions, making the alignment imperfect.

I use a similar algorithm at work to detect if pages match a template and then align the image to OCR from mapped fields. Not perfect but pretty effective.

Affine transformations are very useful for algorithmic image rotation&#x2F;translation&#x2F;reflection [0].<p>[0] <a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Affine_transformation" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Affine_transformation</a>

This is the kind of post I love to find in HN: an interesting topic and code. Does anyone know what is the theme of the blog? I am interested in how the references are created. I suppose it is based on something like RST.