Imgdiff: Faster than the fastest pixel-by-pixel image difference tool

Compared to the other tool this is just &quot;slow code, but now with threads.&quot;<p>This problem is perfect to exploit capabilities of modern CPUs, and neither of the projects does so. At least <i>this</i> one iterates over the <i>y</i> coordinates in the outer loop.<p>Nothing about what either tool does is noteworthy.

When making an <i>image</i> manipulation&#x2F;creation&#x2F;diff&#x2F;etc tool I&#x27;d highly recommend showing at least a screenshot or two of the result in the Github itself.

I&#x27;m skeptical about it&#x27;s high claims. Looking at the code its just using the Go standard library image package, looping over the bounds and calculating delta?<p>Doesn&#x27;t seem like there&#x27;s anything special here? It&#x27;s roughly what any novice would build if asked to diff two images in Go?

I&#x27;m finding &quot;gm compare&quot; to be faster.<p><pre><code> $ time .&#x2F;imgdiff cypress.png cypress.png Success! Images are equal. 3.22user 0.10system 0:00.50elapsed 661%CPU (0avgtext+0avgdata 478524maxresident)k 0inputs+0outputs (0major+48339minor)pagefaults 0swaps $ time gm compare -highlight-style assign -highlight-color purple -file diff.png cypress.png cypress.png 1.99user 0.21system 0:02.21elapsed 99%CPU (0avgtext+0avgdata 874948maxresident)k 0inputs+5240outputs (0major+217173minor)pagefaults 0swaps $ time gm compare -metric mse cypress.png cypress.png Image Difference (MeanSquaredError): Normalized Absolute ============ ========== Red: 0.0000000000 0.0 Green: 0.0000000000 0.0 Blue: 0.0000000000 0.0 Opacity: 0.0000000000 0.0 Total: 0.0000000000 0.0 0.94user 0.14system 0:00.67elapsed 161%CPU (0avgtext+0avgdata 585832maxresident)k 0inputs+0outputs (0major+144881minor)pagefaults 0swaps</code></pre>

Years ago I worked with image processing and thought of writing a CLI image diff tool. I then thought meh, anyone can quickly roll their own.<p>Turns out it seems to be something people care about.<p>I should just start writing the CLI tools I think of, without dismissing their utility.

Why not use the GPU? This is exactly the kind of tasks GPUs were designed for. Gimp can do this in real time using the difference layer mode.

Worth noting it&#x27;s ~100-lines of Go code that&#x27;s quite readable:<p><a href="https:&#x2F;&#x2F;github.com&#x2F;n7olkachev&#x2F;imgdiff&#x2F;blob&#x2F;master&#x2F;pkg&#x2F;imgdiff&#x2F;imgdiff.go" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;n7olkachev&#x2F;imgdiff&#x2F;blob&#x2F;master&#x2F;pkg&#x2F;imgdif...</a><p><a href="https:&#x2F;&#x2F;github.com&#x2F;n7olkachev&#x2F;imgdiff&#x2F;blob&#x2F;master&#x2F;pkg&#x2F;yiq&#x2F;delta.go" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;n7olkachev&#x2F;imgdiff&#x2F;blob&#x2F;master&#x2F;pkg&#x2F;yiq&#x2F;de...</a>

This feels like an application where an APL one-liner would just be the fastest thing that there is.

What are some of the common applications for diffing an image?

A bit meta and very cruddy - a cheap and cheerful ruby tool I whipped up back in 2014 that would render two websites and then diff them.<p>It was achieved by looping over every RGB pixel from the PNG rendered image of each URL.<p><a href="https:&#x2F;&#x2F;github.com&#x2F;sammcj&#x2F;urldiff" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;sammcj&#x2F;urldiff</a><p>Not exactly something I&#x27;m proud of, but the pixel loop solution did make me laugh at the time.

For those interested, I created a gem in Ruby for comparing the similarity of images by computing the dhash of an image and then calculating the Hamming distance between the hashes: <a href="https:&#x2F;&#x2F;github.com&#x2F;rohanpatel2602&#x2F;ruby-dhash" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;rohanpatel2602&#x2F;ruby-dhash</a>

You should probably provide instructions as to how to build it. The <i>build.sh</i> script does not work for me because <i>cmd&#x2F;main.go</i> is not in GOROOT. <i>go run main.go</i> and <i>go build</i> inside the <i>cmd&#x2F;</i> directory works though.<p>I am really used to just being able to do <i>go build</i> from the root directory of the repository.

This can be done much faster. The problem is memory&#x2F;disk bound.<p>Decoding the PNGs should be most of the work if written properly.<p>Edit: Yes, this is more reasonable: <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=25405595" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=25405595</a>

Would it be possible to compare image files before decompression ? For JPEG, directly compare the DCT coefficients

Congratulations on writing a tool that has 3 times the performance of the most commonly used competitor. The fact that you exploited simply straightforward technique in terms using Go&#x27;s multithreading is a plus in my book, no magic just simple engineering. Also, thanks for sharing it :)

gm compare -metric mse .&#x2F;a.jpeg .&#x2F;b.jpeg -file .&#x2F;difference.png

I solved this problem for my use case by using an approximate diff.<p>Basically diffing took too long ( I was diffing to remove duplicate frames from a virtualized browser to browser screencast ), so after trying to various options (hashing, diffs, etc) I just went with a simple check of a basically random but fixed set of pixels, which worked really well. I don&#x27;t have data on the actual false positives&#x2F; negatives but for the thing important in my case, perceptual difference and not sending the same frame twice (to save bandwidth and speed) it worked great, and was &quot;constant time&quot;.<p>The &quot;code&quot; is here: <a href="https:&#x2F;&#x2F;github.com&#x2F;c9fe&#x2F;ViewFinder&#x2F;blob&#x2F;84620bd87abb32b2f2c3dc0b79a0be5212fa14ee&#x2F;zombie-lord&#x2F;screenShots.js#L9" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;c9fe&#x2F;ViewFinder&#x2F;blob&#x2F;84620bd87abb32b2f2c3...</a><p>And a demo of the project is: <a href="https:&#x2F;&#x2F;demo.browsergap.dosyago.com" rel="nofollow">https:&#x2F;&#x2F;demo.browsergap.dosyago.com</a> (if you check it on Safari mobile you need to plant your clicks about half a finger under where you think they should go, some bug!)<p>I&#x27;d like to know some idea of how this posted fast diff works. I checked around the repo and couldn&#x27;t figure it out, but it seems to be splitting across multiple cores, but still looping over each pixel (but divided by the number of cores). I thought it might be doing something like checking 64-bit blocks somehow, but seems not.

Looks like a for loop iterating over each pixel one row at a time.<p>What makes this fast?