Elasticsearch Based Image Search Using RGB Signatures

For anyone interested in the computer vision side of this topic, the author here is using a variant of color histograms, which was state of the art around 1990 [1][2]. Since 2003, bag of visual words approaches have usually meant extracting SIFT-like features from a database of images, quantizing the features down to a list of thousands or millions of &quot;words&quot;, and then treating the images like documents containing those &quot;visual words&quot; [3][4]. (Nothing wrong with the approach he&#x27;s using [simple and fast], but the bag of words terminology in the article usually suggests a different class of approaches.)<p>[1] <a href="https:&#x2F;&#x2F;staff.fnwi.uva.nl&#x2F;r.vandenboomgaard&#x2F;IPCV&#x2F;_downloads&#x2F;swainballard.pdf" rel="nofollow">https:&#x2F;&#x2F;staff.fnwi.uva.nl&#x2F;r.vandenboomgaard&#x2F;IPCV&#x2F;_downloads&#x2F;...</a><p>[2] <a href="https:&#x2F;&#x2F;www.cs.utexas.edu&#x2F;users&#x2F;dana&#x2F;Swain1.pdf" rel="nofollow">https:&#x2F;&#x2F;www.cs.utexas.edu&#x2F;users&#x2F;dana&#x2F;Swain1.pdf</a><p>[3] <a href="http:&#x2F;&#x2F;www.robots.ox.ac.uk&#x2F;~vgg&#x2F;publications&#x2F;papers&#x2F;sivic03.pdf" rel="nofollow">http:&#x2F;&#x2F;www.robots.ox.ac.uk&#x2F;~vgg&#x2F;publications&#x2F;papers&#x2F;sivic03....</a><p>[4] <a href="http:&#x2F;&#x2F;www-inst.eecs.berkeley.edu&#x2F;~cs294-6&#x2F;fa06&#x2F;papers&#x2F;nister_stewenius_cvpr2006.pdf" rel="nofollow">http:&#x2F;&#x2F;www-inst.eecs.berkeley.edu&#x2F;~cs294-6&#x2F;fa06&#x2F;papers&#x2F;niste...</a>

My &quot;Ghost in The Search Machine&quot; talk builds a really naive image search demo (which Sujit uses for his starting point). You might enjoy that:<p><a href="https:&#x2F;&#x2F;www.elastic.co&#x2F;elasticon&#x2F;conf&#x2F;2016&#x2F;sf&#x2F;opensource-connections-the-ghost-in-the-search-machine" rel="nofollow">https:&#x2F;&#x2F;www.elastic.co&#x2F;elasticon&#x2F;conf&#x2F;2016&#x2F;sf&#x2F;opensource-con...</a>

If you&#x27;re interested in searching photos with ES - I took a some what simpler approach focusing just on major colors in the image.<p>But with some of the machine vision API google cloud etc. you could extend to other &quot;features&quot;<p>Details: <a href="http:&#x2F;&#x2F;blog.sandeepchivukula.com&#x2F;posts&#x2F;2016&#x2F;03&#x2F;06&#x2F;photo-search&#x2F;" rel="nofollow">http:&#x2F;&#x2F;blog.sandeepchivukula.com&#x2F;posts&#x2F;2016&#x2F;03&#x2F;06&#x2F;photo-sear...</a>

In my [limited] experience with CBIR and image based search, I found that using a color space with perceptual spatial qualities (such as one of the CIE LaB variants) to be more effective than a purely normalized geometric color space (such as RGB or HSV), as color similarities in the latter may not make much sense to a human.

I have put quite a bit of similar effort to image retrieval using Elasticsearch before. While it is nice and convenient, what I found was that Elastic Search is too slow for larger scale (million of Images with dictionary of millions visual words in BoW model). May be there&#x27;re some steps that I did not do right, but I gave up.

Really cool. But maybe its just me, aren&#x27;t butterflies kind of hard to distinguish between each other. I feel as though his search result page- i couldn&#x27;t really tell if it was good or not because they all looked kind of similar shape-wise. Only difference is color and even then its fairly little color.

Another open-source implementation using a similar concept and Elasticsearch is image-match: <a href="https:&#x2F;&#x2F;github.com&#x2F;ascribe&#x2F;image-match" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;ascribe&#x2F;image-match</a>

Does anyone have a solution to this where the input (search) is also a vector as opposed to a single color, but would still allow for exact color matches?<p>Bucketing means that you can&#x27;t get the granularity of a specific shade of a color.

I wonder if pHash wouldn&#x27;t make this a lot more effective. Anyone tried building an ES-usable distance function for pHash?