For anyone interested in the computer vision side of this topic, the author here is using a variant of color histograms, which was state of the art around 1990 [1][2]. Since 2003, bag of visual words approaches have usually meant extracting SIFT-like features from a database of images, quantizing the features down to a list of thousands or millions of "words", and then treating the images like documents containing those "visual words" [3][4]. (Nothing wrong with the approach he's using [simple and fast], but the bag of words terminology in the article usually suggests a different class of approaches.)<p>[1] <a href="https://staff.fnwi.uva.nl/r.vandenboomgaard/IPCV/_downloads/swainballard.pdf" rel="nofollow">https://staff.fnwi.uva.nl/r.vandenboomgaard/IPCV/_downloads/...</a><p>[2] <a href="https://www.cs.utexas.edu/users/dana/Swain1.pdf" rel="nofollow">https://www.cs.utexas.edu/users/dana/Swain1.pdf</a><p>[3] <a href="http://www.robots.ox.ac.uk/~vgg/publications/papers/sivic03.pdf" rel="nofollow">http://www.robots.ox.ac.uk/~vgg/publications/papers/sivic03....</a><p>[4] <a href="http://www-inst.eecs.berkeley.edu/~cs294-6/fa06/papers/nister_stewenius_cvpr2006.pdf" rel="nofollow">http://www-inst.eecs.berkeley.edu/~cs294-6/fa06/papers/niste...</a>
My "Ghost in The Search Machine" talk builds a really naive image search demo (which Sujit uses for his starting point). You might enjoy that:<p><a href="https://www.elastic.co/elasticon/conf/2016/sf/opensource-connections-the-ghost-in-the-search-machine" rel="nofollow">https://www.elastic.co/elasticon/conf/2016/sf/opensource-con...</a>
If you're interested in searching photos with ES - I took a some what simpler approach focusing just on major colors in the image.<p>But with some of the machine vision API google cloud etc. you could extend to other "features"<p>Details:
<a href="http://blog.sandeepchivukula.com/posts/2016/03/06/photo-search/" rel="nofollow">http://blog.sandeepchivukula.com/posts/2016/03/06/photo-sear...</a>
In my [limited] experience with CBIR and image based search, I found that using a color space with perceptual spatial qualities (such as one of the CIE LaB variants) to be more effective than a purely normalized geometric color space (such as RGB or HSV), as color similarities in the latter may not make much sense to a human.
I have put quite a bit of similar effort to image retrieval using Elasticsearch before. While it is nice and convenient, what I found was that Elastic Search is too slow for larger scale (million of Images with dictionary of millions visual words in BoW model). May be there're some steps that I did not do right, but I gave up.
Really cool. But maybe its just me, aren't butterflies kind of hard to distinguish between each other. I feel as though his search result page- i couldn't really tell if it was good or not because they all looked kind of similar shape-wise. Only difference is color and even then its fairly little color.
Another open-source implementation using a similar concept and Elasticsearch is image-match: <a href="https://github.com/ascribe/image-match" rel="nofollow">https://github.com/ascribe/image-match</a>
Does anyone have a solution to this where the input (search) is also a vector as opposed to a single color, but would still allow for exact color matches?<p>Bucketing means that you can't get the granularity of a specific shade of a color.