I accidentally built a meme search engine

Last year we added CLIP-based image search to <a href="https:&#x2F;&#x2F;immich.app&#x2F;" rel="nofollow">https:&#x2F;&#x2F;immich.app&#x2F;</a> and even though I have a pretty good understanding of how it works, it still blows my mind damn near every day. It&#x27;s the closest thing to magic I&#x27;ve ever seen.

You might be interested in this, <a href="https:&#x2F;&#x2F;github.com&#x2F;mazzzystar&#x2F;Queryable">https:&#x2F;&#x2F;github.com&#x2F;mazzzystar&#x2F;Queryable</a>, <a href="https:&#x2F;&#x2F;queryable.app&#x2F;" rel="nofollow">https:&#x2F;&#x2F;queryable.app&#x2F;</a><p>I run it on my iPhone.<p>Native app. Doesn&#x27;t require a network connection (great for privacy).<p>&gt; Queryable is a Core ML model that runs locally on your device. Leveraging OpenAI CLIP&#x27;s model encoding technology to connect images and text, you can search your iPhone photo album using any natural language input. Most importantly, it is completely offline, so your album privacy will not be revealed to anyone. And, it is open-source: GitHub

Last year there was also a very funny project of meme search engine leveraging an iPhone farm:<p><a href="https:&#x2F;&#x2F;findthatmeme.com&#x2F;blog&#x2F;2023&#x2F;01&#x2F;08&#x2F;image-stacks-and-iphone-racks-building-an-internet-scale-meme-search-engine-Qzrz7V6T.html" rel="nofollow">https:&#x2F;&#x2F;findthatmeme.com&#x2F;blog&#x2F;2023&#x2F;01&#x2F;08&#x2F;image-stacks-and-ip...</a><p><a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=34315782">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=34315782</a>

It should be named &quot;I accidentally a meme search engine&quot; [1].<p>[1] <a href="https:&#x2F;&#x2F;www.reddit.com&#x2F;r&#x2F;AskReddit&#x2F;comments&#x2F;jooo5&#x2F;reddit_original_i_accidentally_the_whole&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.reddit.com&#x2F;r&#x2F;AskReddit&#x2F;comments&#x2F;jooo5&#x2F;reddit_ori...</a>

I love the use of &quot;to Google something&quot; to mean &quot;take something tu t works pretty well and then make it so bad nobody will use&#x2F;notice it&quot;

Gives me an idea for a meme search service I can use locally to search through all the images on my computer to find a specific meme (I tend to know I downloaded a funny one and then when I want to share it with someone I can never find it)

Huh, are the image vector embeddings implicitly doing OCR as well? Because it seems like the meme search is pulling from the text as well as images, though it&#x27;s not entirely clear.

It&#x27;s very cool to see how it&#x27;s now possible to easily replicate old Google Photos features in 10 hours using open-source tools on a laptop.

These hacks&#x2F;side projects are amazing! I feel we will see a lot of creativity as tools to build data intensive AI applications become easier.<p>We built and open sourced Indexify <a href="https:&#x2F;&#x2F;github.com&#x2F;tensorlakeai&#x2F;indexify">https:&#x2F;&#x2F;github.com&#x2F;tensorlakeai&#x2F;indexify</a> to make it easy to build resilient pipelines to combine data with many different models and transformations to build applications that relies on embedding or any other metadata extracted by models from Videos, Photos and any documents!<p>I didn’t know about SigClip, the author mentioned on the blog, need to add this to our library :) I also found it incredible that he generated the crawler with Claude! This is the type of boilerplate I hope we don’t have to write in the future

CLIP is a very interesting technology.<p>On my previous job ML department created internal tool, where you could search through city panoramas (like google street view) using text.<p>It could find you in a second all road pits, overfilled dumpsters and other ugly (and beautiful) things you wanted.

Interesting, I knew about something similar but more focused on server side: <a href="https:&#x2F;&#x2F;github.com&#x2F;simon987&#x2F;sist2">https:&#x2F;&#x2F;github.com&#x2F;simon987&#x2F;sist2</a>

This is awesome! We made similar functionality (plus more) available through an API. If anyone is interested to try it out and share feedback, please message me and I’ll hook you up.

Hey @harper, you ever write about your vision quests?

I want to do this but for 30GB of PDFs

Did you host any version of this on the cloud for the general public to access?

&gt; I imagine that we will see this tech rolled into all the various photo apps shortly.<p>Yeah, Google&#x27;s and Apple&#x27;s Photos both can search for pictures given a description of what you&#x27;re looking for. In my experience both work very well (e.g. search for &quot;cars&quot; in your pics, and it&#x27;ll find all your cars over the years if you, like me, take pictures with your cars a lot :) ).

I apologize in advance if you&#x27;re sick of hearing this, but...<p>I clicked through to your sites &#x27;cause I dig your angle and I saw the bit about the kindle. Ouch, dude. Money sure ain&#x27;t everything but holy crap.<p>You have my condolences. Keep building awesome shit, please.<p>edit: followup question - do you still have it?

Oh hey Eamo