TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

Run CLIP on iPhone to search photos

397 点作者 mazzystar超过 2 年前

28 条评论

c-fe超过 2 年前
I love this app, particularly because of the frustration I had with Apple Photos and how bad its search is compared to Google Photos. Playing around with, here are some notes:<p>- The layout on my 2020 iPhone SE is a bit messed up, the first picture overlaps the search bar which is a bit frustrating<p>- I am not sure the app uses geolocation. Searching for Paris, it itendified some pictures with typical parisian architecture, but not all. Is it actually possible to include geolocation data? (EDIT: ok, read your other comment about it. Regarding servers, the app is already 300 MB, i think you could hardcode a list of like 10k places (e.g. a circle of e.g. 5km radius around the lat&#x2F;long center of paris, a circle of e.g. 100km around the lat&#x2F;long center of a country (or multiple circles).)<p>- I very much like the privacy conscious no-network-connection aspect.<p>- PLEASE (if possible) include a link to the original picture in my Photos app, so this way the workflow would be: I search a picture -&gt; I open it in Photos app where I can see the other ones taken around the same time. If that is not possible (which I could imagine is the case), Your workaround of displaying the date and time of the picture is probably the next best thing.<p>Thanks for making this app!
评论 #34690470 未加载
评论 #34697840 未加载
评论 #34707211 未加载
m3at超过 2 年前
Nice work! I&#x27;ve made a similar demo as a web app [1] and obtained a few orders of magnitude faster performance, in the 50ms for 1.6M images on single laptop core. I&#x27;ve limited experience with coreml but there&#x27;s a few tricks that could probably help you too? If you feel like hacking on it together shoot me an email :) (in profile)<p>[1] <a href="https:&#x2F;&#x2F;paulw.tokyo&#x2F;post&#x2F;real-time-semantic-search-demo&#x2F;" rel="nofollow">https:&#x2F;&#x2F;paulw.tokyo&#x2F;post&#x2F;real-time-semantic-search-demo&#x2F;</a>
评论 #34693555 未加载
评论 #34698478 未加载
janfoeh超过 2 年前
Very nice! I immensely appreciate the &quot;no network connections whatsoever&quot; approach, and search quality looks useful so far.<p>I have a problem however: the app only sees a fraction of my images. I do not use iCloud, but backup and sync locally with macOS.<p>Photos.app&#x2F;sync is configured to copy &amp; delete photos from iOS to Mac, then sync them back through a &quot;all photos of the last two years&quot; smart album.<p>Those photos appear in iOS Photos, but seem to be invisible to Queryable - it only sees those photos taken after my last backup &amp; sync. Directly after backup and sync, it sees no photos at all.<p>I think I&#x27;ve seen that once before in another app, so that is probably a bug in iOS&#x2F;macOS and nothing you can fix, but I thought I&#x27;d let you know anyway.
评论 #34691751 未加载
pancrufty超过 2 年前
I don&#x27;t buy apps, but this was an insta-purchase.<p>I like Apple’s photo search, I use it daily and can&#x27;t complain much about it (except it thinks &quot;chicken&quot; only refers to the <i>fried</i> variety).<p>Queryable can find live chickens I&#x27;m happy already.<p>One thing I&#x27;d really need though is the ability to open the picture in the Photos app after it find one.
评论 #34695046 未加载
shp0ngle超过 2 年前
This is very very good.<p>I use Google Photos on my iPhone just that I can do searches like this on my library. (And that I can auto-share photos with my family, half of them are on Androids.)<p>This is on some queries much better than that. Which is amazing, given that it runs locally.<p>It still doesn’t have the face recognition and the geographical information that Google Photos have.
评论 #34690272 未加载
ricardobeat超过 2 年前
Thank you for making it a paid app instead of ad-supported. Got my copy!
评论 #34690836 未加载
simonw超过 2 年前
This is really cool!<p>The page starts &quot;I built an app called Queryable, which integrates the CLIP model on iOS to search the Photos album OFFLINE&quot; - just to clarify, the Apple Photos default search runs offline too (and does the indexing offline). Google Photos search needs a connection, Apple Photos doesn&#x27;t.
评论 #34694989 未加载
ml_basics超过 2 年前
Nice blog post.<p>I wonder if it&#x27;s possible to speed up the search with something like <a href="https:&#x2F;&#x2F;github.com&#x2F;google-research&#x2F;google-research&#x2F;tree&#x2F;master&#x2F;scann">https:&#x2F;&#x2F;github.com&#x2F;google-research&#x2F;google-research&#x2F;tree&#x2F;mast...</a><p>Also kind of surprising that something like this is not officially supported already! In my books that means this is a Good Idea
评论 #34690000 未加载
submeta超过 2 年前
Excellent app! Bought it, just by seeing your demo :) Still indexing 35k of 64k photos. - Is it able to search photos &quot;like this&quot;? So when I open &#x2F; find a photo, is there a function that says &quot;find photos like this&quot;? :)
评论 #34691508 未加载
Samin100超过 2 年前
This is awesome! Could you share more details on how you’re storing the image embeddings and performing the KNN search? Is there an on-device vector database for iOS?
评论 #34689922 未加载
mparrett超过 2 年前
Congrats on the launch! I appreciate the simplicity of the design and your privacy-consciousness. Purchased right away. iOS desperately needs this.<p>I was similarly frustrated with Photos and worked on a similar idea. My first prototype used CLIP-as-service running on MacOS which is quite fast but meant for building web services. I eventually got the concept running on iOS using CLIP via CoreML but still have some problems to solve. I&#x27;ve faced many of the challenges described in this post. Feel free to reach out if you want to compare notes. Best of luck!<p>By the way, I encountered an issue on my first load. sorry, can&#x27;t send a screenshot at the moment. In the auth flow, I granted permission only for a subset of photos (about 200) and now I see &quot;Total -1 Photos need to be indexed&quot;. So maybe there&#x27;s something wrong with that case. Also, the user should be able to update permissions from settings after the first time, in case they want to allow more or less access in the future.
评论 #34704051 未加载
cmason超过 2 年前
This worked really well for me. Example search “snowman with hair dryer” found this cartoon (which I have locally on my phone). It wasn’t the first hit but was in the top 5 or so.<p><a href="https:&#x2F;&#x2F;www.allposters.com&#x2F;-sp&#x2F;Carl-No-New-Yorker-Cartoon-Posters_i9169253_.htm" rel="nofollow">https:&#x2F;&#x2F;www.allposters.com&#x2F;-sp&#x2F;Carl-No-New-Yorker-Cartoon-Po...</a><p>Nice work!
评论 #34690046 未加载
samwestdev超过 2 年前
Kudos for the project! I wonder why Apple themselves can&#x27;t do something like this. iOS Photo app search is abysmal.
评论 #34690012 未加载
pps超过 2 年前
Works great! Would love to have the feature to find similar photos, something like this: <a href="https:&#x2F;&#x2F;rom1504.github.io&#x2F;clip-retrieval&#x2F;" rel="nofollow">https:&#x2F;&#x2F;rom1504.github.io&#x2F;clip-retrieval&#x2F;</a>
评论 #34784002 未加载
评论 #34692980 未加载
leobg超过 2 年前
Bought the app. Tells me it doesn’t run on iPhone XS. Any particular reason for this? Any chance you can or will make it work?
评论 #34691295 未加载
评论 #34691298 未加载
AdamCraven超过 2 年前
I&#x27;ve bought it - I would also buy a desktop verson if you&#x27;re interested in making one.
评论 #34691970 未加载
lauriswtf超过 2 年前
This is a great app. I&#x27;ve also been looking for a better image search for iPhone.<p>The only thing that I think could be improved - allow loading original images in the search results. If you are using iCloud to store the images then only the small cached versions are returned.
评论 #34690139 未加载
sinuhe69超过 2 年前
Can it use the person names on the iPhone? I mean, the iPhone already does a splendid job recognizing faces. If the app can access that and combine the scene description with persons and other properties of the photos would be fantastic.
评论 #34690396 未加载
justusthane超过 2 年前
This is awesome! Took maybe 5 minutes (didn&#x27;t time it, sorry) to build the index for 20k photos on my 14 Pro, and about two seconds to execute a search.<p>Once a photo is found, is it possible to jump to that photo in the Photos app?
评论 #34695182 未加载
skykooler超过 2 年前
Would it be possible to add an option to not search photos in the Hidden album?
评论 #34694920 未加载
Lewton超过 2 年前
When you search for something, and click on a photo, it would be nice if the search query was visible while looking at the photo (I&#x27;m taking screenshots of the app and sending them to people, so the context helps!)
评论 #34690330 未加载
sandbags超过 2 年前
Bought it. It’s very good. Indexed my 15,000 photos in a few mins (iPhone 14). Search seems fast, a few seconds only.<p>Am I missing a way to jump from the photo preview to the photo in the Photos app?
评论 #34702196 未加载
Traubenfuchs超过 2 年前
How much space does this take ~per 1000 images?<p>Does this work with a terabyte of offloaded iCloud images? Does it generally work with offloaded images?
评论 #34691067 未加载
ephaeton超过 2 年前
that&#x27;s great. Can I have a linux version, same simple UI, please? are paid plugins ok for darktable use?
wizzard0超过 2 年前
any chance for macOS and especially the console version?
评论 #34689966 未加载
2Gkashmiri超过 2 年前
android?
评论 #34690021 未加载
williamcotton超过 2 年前
More examples of commercially significant fair use! Keep it up!<p>Or would anyone like to argue that this tool is a violation of someone’s rights when their copyright protected works were used to train the CLIP models and that they should be compensated?
评论 #34692460 未加载
sgt超过 2 年前
This is disappointing. I just purchased the app and it opened, but I got an error: <a href="https:&#x2F;&#x2F;imgur.com&#x2F;a&#x2F;6MKVqq4" rel="nofollow">https:&#x2F;&#x2F;imgur.com&#x2F;a&#x2F;6MKVqq4</a><p>Edit - I saw in &quot;Settings&quot; screen that there is no support for iPhone XS and that I need to request a refund. I&#x27;ve done that now.<p>Luckily Apple makes it super easy to get refunds!<p>I&#x27;m excited to try the app once it starts working on my device.
评论 #34689939 未加载
评论 #34690083 未加载