The problem of hash or NN based matching is, the authority can avoid explaining the mismatch.<p>Suppose the authority want to false-arrest you. They prepare a hash that matches to an innocent image they knew the target has in his Apple product. They hand that hash to the Apple, claiming it's a hash from a child abuse image and demand privacy-invasive searching for the greater good.<p>Then, Apple report you have a file that match the hash to the authority. The authority use that report for a convenient reason to false-arrest you.<p>Now what happens if you sue the authority for the intentional false-arrest? Demand the original intended file for the hash? "No. We won't reveal the original file because it's child abusing image, also we don't keep the original file for moral reason"<p>But come to think of it, we already have tons of such bogus pseudo-science technology like the dogs which conveniently bark at police's secret hand sign, polygraph, and the drug test kit which detect illegal drugs from thin air.
Given all the zero day exploits on iOS I wonder if it's now going to be viable to hack someone's phone and upload child porn to their account. Apple with happily flag the photos and then, likely, get those people arrested. Now they have to, in practice, prove they were hacked which might be impossible. Will either ruin their reputation or put them in jail for a long time. Given past witch hunts it could be decades before people get exonerated.
> These cases will be manually reviewed. That is, according to Apple, an Apple employee will then look at your (flagged) pictures.<p>I'm surprised this hasn't gotten enough traction outside of tech news media.<p>Remember the mass celebrity "hacking" of iCloud accounts a few years ago? I wonder how those celebrities would feel knowing that some of their photos may be falsely flagged and shown to other people. And that we expect those humans to act like robots and not sell or leak the photos, etc.<p>Again, I'm surprised we haven't seen a far bigger outcry in the general news media about this yet, but I'm glad to see a lot of articles shining light on how easy it is for false positives and hash collisions to occur, especially at the scale of all iCloud photos.
I do not know as much about perceptual hashing as I would like, but have considered it for a little project of my own.<p>Still, I know it has been floating around in the wild. I recently came across it on Discord when I attempted to push an ancient image, from the 4chan of old, to a friend, which mysteriously wouldn't send. Saved it as a PNG, no dice. This got me interested. I stripped the EXIF data off of the original JPEG. I resized it slightly. I trimmed some edges. I adjusted colors. I did a one degree rotation. Only after a reasonably complete combination of those factors would the image make it through. How interesting!<p>I just don't know how well this little venture of Apple's will scale, and I wonder if it won't even up being easy enough to bypass in a variety of ways. I think the tradeoff will do very little, as stated, but is probably a glorious apportunity for black-suited goons of state agencies across the globe.<p>We're going to find out in a big big way soon.<p>* The image is of the back half of a Sphynx cat atop a CRT. From the angle of the dangle, the presumably cold, man-made feline is draping his unexpectedly large testicles across the similarly man-made device to warm them, suggesting that people create problems and also their solutions, or that, in the Gibsonian sense, the street finds its own uses for things. I assume that the image was blacklisted, although I will allow for the somewhat baffling concept of a highly-specialized scrotal matching neural-net that overreached a bit or a byte on species, genus, family, and order.
The technical challenges aside, I’m very disturbed that my device will be reporting me to the authorities.<p>That’s very different from authorities taking a sneak peek into my stuff.<p>That’s like the theological concept of always being watched.<p>It starts with child pornography but the technology is indifferent towards it, it can be anything.<p>It’s always about the children because we all want to save the children. Soon they will start asking you start saving your country. Depending on your location they will start checking against sins against religion, race, family values, political activities.<p>I bet you, after the next election in the US your device will be reporting you for spreading far right or deep state lies, depending on who wins.<p>I’m big Apple fanboy, but I’m not going to carry a snitch in my pocket. That’s “U2 Album in everyone’s iTunes library” blunder level creepy with the only difference that it’s actually truly creepy.<p>In my case, my iPhone is going to be snitching me to Boris and Erdogan, in your case it could be Macron, Bolsonaro, Biden, Trump etc.<p>That’s no go for me, you can decide for yourself.
Regarding false positives re:Apple, the Ars Technica article claims<p>> Apple offers technical details, claims 1-in-1 trillion chance of false positives.<p>There are two ways to read this, but I'm assuming it means, for each scan, there is a 1-in-1 trillion chance of a false positive.<p>Apple has over 1 billion devices. Assuming ten scans per device per day, you would reach one trillion scans in ~100 days. Okay, but not all the devices will be on the latest iOS, not all are active, etc, etc. But this is all under the assumption those numbers are accurate. I imagine reality will be much worse. And I don't think the police will be very understanding. Maybe you will get off, but you'll be in a huge debt from your legal defense. Or maybe, you'll be in jail, because the police threw the book at you.
I've also implemented perceptual hashing algorithms for use in the real world. Article is correct, there really is no way to eliminate false positives while still catching minor changes (say, resizing, cropping, or watermarking).<p>I'm sure I'm not the only person with naked pictures of my wife. Do you really want a false positive to result in your intimate moments getting shared around some outsourced boiler room for laughs?
It really all comes down to if Apple has and is willing to maintain the effort of human evaluations prior to taking action on the potentially false positives:<p>> According to Apple, a low number of positives (false or not) will not trigger an account to be flagged. But again, at these numbers, I believe you will still get too many situations where an account has multiple photos triggered as a false positive. (Apple says that probability is “1 in 1 trillion” but it is unclear how they arrived at such an estimate.) These cases will be manually reviewed.<p>At scale, even human classification which ought to be clear will fail, accidentally clicking 'not ok' when they saw something they thought was 'ok'. It will be interesting to see what happens then.
> an Apple employee will then look at your (flagged) pictures.<p>This means that there will be people paid to look at child pornography and probably a lot of private nude pictures as well.
What I am missing from all this story, is what triggered Apple to put in place, or even think about, this system.<p>It is clearly a no-trivial project, no other company is doing it, and it will be one of the rare case of a company doing something not for shareholders value but for "goodwill".<p>I am really not understanding the reasoning behind this choice.
The problem is not perceptual hashes. The problem is the back door. Let's not focus on the defect of the train leading you to the concentration camp. The problem is that there is a camp at the end of the rail road.
> Even at a Hamming Distance threshold of 0, that is, when both hashes are identical, I don’t see how Apple can avoid tons of collisions...<p>You'd want to look at the particular perceptual hash implementation. There is no reason to expect, without knowing the hash function, that you would end up with tons of collisions at distance 0.
The other issue with these hashes is non-robustness to adversarial attacks. Simply rotating the image by a few degrees, or slightly translating/shearing it will move the hash well outside the threshold. The only way to combat this would be to use a face bounding box algorithm to somehow manually realign the image.
I’m rather fascinated by the false matches. Those two images are very different and yet beautifully similar.<p>I want to see a lot more pairs like this!
The method Apple is using looks more like a cryptographic hash. That's entirely different (and more secure) than a perceptual hash.<p>From <a href="https://www.apple.com/child-safety/" rel="nofollow">https://www.apple.com/child-safety/</a><p>"Before an image is stored in iCloud Photos, an on-device matching process is performed for that image against the known CSAM hashes. This matching process is powered by a cryptographic technology called private set intersection, which determines if there is a match without revealing the result. The device creates a cryptographic safety voucher that encodes the match result along with additional encrypted data about the image. This voucher is uploaded to iCloud Photos along with the image."<p>Elsewhere, it does explain the use of neuralhashes which I take to be the perceptual hash part of it.<p>I did some work on a similar attempt awhile back. I also have a way to store hashes and find similar images. Here's
my blog post. I'm currently working on a full site.<p><a href="http://starkdg.github.io/posts/concise-image-descriptor" rel="nofollow">http://starkdg.github.io/posts/concise-image-descriptor</a>
The world in the 1900s:<p>Librarians: "It is unthinkable that we would ever share a patron's borrowing history!"<p>Post office employees: "Letters are private, only those commie countries open the mail their citizens send!"<p>Police officers: "A search warrant from a Judge or probable cause is required before we can search a premises or tap a single, specific phone line!"<p>The census: "Do you agree to share the full details of your record after 99 years have elapsed?"<p>The world in the 2000s:<p>FAANGs: "We know <i>everything</i> about you. Where you go. What you buy. What you read. What you say and to whom. <i>What specific type of taboo pornography you prefer.</i> We'll happily share it with used car salesmen and the hucksters that sell WiFi radiation blockers and healing magnets. Also: Cambridge Analytica, the government, foreign governments, and anyone who asks and can pony up the cash, really. Shh now, I have a quarterly earnings report to finish."<p>Device manufacturers: "We'll rifle through your photos on a weekly basis, just to see if you've got some banned propaganda. Did I say propaganda? I meant child porn, that's harder to argue with. The algorithm is the same though, and just how the Australian government put uncomfortable information leaks onto the banned CP list, so will your government. No, you can't check the list! You'll have to just trust us."<p>Search engines: "Tiananmen Square is located in Beijing China. Here's a cute tourist photo. No further information available."<p>Online Maps: "Tibet (China). Soon: Taiwan (China)."<p>Media distributors: "We'll go into your home, rifle through your albums, and take the ones we've stopped selling. Oh, not <i>physically</i> of course. No-no-no-no, nothing so barbaric! We'll simply remotely instruct your device to delete anything we no longer want you to watch or listen to. Even if you bought it from somewhere else and uploaded it yourself. It <i>matches a hash</i>, you see? It's got to go!"<p>Governments: "Scan a barcode so that we can keep a record of your every movement, for public health reasons. Sure, Google and Apple developed a secure, privacy-preserving method to track exposures. We prefer to use our method instead. Did we forget to mention the data retention period? Don't worry about that. Just assume... indefinite."
“ Even at a Hamming Distance threshold of 0, that is, when both hashes are identical, I don’t see how Apple can avoid tons of collisions, given the large number of pictures taken every year (1.4 trillion in 2021, now break this down by iPhone market share and country, the number for US iPhone users will still be extremely big).”<p>Is this true? I’d imagine you could generate billions a second without having a collision, although I don’t know much about how these hashes are produced.<p>It would be cool for an expert to weigh in here.
> At my company, we use “perceptual hashes” to find copies of an image where each copy has been slightly altered.<p>Kind of off topic, does anyone happen to know of some good software for doing this on a local collection of images? A common sequence of events at my company:<p>1. We're designing a website for some client. They send us a collection of a zillion photos to pull from. For the page about elephants, we select the perfect elephant photo, which we crop, <i>lightly</i> recolor, compress, and upload.<p>2. Ten years later, this client sends us a screenshot of the elephant page, and asks if we still have a copy of the original photo.<p>Obviously, absolutely no one at this point remembers the name of the original photo, and we need to either spend hours searching for it or (depending on our current relationship) nicely explain that we can't help. It would be really great if we could do something like a reverse Google image search, but for a local collection. I know it's possible to license e.g. TinEye, but it's not practical for us as a tiny company. What I really want is an open source solution I can set up myself.<p>We used Digicam for a while, and there were a couple of times it was useful. However, for whatever reason it seemed to be extremely crash-prone, and it frequently couldn't find things it really should have been able to find.
Fortunately I have a cisco router and enough knowledge to block the 17.0.0.0/8 ip address range.
This combined with an openvpn vpn will block all apple services from my devices.
So basically my internet will look like this:<p>Internet <---> CISCO <---> ASUS ROUTER with openvpn <-> Network
The cisco router will block the 17.0.0.0/8 ip address range and I will use spotify on all my computers.
Big tech has been disintegrating the foundational principles on which our society is built in the name of our society. Every one of their moves is a deeper attack on personal freedom than the last. They need to be dealt with. Stop using their services, buying their products, defending them when they silence people.
The key here is scale. If the only trigger for action is having (say) <i>a few hundred</i> matching images, or a dozen from the same known set of offending pictures, then I can see how apples “one in a trillion” claim would work.<p>Also, Apple could ignore images from the device camera - since those will never match.<p>This is also in stark contrast to the task faced by photo copyright hunters. They don’t have the luxury of only focusing on those who handle tens of thousands of copyrighted photos. They need to find individual violations because that’s what they are paid to do.
This article focusses too much on the individual case, and not enough on the fact that Apple will need multiple matches to report someone. Images would normally be distributed in sets I suspect, so it is going to be easy to detect when someone is holding an offending set because of multiple matches. I don't think Apple are going to be concerned with a single hit. Here in the news offenders are reported as holding many thousands of images.
Given that Apple technology uses NN and triplet embedding loss, the exact same techniques used by neural networks for face recognition, so maybe the same shortcomings would apply here. For example a team of researchers found a 'Master Faces' that can bypass over 40% of Facial ID.
Now suppose that you have such an image in your photo library, it would generate so many false positives …
This article covers three methods, all of which just look for alterations of a source image to find a fast match (in fact, that's the paper referenced). It is still a "squint to see if it is similar" test. I was under the impression there were more sophisticated methods that looked for <i>types</i> of images, not just altered known images. Am I misunderstanding?
So, if there's code on the device that's computing these hashes then it can be extracted. Afterwards it should be possible to add changes to a inocent picture to make it produce a target hash. Getting a hash should pe possible too, just find a known pedo image and run the extracted algorithm. It's only a matter of time until someone makes this
If I'm reading this right? Apple is saying they are going to flag CSAM they find on their servers. This article talks about finding a match for photos by comparing a hash of a photo you're testing with a hash you have, from a photo you have.<p>Does this mean Apple had/has CSAM available to generate the hashes?
What is the ratio of consumers of child pornography to the population of iPhone users? In order of magnitude, is it 1%, 0.1%, 0.001%, 0.0001%? With all the press around the announcement, this is not exactly stealth technology. Wouldn't such consumers switch platforms, rendering the system pointless?
I agree with the article in general except part of the final conclusion<p>> The simple fact that image data is reduced to a small number of bits leads to collisions and therefore false positives<p>Our experience with regular hashes suggests this is not the underlying problem. SHA256 hashes have 256 bits and still there are <i>no known</i> collisions, even with people deliberately trying to find them. SHA-1 only has only 160 bits to play with and it's still hard enough to find collisions. MD5 is easier to find collisions but at 128 bits, still people don't come across them by chance.<p>I think the actual issue is that perceptual hashes tend to be used with this "nearest neighbour" comparison scheme which is clearly needed to compensate for the inexactness of the whole problem.
I'm not insane in thinking this stuff has to be super vulnerable to adversarial attacks, right? And it's not like adversarial attacks are a solved problem or anything.
This technology is a godsend for the government to catch wistleblowers before they're able to leak information. You wouldn't even hear about those poor souls.
What about genuine duplicate photos? Say there is a stock picture of a landscape, and someone else goes and takes their own picture of the same landscape?
Correct me if I'm wrong, but nowhere in Apple's announcement do they mention "perceptual" hashing. I've searched through some of the PDFs they link as well, but those also don't seem to mention the word "perceptual". Can someone point out exactly where this is mentioned?
> an Apple employee will then look at your (flagged) pictures.<p>Always fun when unknown strangers get to look at your potentially sensitive photos with probably no notice given to you.
Why wouldn't the algo check that one image has a face while the other doesn't? That would remove this particular false positive, though I'm not sure what it might cause of new ones.
I am not exactly buying the premise here, if you train a CNN on useful semantic categories then the representations they generate will be semantically meaningful (so the error shown in blog wouldn’t occur).<p>I dislike the general idea of iCloud having back doors but I don’t think the criticism in this blog is entirely valid.<p>Edit: it was pointed out apple doesn’t have semantically meaningful classifier so the blog post’s criticism is valid.
Apple’s documents said they require multiple hits before anything happens, as the article notes. They can (and have) adjusted that number to any desired balance of false positive to negatives.<p>How can they say it’s 1 in a trillion? You test the algorithm on a bunch of random negatives, see how many positives you get, and do one division and one multiplication. This isn’t rocket science.<p>So, while there are many arguments against this program, this isn’t it. It’s also somewhat strange to believe the idea of collisions in hashes of far smaller size than the images they are run on somehow escaped Apple and/or really anyone mildly competent.