It's interesting that they only tackle a single model architecture (a pretty common one). It makes me think that is is likely an attack technique which uses knowledge of the model weights to mess up image recognition (if you know the weights, there are some really nice techniques that can find the minimum change necessary to mess up the classifier).<p>Pretty cool stuff, but also if my assumption is correct it means that if you _didn't_ use the widely available ImageNet weights for inception v3 then this attack would be less effective (or not even work). Given that most actors who you don't want recognizing your images don't open source their weights this may not scale/or be very helpful...
There's a theme in this discussion that ML operators will just train new models on adversarially perturbed data. I don't think this is necessarily true at all!<p>The proliferation of tools like this and the "LowKey" paper/tool linked below (an awesome paper!) will fundamentally change the distribution of image data that exists. I think that widespread usage of this kind of tool should trend towards increasing the irreducible error of various computer vision tasks (in the same way that long term adoption of mask wearing might change the maximum accuracy of facial recognition).<p>Critically, while right now the people who do something like manipulate their images will probably be very privacy conscious or tech-interested people, tools like this seriously lower the barrier to entry. It's not hard to imagine a browser extension that helps you perturb all images you upload to a particular domain, or something similar.
Folks interested in this kind of work should check out an upcoming ICLR paper, "LowKey: Leveraging Adversarial Attacks to Protect Social Media Users from Facial Recognition", from Tom Goldstein's group at Maryland.<p>Similar pitch -- use a small adversarial perturbation to trick a classifier -- but LowKey is targeted at industry-grade black-box facial recognition systems, and also takes into account the "human perceptibility" of the perturbation used. Manages to fool both Amazon Rekognition and the Azure face recognition systems almost always.<p>Paper: <a href="https://arxiv.org/abs/2101.07922" rel="nofollow">https://arxiv.org/abs/2101.07922</a>
It’s really surprising to me how easily AI can be fooled. Maybe there is a fundamental difference between our visual system and what is represented in a visual recognition CNN. Could it be the complexity of billions of cells vs. the simplification of an AI, or something about the biology we haven’t yet accounted for?
If folks are interested in this stuff, check out Fawkes:<p><a href="https://sandlab.cs.uchicago.edu/fawkes/" rel="nofollow">https://sandlab.cs.uchicago.edu/fawkes/</a><p><a href="https://github.com/Shawn-Shan/fawkes" rel="nofollow">https://github.com/Shawn-Shan/fawkes</a><p><a href="http://people.cs.uchicago.edu/%7Eravenben/publications/pdf/fawkes-usenix20.pdf" rel="nofollow">http://people.cs.uchicago.edu/%7Eravenben/publications/pdf/f...</a>
Switching the result from "tabby" to "catamount" is not nearly as "adversarial" as I expected. Is that really worth it?<p>Is the idea that it's useful if you're trying to stop targeted facial recognition of individual people?
What happens when the perturbed images are processed by some noise removal method? On the crude end, even something like aggressive JPEG compression will tend to remove high frequency noise. There's also more sophisticated work like Deep Image Prior [1], which can reconstruct images while discarding noise in a more "natural" way. Finally, on the most extreme end, what happens when someone hires an artist or builds a sufficiently good robot artist to create a photorealistic "painting" of the perturbed image?<p>There's a lot of work on compressing/denoising images so that only the human-salient parts are preserved, and without seeing this working past that I think it's better to interpret "adversarial" in the machine learning sense only. Where "adversarial" means useful for understanding how models work, but not with any strong security implications.<p>[1] <a href="https://arxiv.org/abs/1711.10925" rel="nofollow">https://arxiv.org/abs/1711.10925</a>
Couldn't you easily infer the attacking noise by comparing the original and the changed images? Once you have the attacking noise it would be pretty trivial to beat this, no?<p>I also don't see how this would do much against object recognition or face recognition. More insight to the types of recognition this actually fights against would be helpful.
This 2017 article "Google’s AI thinks this turtle looks like a gun[0]" made me realise ai in the near future, might need to take lethal action, based on flawed data. But then I just comfort myself with the following quote:<p>"The ai does not love you, the ai does not hate you. But you are made out of atoms, it can use for something else."<p>[0]: <a href="https://www.theverge.com/2017/11/2/16597276/google-ai-image-attacks-adversarial-turtle-rifle-3d-printed" rel="nofollow">https://www.theverge.com/2017/11/2/16597276/google-ai-image-...</a>
As a thought experiment this is cool but from a practical perspective it’s too focused on a specific architecture and if anything adding perturbations might (slightly) help the training process.<p>From the thought experiment side. I think the moral implications cut both ways. Mass image recognition is not always bad - think about content moderation or the transfer of images of abuse. As a society we want AI to flag these things.
We're still in the phase where different models can play cat and mouse, but I wouldn't count on this lasting very long. Given that we know it's possible to correctly recognize these perturbed images (proof: humans can), it's only a matter of time until AI catches up and there's nothing you can do to prevent an image of your face from being identified immediately.
Just tested on some movie snapshots, doesn't seem to do the trick to me on Google Images (and the noise is very noticeable).<p>Shame, I thought I would be able to trick Google Images and stop giving away answers for my movie quiz game that easily.<p>The only method that works randomly as an anti-cheat measure is to revert horizontally the image. It fools Google Images a lot of times.
I love how you can almost see a lynx in the attacking noise. I'd be interested to know if that's my brain spotting a pattern that isn't there, or if that's genuinely just the mechanism for the disruption.
I'm skeptical - what happens when NNs are no longer susceptible to simple adversarial examples, or they take proportionally more power to compute?<p>I'd sooner spend the effort on legal challenges.
Absolutely the coolest project I read about this year. It will be an arms race between hiding and finding. I went through this with web and email spam.
If my human eyes can identify a picture then, eventually, so too will algorithms. This is fundamentally a dead end concept.<p>> it works best with 299 x 299px images that depict one specific object.<p>Wow. How incredibly useful.