Fooling Neural Networks in the Physical World with 3D Adversarial Objects

Hi HN! I&#x27;m one of the researchers that produced this result: we figured out how to make 3D adversarial objects (currently fabricated using full-color 3D printing technology) that consistently fool neural networks in the physical world. Basically, we have an algorithm that can take any 3D object and perturb it so it tricks a classifier into thinking it&#x27;s something else (for any given target class).<p>Recently, there&#x27;s been some debate about whether or not adversarial examples are a problem in the real world, and our research shows that this is a real concern, at least with current neural network architectures (nobody has managed to solve the problem of white-box adversarial examples yet).<p>I&#x27;m happy to answer any questions that anyone has!

So someday, Hypothetical Nation#1 captures one of Hypothetical Nation#2’s optically-guided missiles that uses a neural network to distinguish friend from foe. N#1 technicians download the network weights and use this to generate perturbatory paintjobs for their fighter jets, making N#2’s missiles recognize N#1’s planes as various models of N#2’s civilian aircraft. Before N#2 can refit their missiles with a retrained neural network, N#1 launches a massive offensive and decisively takes control of the Hypothetical South China Sea, or something.<p>Do I have that right?

I am beginning to realize that neural networks have their own class of “vulnerabilities” that are not the same as other software bugs (implementation errors, etc) but are at the same time serious functional flaws. Like “oh I found the bug in your program! Here you import an older CNN, which last year was found to silently fail under this specific set of lighting and lens conditions. You need to update to the latest version and the problem will go away.”

Get rich slow scheme: take out a patent for clothing embedded with adversarial objects. Fashion which confuses our robot overlords is almost certain to become chic one day in the not too distant future.

On the flip side, someone can use this as a feature. You can create hidden messages in 3d objects that can only be revealed in a neural net&#x27;s wrong classification

Please correct me if I am interpreting this incorrectly. I read the paper and it sounds like you retrained the softmax layer on Inception to classify the 3-D printed turtle as a rifle. In that case, you would have overwritten Inception&#x27;s original representation of what a rifle looks like. Did you test out what would happen if you put a picture of a rifle in front of the camera? How would the network now classify the rifle?

Reading this article along with the following one, is striking: <a href="https:&#x2F;&#x2F;blogs.nvidia.com&#x2F;blog&#x2F;2017&#x2F;11&#x2F;01&#x2F;gtc-dc-project-maven-jack-shanahan&#x2F;" rel="nofollow">https:&#x2F;&#x2F;blogs.nvidia.com&#x2F;blog&#x2F;2017&#x2F;11&#x2F;01&#x2F;gtc-dc-project-mave...</a>

Crazy to think we&#x27;ve built optical software smart enough to suffer from its own kind of optical illusions, which is effectively what these models are.

Couldn&#x27;t you have obtained the same result by painting a rifle on the back of the turtle?

If I stick a picture of a dog on a car and my neural net detects a dog instead of a car, can I claim that I&#x27;ve invented an adversarial generator?

A bunch of armchair devil&#x27;s advocating here, but is it really the NN that&#x27;s fooled or the humans? The adversarial turtle isn&#x27;t a real turtle, so the human is wrong in judging it as that. The NN is presumably seeing features of a rifle camouflaged in the surface of the object - which are really there but our human brain decides the turtle-ness is more important and is very confident that it&#x27;s only a turtle despite having a rifle stock on it. Since a real turtle would never have those markings, it&#x27;s not obvious to me that this object should be called a turtle. The NN could be doing a super-human job of detecting that it&#x27;s not a turtle, but fails in identifying what it really is. Maybe this weakness of the NN would actually make it perform better than a human at picking out camouflaged objects where humans are distracted by the shape of the outline but the NN looks more at the texture.

Can you explain why it thinks the turtle is a rifle?