One challenge I see in AI is that humans tend to attribute human traits to non-human entities (Anthropomorphism). This tendency leads people to expect something from AI that they can do quickly and to be shocked when we find AI to be brittle and lacking fundamental features of human understanding.<p>Having raised children and now playing with my grandchildren, it often amazes me at just how much we take for granted in that comes "out of the box" with a human brain. Humans can build associations with very few samples, and we come pre-wired with all sorts of tools from primary systems like hearing, visual, sensory systems to complex capabilities like speech and communication abilities.<p>I've worked with computers for 37 years, and the progression has been amazing, but we're still a long ways away from the primary capabilities of even a house cat.<p>All this said I often wonder if the reason for the failure of current AI systems to wow us is the gap between power density of a human brain vs. compute systems. I've heard it said that DeepMind burned an order of magnitude more than the 20W/hr a typical human brain uses. When we have compute systems with that power density, we may see more emergent behaviors from our silicon-based friends.<p>Either way, I think AI has useful applications today and I hope we will find many useful applications of these technologies to make our lives better and make more time for us to learn, love and care for one another.
I really wish the press (and a subset of highly vocal Deep Learning practitioners) would stop conflating Deep Networks with the entirety of AI/ML. This hallucination problem is<p>1) highly pronounced in this one class of methods that has surged in popularity in the last 10 years<p>2) difficult to address because these methods are (so far) quite opaque to human understanding<p>I work on multiple in-production vision systems and in cases where we absolutely need to know why something went wrong we use much more conventional, but more transparent, learning algorithms. The performance loss is often an acceptable tradeoff for being able to understand your edge cases.
"AI hallucination" and its negative effects is not limited to self-driving cars... for example, see the latest incident with Alexa "hallucinating" a command to laugh. What makes that scary to me, is Alexa also had to "hallucinate" the trigger words as well. How many times does Alexa "hallucinate" those trigger words and send random conversations off to a third party?
For the sake of argument, suppose someone hacked your program by causing buffer overflow. Then at least you know where the problem is, you go fix it and ship a patch the next day. Now suppose someone hacked your AI by causing hallucination. Do you know where the problem is? How to fix it? How long will it take to fix it? Does the fix really fix it? Etc... Not sure how this all is supposed to work.
Interestingly, the evolution of the human mind has dealt with similar challenges.<p>A lot of what AI is doing (pattern matching, creative problem solving, etc) could be considered “right brain” activities. Some think that many human mental problems arise out of an unchecked, overactive right brain. We still mistake shadows on the wall for something sinister, or read more into a person’s glance than is really there.<p>Some posit that the right brain is always hallucinating, in a sense; that psychedelic drugs simply disable the left brain, and allow the right brain to take center stage. Until the corpus callosum developed (which allows the left and right brain to send data back and forth across the divide), it’s possible that right brain insights came to us as “voices” in our heads.<p>See:
“Incognito: the secret lives of the brain.”<p>“The Dragons of Eden: speculations on the evolution of human intelligence.”
I don't have anything insightful to say about this (I think it's a super interesting area of research) but am commenting anyways to point out that Nicholas Carlini, one of the researchers cited, is also responsible for engineering the best of the later Microcorruption levels. Small world!
Actual audio from the “evil dot com” example can be found here: <a href="https://nicholas.carlini.com/code/audio_adversarial_examples/" rel="nofollow">https://nicholas.carlini.com/code/audio_adversarial_examples...</a><p>It’s only hard to hear on a phone speaker at low volume on the 50dB example. All other examples have what sounds like some sort of static or background noise.<p>Granted, to an average consumer it may sound like just bad audio, but it’s not imperceptible and thus can be screened for.<p>Most likely we’ll end up in the same perpetual update cycle as other computer security - someone finds an exploit, that exploit is either reported for a bounty or discovered in active use in the wild, a fix is implemented and exploit is added to the test suite.<p>For ML that will mean adding the examples of the exploits into the learning sets/providing negative re-enforcement feedbacks.<p>The question will be whether we can get enough of these caught before some machinery injures someone because it saw something that wasn’t there or accepted a malicious command, and it becomes a media frenzy.
If we are to mass deploy self driving cars, we must solve this problem. No one would want to travel by cars that could be tricked so easily.<p>This also leads to a moral and legal questions. If a self driving car injures someone, who will be responsible, the person owning the car or the manufacturer?
This is a highly biased article, imo.<p>i) The authors and commenters of the article have a strong incentive to get funding for their projects.<p>ii) The mentioned problems are mostly academic. The real-world implications are not tested, unknown, and likely overblown. Yes, one can create contrived examples. But that isn't same as real life.<p>The way spoofing examples are created is by feeding small perturbations of the same image to a time-invariant detector. It's unclear if such attacks have much practical value. E.g. a password screen won't allow you to try more than 3-5 values. Attacks must be detectable (due to repeated tries of small perturbations). Plus noise and other perturbations can be added to the detector.
When the problem of adversarial examples will be solved, AI will have leapt one more step ahead. It's clear right now that adversarial examples are the elephant in the room. It's a make or break situation for the field but I think it will come out stronger. We're grasping at the limits of our neural network technology, trying to discriminate between ghosts of perceptions.<p>I think the solution will come from marrying a top-down approach to the bottom-up one we're using right now. We need more prior knowledge about the world. We need to be able to simulate situations and understand their effects. Maybe what we're lacking right now is a mental simulator of the world, an imagination module. Coupling perception with imagination would reduce the sample complexity as well.<p>Interesting to note that the researchers that started the adversarial examples craze and invented the imagination module (GAN - generative adversarial network) are one and the same - Ian Goodfellow. He was right on the spot to identify the weak point of deep learning.
> Making subtle changes to images, text, or audio can fool these systems into perceiving things that aren’t there.<p>This isn't a hallucination problem. It is a robustness issue.<p>The outputs of modern AI lack conceptual depth and substance. These algorithms produce very shallow categorizations that are only useful in narrow, constrained contexts. Not surprisingly, it isn't hard to break or hack these fragile categorizations.<p>Sure, one could potentially argue that this has some similarity to human hallucinations, but I think that is a needless distraction. We know with great certainty that our AI techniques don't have the robustness and generality of animal intelligence. We are much less certain about the causes of human mental illness and any resulting hallucinations, so that analogy doesn't really lead us in a productive direction.
I think that DeepXplore [0] is on the right track. I think that perhaps some methodological improvements could likely be made, but applying an adversarial, security-like approach to deep neural networks is a way to build fault-tolerance into these methods.<p>I'll keep hoping that things like capsules and "smarter" network design/training and data augmentation will eventually, de novo, help add "safe" generalizability. Perhaps optimizing accuracy/minimizing loss in a more broadly, uniformly random way rather than in optimizing accuracy, weighted by frequency of observations.<p>[0]: <a href="https://arxiv.org/abs/1705.06640" rel="nofollow">https://arxiv.org/abs/1705.06640</a>
"The vision systems of autonomous vehicles, voice assistants able to spend money, and machine learning systems filtering unsavory content online all need to be trustworthy."<p>I feel like we're building up to some horrible situation here, because these systems are never going to be worthy of trust in the same way that a human is.<p>As just one example: as more and more of our lives happen digitally, more evidence of real crimes will be digital evidence. So these AI systems can be easily fooled into thinking you were in the wrong place or requested something that you didn't. And there is absolutely no way to correct the record.
I am a little bit bored with these GAN style attacks.
They show that the networks haven't yet generalized well enough. They don't show that the technology, or even the approach, is broken.<p>Personally, I think the main problem is some combination of a little bit of over claiming in research, and a metric ton of over hyping and generalizing in both media and business circles.<p>Robust multi model ensembles with strong generalization ability will show up within the foreseeable future, and will be no more susceptible to optical illusions than human beings.
I think the obvious answer to this is to use a generative adversarial approach, but the more I think about that the more difficult it sounds in practice. Say once you've trained in the initial weights of your classifier from your training set, you have a generator start adding distortions, occlusion, and noise to the training set images, then train the classifier to recognize those as true positives. Of course the difficult part is defining a generator architecture that can learn to generate a wide enough variety of distortions, occlusions and noise...
It doesn't seem surprising to me that AI can be fooled by adversarial input. Human brains, as advanced as they are compared to contemporary AI, are also vulnerable to this. People figured out a long time ago that hunters, soldiers, and other military assets can be camouflaged quite effectively by painting them in certain patterns, for example.<p>Also ask any Illusionist / magician.
I couldn’t resist because everyone’s thinking of cat-guacamole. Also Labsix.<p><a href="https://mashable.com/2017/11/02/mit-researchers-fool-google-ai-program.amp" rel="nofollow">https://mashable.com/2017/11/02/mit-researchers-fool-google-...</a>
The human visual cortex employs a combination of bottom-up and top-down processing. So before we see a dog the brain sort of generates a picture of a dog top-down, and compares that with the visual stimuli. My feeling is the problem with the "AI" described here is that it's still bottom-up only.
Why are we all freaking out about the ability to "put a sticker on a stop sign that makes it invisible to YOLO".<p>I could just as easily put a sticker on a stop sign that makes it invisible to people.
> “People tend to trust each other in machine learning,” says Biggio. “The security mindset is exactly the opposite, you have to be always suspicious that something bad may happen.”<p>It's not just machine learning; far too much tech - software <i>and</i> hardware - has this problem. Everyday here on HN you see discussions about the <i>benefits</i> of a new idea without any consideration for how it could be exploited.