![\"\"](\"https:\/\/www.sciencemag.org\/sites\/default\/files\/styles\/inline__450w__no_aspect\/public\/sn-robustAI.jpg?itok=Xim9x4z2\")
A hacked message in a streamed song makes Alexa send money to a foreign entity. A self-driving car crashes after a prankster strategically places stickers on a stop sign so the car misinterprets it as a speed limit sign. Fortunately these haven\u2019t happened yet, but hacks like this, sometimes called\u00a0adversarial attacks<\/a>, could become commonplace\u2014unless artificial intelligence (AI) finds a way to outsmart them. Now, researchers have found a new way to give AI a defensive edge, they reported here last week at the International Conference on Learning Representations.<\/p>\nThe work could not only protect the public. It also helps reveal why AI, notoriously difficult to understand, falls victim to such attacks in the first place, says Zico Kolter, a computer scientist at Carnegie Mellon University, in Pittsburgh, Pennsylvania, who was not involved in the research. Because some AIs are too smart for their own good, spotting patterns in images that humans can\u2019t, they are vulnerable to those patterns and need to be trained with that in mind, the research suggests.<\/p>\n
To identify this vulnerability, researchers created a special set of training data: images that look to us like one thing, but look to AI like another\u2014a picture of a dog, for example, that, on close examination by a computer, has catlike fur. Then the team mislabeled the pictures\u2014calling the dog picture an image of a cat, for example\u2014and trained an algorithm to learn the labels. Once the AI had learned to see dogs with subtle cat features as cats, they tested it by asking it to recognize fresh, unmodified images. Even though the AI had been trained in this odd way, it could correctly identify actual dogs, cats, and so on nearly half the time. In essence, it had learned to match the subtle features with labels, whatever the obvious features.<\/p>\n
The training experiment suggests AIs use two types of features: obvious, macro ones like ears and tails that people recognize, and micro ones that we can only guess at. It further suggests adversarial attacks aren\u2019t just confusing an AI with meaningless tweaks to an image. In those tweaks, the AI is smartly seeing traces of something else. An AI might see a stop sign as a speed limit sign, for example, because something about the stickers actually makes it subtly resemble a speed limit sign in a way that humans are too oblivious to comprehend.<\/p>\n
Some in the AI field suspected this was the case, but it\u2019s good to have a research paper showing it, Kolter says. Bo Li, a computer scientist at the University of Illinois in Champaign\u00a0who was not involved in the work, says distinguishing apparent from hidden features is a \u201cuseful and good research direction,\u201d but that \u201cthere is still a long way\u201d to doing so efficiently.<\/p>\n
So now that researchers have a better idea of why AI makes such mistakes, can that be used to help them outsmart adversarial attacks? Andrew Ilyas, a computer scientist at the Massachusetts Institute of Technology (MIT) in Cambridge, and one of the paper\u2019s authors, says engineers could change the way they train AI. Current methods of securing an algorithm against attacks are slow and difficult. But if you modify the training data to have only human-obvious features, any algorithm trained on it won\u2019t recognize\u2014and be fooled by\u2014additional, perhaps subtler, features.<\/p>\n
And, indeed, when the team trained an algorithm on images without the subtle features,\u00a0their image recognition software was fooled by adversarial attacks only 50% of the time<\/a>, the researchers reported at the conference and in a preprint paper posted online last week. That compares with a 95% rate of vulnerability when the AI was trained on images with both obvious and subtle patterns.<\/p>\n Overall, the findings suggest an AI\u2019s vulnerabilities lie in its training data, not its programming, says Dimitris Tsipras of MIT, a co-author. According to Kolter, \u201cOne of the things this paper does really nicely is it drives that point home with very clear examples\u201d\u2014like the demonstration that apparently mislabeled training data can still make for successful training\u2014\u201cthat make this connection very visceral.\u201d<\/p>\n