Like a brain, a deep neural network has layers of neurons\u2014artificial ones that are figments of computer memory. When a neuron fires, it sends signals to connected neurons in the layer above. During deep learning, connections in the network are strengthened or weakened as needed to make the system better at sending signals from input data\u2014the pixels of a photo of a dog, for instance\u2014up through the layers to neurons associated with the right high-level concepts, such as \u201cdog.\u201d After a deep neural network has \u201clearned\u201d from thousands of sample dog photos, it can identify dogs in new photos as accurately as people can. The magic leap from special cases to general concepts during learning gives deep neural networks their power, just as it underlies human reasoning, creativity and the other faculties collectively termed \u201cintelligence.\u201d Experts wonder what it is about deep learning that enables generalization\u2014and to what extent brains apprehend reality in the same way.<\/p>\n Last month, a\u00a0YouTube video<\/a>\u00a0of a conference talk in Berlin, shared widely among artificial-intelligence researchers, offered a possible answer. In the talk,\u00a0Naftali Tishby<\/a>, a computer scientist and neuroscientist from the Hebrew University of Jerusalem, presented evidence in support of a new theory explaining how deep learning works. Tishby argues that deep neural networks learn according to a procedure called the \u201cinformation bottleneck,\u201d which he and two collaborators\u00a0first described in purely theoretical terms in 1999<\/a>. The idea is that a network rids noisy input data of extraneous details as if by squeezing the information through a bottleneck, retaining only the features most relevant to general concepts. Striking new\u00a0computer experiments<\/a>\u00a0by Tishby and his student Ravid Shwartz-Ziv reveal how this squeezing procedure happens during deep learning, at least in the cases they studied.<\/p>\n Tishby\u2019s findings have the AI community buzzing. \u201cI believe that the information bottleneck idea could be very important in future deep neural network research,\u201d said\u00a0Alex Alemi<\/a>\u00a0of Google Research, who has already\u00a0developed new approximation methods<\/a>\u00a0for applying an information bottleneck analysis to large deep neural networks. The bottleneck could serve \u201cnot only as a theoretical tool for understanding why our neural networks work as well as they do currently, but also as a tool for constructing new objectives and architectures of networks,\u201d Alemi said.<\/p>\n Some researchers remain skeptical that the theory fully accounts for the success of deep learning, but\u00a0Kyle Cranmer<\/a>, a particle physicist at New York University who uses machine learning to analyze particle collisions at the Large Hadron Collider, said that as a general principle of learning, it \u201csomehow smells right.\u201d<\/p>\n Geoffrey Hinton<\/a>, a pioneer of deep learning who works at Google and the University of Toronto, emailed Tishby after watching his Berlin talk. \u201cIt\u2019s extremely interesting,\u201d Hinton wrote. \u201cI have to listen to it another 10,000 times to really understand it, but it\u2019s very rare nowadays to hear a talk with a really original idea in it that may be the answer to a really major puzzle.\u201d<\/p>\n According to Tishby, who views the information bottleneck as a fundamental principle behind learning, whether you\u2019re an algorithm, a housefly, a conscious being, or a physics calculation of emergent behavior, that long-awaited answer \u201cis that the most important part of learning is actually forgetting.\u201d<\/p>\n The Bottleneck<\/strong><\/p>\n Tishby began contemplating the information bottleneck around the time that other researchers were first mulling over deep neural networks, though neither concept had been named yet. It was the 1980s, and Tishby was thinking about how good humans are at speech recognition\u2014a major challenge for AI at the time. Tishby realized that the crux of the issue was the question of relevance: What are the most relevant features of a spoken word, and how do we tease these out from the variables that accompany them, such as accents, mumbling and intonation? In general, when we face the sea of data that is reality, which signals do we keep?<\/p>\n \u201cThis notion of relevant information was mentioned many times in history but never formulated correctly,\u201d Tishby said in an interview last month. \u201cFor many years people thought information theory wasn\u2019t the right way to think about relevance, starting with misconceptions that go all the way to Shannon himself.\u201d<\/p>\n Claude Shannon, the founder of information theory, in a sense liberated the study of information starting in the 1940s by allowing it to be considered in the abstract\u2014as 1s and 0s with purely mathematical meaning. Shannon took the view that, as Tishby put it, \u201cinformation is not about semantics.\u201d But, Tishby argued, this isn\u2019t true. Using information theory, he realized, \u201cyou can define \u2018relevant\u2019 in a precise sense.\u201d<\/p>\n Imagine X is a complex data set, like the pixels of a dog photo, and Y is a simpler variable represented by those data, like the word \u201cdog.\u201d You can capture all the \u201crelevant\u201d information in X about Y by compressing X as much as you can without losing the ability to predict Y. In their 1999 paper, Tishby and co-authors\u00a0Fernando Pereira<\/a>, now at Google, and\u00a0William Bialek<\/a>, now at Princeton University, formulated this as a mathematical optimization problem. It was a fundamental idea with no killer application.<\/p>\n \u201cI\u2019ve been thinking along these lines in various contexts for 30 years,\u201d Tishby said. \u201cMy only luck was that deep neural networks became so important.\u201d<\/p>\n Eyeballs on Faces on People on Scenes<\/strong><\/p>\n Though the concept behind deep neural networks had been kicked around for decades, their performance in tasks like speech and image recognition only took off in the early 2010s, due to improved training regimens and more powerful computer processors. Tishby recognized their potential connection to the information bottleneck principle in 2014 after reading a\u00a0surprising paper<\/a>\u00a0by the physicists\u00a0David Schwab<\/a>\u00a0and\u00a0Pankaj Mehta<\/a>.<\/p>\n The\u00a0duo discovered<\/a>\u00a0that a deep-learning algorithm invented by Hinton called the \u201cdeep belief net\u201d works, in a particular case, exactly like renormalization, a technique used in physics to zoom out on a physical system by coarse-graining over its details and calculating its overall state. When Schwab and Mehta applied the deep belief net to a model of a magnet at its \u201ccritical point,\u201d where the system is fractal, or self-similar at every scale, they found that the network automatically used the renormalization-like procedure to discover the model\u2019s state. It was a stunning indication that, as the biophysicist\u00a0Ilya Nemenman<\/a>\u00a0said at the time, \u201cextracting relevant features in the context of statistical physics and extracting relevant features in the context of deep learning are not just similar words, they are one and the same.\u201d<\/p>\n The only problem is that, in general, the real world isn\u2019t fractal. \u201cThe natural world is not ears on ears on ears on ears; it\u2019s eyeballs on faces on people on scenes,\u201d Cranmer said. \u201cSo I wouldn\u2019t say [the renormalization procedure] is why deep learning on natural images is working so well.\u201d But Tishby, who at the time was undergoing chemotherapy for pancreatic cancer, realized that both deep learning and the coarse-graining procedure could be encompassed by a broader idea. \u201cThinking about science and about the role of my old ideas was an important part of my healing and recovery,\u201d he said.<\/p>\n In 2015, he and his student Noga Zaslavsky\u00a0hypothesized<\/a>\u00a0that deep learning is an information bottleneck procedure that compresses noisy data as much as possible while preserving information about what the data represent. Tishby and Shwartz-Ziv\u2019s new experiments with deep neural networks reveal how the bottleneck procedure actually plays out. In one case, the researchers used small networks that could be trained to label input data with a 1 or 0 (think \u201cdog\u201d or \u201cno dog\u201d) and gave their 282 neural connections random initial strengths. They then tracked what happened as the networks engaged in deep learning with 3,000 sample input data sets.<\/p>\n The basic algorithm used in the majority of deep-learning procedures to tweak neural connections in response to data is called \u201cstochastic gradient descent\u201d: Each time the training data are fed into the network, a cascade of firing activity sweeps upward through the layers of artificial neurons. When the signal reaches the top layer, the final firing pattern can be compared to the correct label for the image\u20141 or 0, \u201cdog\u201d or \u201cno dog.\u201d Any differences between this firing pattern and the correct pattern are \u201cback-propagated\u201d down the layers, meaning that, like a teacher correcting an exam, the algorithm strengthens or weakens each connection to make the network layer better at producing the correct output signal. Over the course of training, common patterns in the training data become reflected in the strengths of the connections, and the network becomes expert at correctly labeling the data, such as by recognizing a dog, a word, or a 1.<\/p>\n In their experiments, Tishby and Shwartz-Ziv tracked how much information each layer of a deep neural network retained about the input data and how much information each one retained about the output label. The scientists found that, layer by layer, the networks converged to the information bottleneck theoretical bound: a theoretical limit derived in Tishby, Pereira and Bialek\u2019s original paper that represents the absolute best the system can do at extracting relevant information. At the bound, the network has compressed the input as much as possible without sacrificing the ability to accurately predict its label.<\/p>\n Tishby and Shwartz-Ziv also made the intriguing discovery that deep learning proceeds in two phases: a short \u201cfitting\u201d phase, during which the network learns to label its training data, and a much longer \u201ccompression\u201d phase, during which it becomes good at generalization, as measured by its performance at labeling new test data.<\/p>\n As a deep neural network tweaks its connections by stochastic gradient descent, at first the number of bits it stores about the input data stays roughly constant or increases slightly, as connections adjust to encode patterns in the input and the network gets good at fitting labels to it. Some experts have compared this phase to memorization.<\/p>\n Then learning switches to the compression phase. The network starts to shed information about the input data, keeping track of only the strongest features\u2014those correlations that are most relevant to the output label. This happens because, in each iteration of stochastic gradient descent, more or less accidental correlations in the training data tell the network to do different things, dialing the strengths of its neural connections up and down in a\u00a0random walk<\/a>. This randomization is effectively the same as compressing the system\u2019s representation of the input data. As an example, some photos of dogs might have houses in the background, while others don\u2019t. As a network cycles through these training photos, it might \u201cforget\u201d the correlation between houses and dogs in some photos as other photos counteract it. It\u2019s this forgetting of specifics, Tishby and Shwartz-Ziv argue, that enables the system to form general concepts. Indeed, their experiments revealed that deep neural networks ramp up their generalization performance during the compression phase, becoming better at labeling test data. (A deep neural network trained to recognize dogs in photos might be tested on new photos that may or may not include dogs, for instance.)<\/p>\n It remains to be seen whether the information bottleneck governs all deep-learning regimes, or whether there are other routes to generalization besides compression. Some AI experts see Tishby\u2019s idea as one of many important theoretical insights about deep learning to have emerged recently.\u00a0Andrew Saxe<\/a>, an AI researcher and theoretical neuroscientist at Harvard University, noted that certain very large deep neural networks don\u2019t seem to need a drawn-out compression phase in order to generalize well. Instead, researchers program in something called early stopping, which cuts training short to prevent the network from encoding too many correlations in the first place.<\/p>\n Tishby argues that the network models analyzed by Saxe and his colleagues differ from standard deep neural network architectures, but that nonetheless, the information bottleneck theoretical bound defines these networks\u2019 generalization performance better than other methods. Questions about whether the bottleneck holds up for larger neural networks are partly addressed by Tishby and Shwartz-Ziv\u2019s most recent experiments, not included in their preliminary paper, in which they train much larger, 330,000-connection-deep neural networks to recognize handwritten digits in the 60,000-image\u00a0Modified National Institute of Standards and Technology database<\/a>, a well-known benchmark for gauging the performance of deep-learning algorithms. The scientists saw the same convergence of the networks to the information bottleneck theoretical bound; they also observed the two distinct phases of deep learning, separated by an even sharper transition than in the smaller networks. \u201cI\u2019m completely convinced now that this is a general phenomenon,\u201d Tishby said.<\/p>\n The mystery of how brains sift signals from our senses and elevate them to the level of our conscious awareness drove much of the early interest in deep neural networks among AI pioneers, who hoped to reverse-engineer the brain\u2019s learning rules. AI practitioners have since largely abandoned that path in the mad dash for technological progress, instead slapping on bells and whistles that boost performance with little regard for biological plausibility. Still, as their thinking machines achieve ever greater feats\u2014even stoking fears that\u00a0AI could someday pose an existential threat<\/a>\u2014many researchers hope these explorations will uncover general insights about learning and intelligence.<\/p>\n Brenden Lake<\/a>, an assistant professor of psychology and data science at New York University who studies similarities and differences in how humans and machines learn, said that Tishby\u2019s findings represent \u201can important step towards opening the black box of neural networks,\u201d but he stressed that the brain represents a much bigger, blacker black box. Our adult brains, which boast several hundred trillion connections between 86 billion neurons, in all likelihood employ a bag of tricks to enhance generalization, going beyond the basic image- and sound-recognition learning procedures that occur during infancy and that may in many ways resemble deep learning.<\/p>\n For instance, Lake said the fitting and compression phases that Tishby identified don\u2019t seem to have analogues in the way children learn handwritten characters, which he studies. Children don\u2019t need to see thousands of examples of a character and compress their mental representation over an extended period of time before they\u2019re able to recognize other instances of that letter and write it themselves. In fact, they can learn from a single example.\u00a0Lake and his colleagues\u2019 models<\/a>\u00a0suggest the brain may deconstruct the new letter into a series of strokes\u2014previously existing mental constructs\u2014allowing the conception of the letter to be tacked onto an edifice of prior knowledge. \u201cRather than thinking of an image of a letter as a pattern of pixels and learning the concept as mapping those features\u201d as in standard machine-learning algorithms, Lake explained, \u201cinstead I aim to build a simple causal model of the letter,\u201d a shorter path to generalization.<\/p>\n Meanwhile, both real and artificial neural networks stumble on problems in which every detail matters and minute differences can throw off the whole result. Most people can\u2019t quickly multiply two large numbers in their heads, for instance. \u201cWe have a long class of problems like this, logical problems that are very sensitive to changes in one variable,\u201d Tishby said. \u201cClassifiability, discrete problems, cryptographic problems. I don\u2019t think deep learning will ever help me break cryptographic codes.\u201d<\/p>\n Generalizing\u2014traversing the information bottleneck, perhaps\u2014means leaving some details behind. This isn\u2019t so good for doing algebra on the fly, but that\u2019s not a brain\u2019s main business. We\u2019re looking for familiar faces in the crowd, order in chaos, salient signals in a noisy world.<\/p>\n![\"\"](\"https:\/\/media.wired.com\/photos\/59dbddde0cd98134b30dc460\/master\/w_1000,c_limit\/graph-IL.jpg\")
<\/div>\n<\/div>Humans and Machines<\/h3>\n