Deep learning
Deep learning technology drives much of contemporary society, from web searching to social network content filtering to online shopping recommendations, and is increasingly integrated into consumer goods like cameras and smartphones. Machine-learning systems are employed to recognize objects within images, automatically transcribe spoken words into text, connect news stories, postings, or products to users’ interests, and choose good results for the search. More and more of these applications employ a class of methods known as deep learning.
Traditional deep learning methods were hampered in how well they could process natural data in its raw form. It took decades of development to create a pattern-recognition or machine-learning system with careful engineering and a lot of domain knowledge to come up with a feature extractor that would take the raw data like the pixel values of an image into an appropriate internal representation or feature vector from which the learning subsystem, usually a classifier, would be able to detect or classify patterns in the input. Representation learning is a collection of techniques that enable a machine to be presented with raw data and learn the representations required for detection or classification automatically.
Deep learning techniques are multiple-representation learning approaches with multiple representation levels, gained by combining elementary but non-linear modules that convert the representation level by level from the raw input to a more abstract level by one step only. By chaining together sufficient numbers of such conversion operations, extremely complex functions may be learned. For discrimination tasks, subsequent deep learning layers of representation make more robust those aspects of the input that are discriminative and suppress non-essential variations.
For instance, tensor flow, porch, machine learning, deep learning, learning stage an image arrives as a matrix of pixel intensities, and the features learned in the first representation layer usually indicate the presence or absence of edges at certain orientations and positions within the image. The second level generally identifies motifs by identifying specific patterns of edges, independent of minor differences in the edge positions. The third level can build motifs into bigger combinations that correspond to components of common objects, and later deep learning layers would identify objects as combinations of components.
Supervised learning
The most ubiquitous type of machine learning, whether deep or not, is supervised learning. Suppose that we wish to construct a system that can allow images to contain, for example, a house, a car, a person, or a pet. We first combine a large set of images of houses, cars, people, and pets, with each image tagged with its class. When the machine is trained, it is presented with an image and gives an output in the form of a vector of scores, one for each class. We wish the target category to score the highest among all categories, but this will not occur before training.
We calculate an objective function that quantifies the error or distance between the output scores and the target pattern of scores. The machine then adjusts its internal tunable parameters to minimize this error. These tunable parameters, sometimes referred to as weights, are floating-point numbers that can be thought of as dials that set the input-output function of the deep learning machine.
In a typical deep-learning system, there are hundreds of millions of these tunable weights and hundreds of millions of labeled examples on which to train the machine. To correctly scale the weight vector, the learning algorithm calculates a gradient vector whose components, for each weight, tell how much the error would rise or fall if the weight were raised by a very small amount. The weight vector is then changed in the negative direction of the gradient vector.
Convolutional neural networks
Deep neural networks take advantage of the fact that many natural signals are compositional hierarchies, where higher-level features are derived by composing lower-level ones. In images, local edge combinations create motifs, motifs combine to form parts, and parts combine to create objects. There are similar hierarchies in speech and text from sounds to phones, phonemes, syllables, words, and sentences. The pooling enables representations to change very minimally when elements are in the layer before the change in position and appearance. While the function of the convolutional layer is to identify local conjunctions of features from the previous layer, the function of the pooling layer is to combine semantically similar features into a single one.
Since the relative locations of the features constituting a motif can differ slightly, supervised learning, learning, deep learning, learning stage, ai detecting the motif consistently can be achieved by coarse-graining the location of each feature. A common pooling unit calculates the maximum of a local patch of units within one feature map or several feature maps.
Neighbor pooling units receive input from patches shifted by more than one column or row, thus decreasing the dimension of the representation and producing an invariance to little shifts and distortions. Two or three layers of convolution, non-linearity, and pooling are stacked, followed by additional convolutional and fully connected layers. Backpropagating gradients through a Convent is as straightforward as through an ordinary deep network so that all the weights in all the filter banks can be trained.
Distributed representations and language processing
Deep learning theory demonstrates that deep nets possess two distinct exponential benefits over traditional learning algorithms without distributed representations. Both of these benefits are due to the composition power and rely on the underlying data-generating distribution having the right componential structure. First, the ability to learn distributed representations makes generalization possible to novel combinations of the values of learned features not encountered during training, say 2n combinations are available with n binary features.
Second, constructing layers of representation in a deep net introduces the possibility of another exponential benefit. The hidden layers of a multilayer neural network learn to represent the network’s inputs in a manner that facilitates easy prediction of the target outputs. This is well illustrated by training a multilayer neural network to predict the next word in a sequence from a local context of previous words.
Conclusion
Unsupervised deep learning served as a stimulus in reactivating interest in deep learning but has since fallen into the shade of the advances of purely supervised learning. Despite not highlighting it in this Review, we anticipate unsupervised learning to gain much greater prominence in the long term. Human and animal learning is substantially unsupervised, we find out about the layout of the world by watching it, not by being instructed in the label of each thing.
Human sight is an active process that selectively samples the optic array in an intelligent, task-dependent manner utilizing a small high-resolution fovea with a large low-resolution surround. We anticipate most of the future advances in vision to arise from end-to-end trained systems that integrate Convents with RNNs that employ reinforcement learning to determine where to look.
Systems that integrate deep learning and reinforcement learning are still in their early stages, but they already beat passive vision systems at classification tasks and achieved remarkable results in learning to play a wide variety of video games. Natural language processing is another field where deep learning is likely to have a huge impact in the next couple of years. We anticipate that systems based on RNNs to comprehend sentences or entire documents will improve significantly as they learn techniques system for selectively focusing on one at a time.