Deep learning
Deep learning is a subset of machine learning characterized by the use of multilayered artificial neural networks that enable systems to learn from experience rather than relying solely on pre-programmed rules. This technology, which draws inspiration from brain science, has become fundamental to advancements in artificial intelligence (AI), leading to significant breakthroughs in automation and generative AI applications. The process involves training a neural network with large datasets, where the machine identifies objects by processing the information through various layers, gradually moving from simple to complex interpretations to arrive at an answer.
The history of deep learning dates back to the 1950s with the introduction of neural networks, but it faced challenges and periods of stagnation until the mid-1980s when researchers revived interest by demonstrating that multilayered networks could learn from past mistakes. A pivotal moment occurred in 2006 with the development of deep belief networks, which allowed for more effective classification of raw data through probabilistic comparisons. By the early 2010s, deep learning gained enormous traction, leading to notable successes in image and speech recognition, and played a key role in innovations such as autonomous vehicles and advanced drug development.
As deep learning became integral to various computer systems, it also sparked dialogue about the ethical implications of AI, including issues related to errors and algorithmic bias. The field continues to evolve rapidly, with significant attention given to applications like ChatGPT, showcasing the potential of deep neural networks in natural language processing and beyond.
On this Page
Subject Terms
Deep learning
Deep learning is a type of machine learning in which multilayered (or "deep") artificial neural networks allow a computer system to "learn" from experience, rather than rely wholly on pre-programmed knowledge. Originally inspired by brain science, it is considered a crucial concept in artificial intelligence (AI), underpinning numerous breakthroughs in automation and generative AI. Deep learning involves feeding a neural network with large amounts of data to train the machine in classification. The machine is given an object to identify and processes it through several network layers. As the process continues, the machine goes from simple layers to ones that are more complicated until an answer is reached. Complex algorithms instruct the neurons how to respond to improve the results.
Brief History
The concept of neural networks was first introduced in the 1950s as biologists were mapping out the workings of the human brain. Computer scientists were looking beyond logical applications to replicate thinking in machines. In 1958, research psychologist Frank Rosenblatt applied these theories to design the perceptron, a single-layered network of simulated neurons using a room-sized computer. Through their connections, the neurons would relay a value, or "weight," of either 1 or 0 to correspond with a shape. However, after several tries, the machine would not recognize the right shape. Rosenblatt applied supervised learning, training the perceptron to output the correct answer with the machine developing an algorithm to tweak the weights to get the correct answer.
Rosenblatt's algorithm, however, did not apply to multilayered networks, limiting the perceptron's ability to perform more complex tasks. In a 1969 book, the scientists Marvin Minsky and Seymour Papert asserted that making more layers would not make perceptrons more useful. Research on artificial neural networks was therefore largely abandoned for nearly two decades.
In the mid-1980s, researchers Geoffrey Hinton and Yann LeCun revived interest in neural networks, with the belief that a brain-like structure was needed to fulfill the potential of AI. Instead of only outputting an answer, their goal was to create a multilayered network that would allow the machine to learn from past mistakes. The duo and other researchers used a learning algorithm called backpropagation that would allow data to pass through multiple layers and the network to make adjustments to give the right answer. This spawned technology in the 1990s that could read handwritten text. However, like perceptrons, backpropagation had its limitations and required much data to be fed into a machine. Meanwhile, other researchers had success developing alternative learning algorithms that did not require neurons. Work on deep learning stalled again.
The development of machine learning and other AI technologies in general was reinvigorated in the early twenty-first century as computing power advanced rapidly. A turning point for deep learning in particular came in 2006, when Hinton and others developed groundbreaking generative models known as deep belief networks. These involve a progressive set of variable or neuron layers, each of which works to detect or recognize a certain feature of input data. This multilayered neural network model allows a system to process and classify raw data based on probabilistic comparison to previous examples. For example, a system that has been "trained" with a large set of images can use the information from those examples to analyze a newly input image and identify it with a high degree of accuracy. In 2012, Hinton and two students won a contest with software that correctly identified one thousand images, demonstrating the effectiveness of deep neural networks.
These advances brought much attention to deep learning throughout the 2010s, and the field continued to progress rapidly. Major technology companies such as Google, Facebook, and Microsoft invested heavily in big data and machine learning to improve speech- and image-recognition products and services, including voice-activated searches, translation tools, and photo searches. Deep learning also spurred advances in areas such as autonomous vehicles and drug development, among others. By the late 2010s, deep learning was integral to many computer systems, from cutting-edge experimental research models to popular consumer applications. In 2018, Hinton, LeCun, and Yoshua Bengio received the prestigious Turing Award for their foundational work on deep neural networks, reflecting the consensus that deep learning had launched a revolution in AI.
Deep learning remained at the forefront of machine learning research into the 2020s. Notably, the 2022 release of the groundbreaking chatbot ChatGPT brought even more public attention to the rapid boom in AI technology. ChatGPT and other generative AI systems based on large language models (LLMs) showcased how deep neural networks helped to enable unprecedentedly complex and powerful functions, such as natural language processing. Proponents continued to hail the benefits of such advances in virtually every area of science, business, and beyond. However, many observers also increasingly raised concerns about AI, including errors and algorithmic bias in deep learning models that could have severe ethical and security implications.
Bibliography
Bengio, Yoshua. "Machines Who Learn." Scientific American, vol. 314, no. 6, 2016, pp. 46–51.
Clark, Don. "Computer Chips Evolve to Keep Up with Deep Learning." Wall Street Journal, 11 Jan. 2017, www.wsj.com/articles/computer-chips-evolve-to-keep-up-with-deep-learning-1484161286. Accessed 22 Dec. 2022.
"Fathers of the Deep Learning Revolution Receive ACM A. M. Turing Award." Association for Computing Machinery, 2018, awards.acm.org/about/2018-turing. Accessed 19 Sept. 2024.
Gillis, Alexander S. "Deep Learning." TechTarget, www.techtarget.com/searchenterpriseai/definition/deep-learning-deep-neural-network. Accessed 19 Sept. 2024.
Goodfellow, Ian, et al. Deep Learning. MIT P, 2016.
Hof, Robert D. "Deep Learning." MIT Technology Review, www.technologyreview.com/s/513696/deep-learning. Accessed 16 Jan. 2017.
Holdsworth, Jim, and Mark Scapicchio. "What Is Deep Learning?" IBM, 17 June 2024, www.ibm.com/topics/deep-learning. Accessed 19 Sept. 2024.
Knight, Will. "AI's Unspoken Problem." MIT Technology Review, vol. 119, no .5, 2016, pp. 28–37.
Knight, Will. "Kindergarten for Computers." MIT Technology Review, vol. 119, no. 1, 2016, pp. 52–58.
Metz, Cade. "2016: The Year That Deep Learning Took Over the Internet." Wired, 25 Dec. 2016, www.wired.com/2016/12/2016-year-deep-learning-took-internet. Accessed 16 Jan. 2017.
Metz, Cade. "Finally, Neural Networks That Actually Work." Wired, 21 Apr. 2015, www.wired.com/2015/04/jeff-dean. Accessed 16 Jan. 2017.
Parloff, Roger. "Why Deep Learning Is Suddenly Changing Your Life." Fortune, 28 Sept. 2016, fortune.com/ai-artificial-intelligence-deep-machine-learning. Accessed 16 Jan. 2017.
Simonite, Tom. "Teaching Machines to Understand Us." MIT Technology Review, 6 Aug. 2015, www.technologyreview.com/s/540001/teaching-machines-to-understand-us/. Accessed 16 Jan. 2017.