Introduction:
Deep learning, machine learning, and pattern recognition are highly relevant topics commonly used in the field of robotics with artificial intelligence. Despite the overlapping similarities, these concepts are not identical. In this article, we will be discussing some of the differences of the three concepts and their applications.
Figure 1: An algorithm to detect the character "3" using sub-blocks
Three Popular Terms Correlated with "Learning"
Pattern recognition is the oldest form of learning and has become a relatively obsolete term. On the other hand, deep learning is a new and popular topic in the field of artificial intelligence. Machine learning, unlike the other two terms, is a fundamental form of learning and is one of the hottest areas in many start-ups and research labs. The Google Trends image below that shows the recent increase in interest for deep learning. Additionally, from the image, we can conclude that:
- Starting from 2010, machine learning is steadily becoming popular again.
- Pattern recognition used to be the hottest topic at the very beginning of the graph but is steadily declining.
- Deep learning is a new and fast-rising area, beating the popularity of pattern recognition in 2015.
Figure 2: The Google search index of the three concepts since 2004 (Picture source: Google Trends)
Pattern Recognition: The Beginning of Intelligent Programs
The term pattern recognition became popular between the 1970s and the 1980s. It focuses on how to make computer programs perform intelligent and human-like tasks, such as an object from an image. Initially, people were not very interested in knowing how machines can achieve this intelligence, as long as it works correctly. However, technologies such as filters, boundary detection, and morphological processing have shown to be effective when applied to an image detection algorithm. Researchers in the pattern recognition community showed an increasing interest in this topic, spawning the field of optical character recognition.
It is appropriate to say that pattern recognition was the most innovative and "intelligent" signal processing of the 1970s, the 1980s, and even the early 1990s. Concepts such as decision tree, heuristic method, and quadratic discriminatory analysis were all introduced during this period. Pattern recognition slowly shifted from being a topic in electrical engineering to a topic of interest in computer science. One of the most famous books in pattern recognition, Pattern Classification by Duda and Hart, was released in 1973. Despite being published more than four decades ago, it is still a good introductory textbook for beginners seeking to know more about the pattern recognition field.
Machine Learning: Intelligent Programs that Learn from Samples
In the early 1990s, many realized that there was a more effective way to create pattern recognition algorithms, particularly replacing researchers with probability and statistics. This paradigm led to the creation of machine learning. The goal of machine learning is to give a computer a collection of data and let the computer make its own conclusion with minimal human intervention. Specifically, it implies that the computer (or the machine) collects statistics from data, and then generating a probabilistic model to determine the most possible outcome. When designed correctly, a machine learning algorithm always performs better than a person would because it is immune to cognitive bias and fatigue.
Figure 3: Typical machine learning process (Picture source: Dr. Natalia Konstantinova's blog).
In the middle of the 21st century, machine learning has emerged as an important research topic in computer science. Scientists have begun applying the concept broadly, creating new businesses using this technology. Machine learning has been used in robotics, genetic analysis, and in predictions for the financial market. Moreover, machine learning's combination with the graph theory created a new topic of research – the graph model.
Machine learning has become a basic skill for many people, but it has also caused a lot of confusion especially to people new to this field. We have seen a wide variety of methods and schools of thoughts in the machine-learning field, all having its own benefits.
Deep Learning: A Framework to Unite the World
Deep learning is currently a hot topic of research, specifically Convolutional Neural Network (or ConvNet), which has been used in large-scale graphic recognition.
Figure 4: ConvNet framework (Picture source: Torch's textbook)
In deep learning, there is minimal human intervention and bias because the parameters in the modes are learned from statistics. However, deep learning is only possible with an ample amount of statistics (big data) and strong arithmetic capabilities (graphic processor or GPU) to optimize the mode. Because convolution computation has been widely applied in computer vision, it is the natural choice for the mode of deep learning.
To understand deep learning, you should have a basic knowledge of linear algebra and programming. If you are not familiar with these topics, we strongly recommend Andrej Karpathy's blog "Hacker's Guide to Neural Networks."
Despite the benefits of deep learning, there are still many unsolved questions in its application. There are no existing theories concerning the validity of deep learning, nor textbooks on specific guidelines of deep learning. There have also been valid concerns for the possibility of artificial intelligence taking over jobs through deep learning. However, successful implementation of deep learning and artificial intelligence still requires plenty of human intervention. A high-quality product requires great vision, expertise of the field, market development, and most of all, the creativity of human beings.
Additional Relevant Technical Terms
- Big Data: is an important concept that covers many aspects, such as the storage of massive data, and the mining of hidden information in data. For an enterprise operation, big data can offer valuable insights in decision-making. It was only several years ago that machine learning was integrated with big data.
- Artificial Intelligence: is the oldest as well as the most encompassing technical term. Artificial intelligence is sometimes used to describe all topics related to learning, and its popularity has fluctuated in the past 50 years. In simple terms, artificial intelligence is the potential of a computer program or a device to think, learn, and interact with a human user. It is widely applied in fields such as healthcare, robotics, and finance.
Conclusion
The three popular terms relevant to artificial intelligence – deep learning, machine learning and pattern recognition – are highly correlated but are also unique are used in different applications. Pattern recognition was the first concept to be introduced in image processing, and eventually evolved to machine learning. To maximize the scope of application for machine learning, researchers actively searched for a method to automate machine learning, creating the field of deep learning. Deep learning is a relatively nascent field; there remains a lot of research to explore the full potential of deep learning.