The difference and connection between machine learning and deep learning in artificial intelligence knowledge sharing

The wave of artificial intelligence is sweeping the globe, and many words are lingering in our ears: Artificial Intelligence, Machine Learning, and Deep Learning. Many people always seem to understand and understand the meaning of these high-frequency vocabulary and the relationship behind them.

To help you better understand artificial intelligence, this article explains the meaning of these words in the simplest language, clarifies the relationship between them, and hopes to help the peers who are just getting started.

Artificial intelligence: from concept to prosperity

In 1956, several computer scientists gathered at the Dartmouth conference to propose the concept of "artificial intelligence", dreaming of using a computer that had just emerged to construct a complex machine with the same essential characteristics as human intelligence. Since then, artificial intelligence has been lingering in people's minds and slowly hatching in research laboratories. In the decades that followed, artificial intelligence has been reversed at the poles, or as a prophecy of the dazzling future of human civilization, or thrown into the garbage by the madness of being a technological madman. Until 2012, these two sounds still exist at the same time.

After 2012, artificial intelligence began to break out due to the increase in data volume, the increase in computing power, and the emergence of new machine learning algorithms (deep learning). According to the "Global AI Talent Report" released by LinkedIn recently, as of the first quarter of 2017, the number of technical personnel in the global AI (Artificial Intelligence) field based on the LinkedIn platform exceeded 1.9 million, and only the domestic artificial intelligence talent gap reached more than 5 million. .

The field of artificial intelligence research is also expanding. Figure 1 shows the various branches of artificial intelligence research, including expert systems, machine learning, evolutionary computation, fuzzy logic, computer vision, natural language processing, recommendation systems, and so on.

Figure 1 Artificial Intelligence Research Branch

However, the current research work is focused on the weak artificial intelligence, and it is hopeful that major breakthroughs will be made in the near future. Most of the artificial intelligence in the film is depicting strong artificial intelligence, and this part is difficult to realize in the current real world. (The artificial intelligence is usually divided into weak artificial intelligence and strong artificial intelligence. The former allows the machine to have the ability to observe and perceive, and can achieve a certain degree of understanding and reasoning, while strong artificial intelligence allows the machine to acquire adaptive capabilities, and solve some problems before. Have encountered problems).

Weak artificial intelligence is hopeful to make a breakthrough, how is it achieved, and where does "intelligence" come from? This is mainly due to a method of implementing artificial intelligence - machine learning.

Machine learning: a way to implement artificial intelligence

The most basic approach to machine learning is to use algorithms to parse data, learn from it, and then make decisions and predictions about events in the real world. Unlike traditional software programs that solve specific tasks and hard code, machine learning uses a large amount of data to "train" and learn how to accomplish tasks from data through various algorithms.

As a simple example, when we browse the online store, information about product recommendations often appears. This is the mall based on your previous shopping records and lengthy collection list, identify which of these are products that you are really interested in and willing to buy. Such a decision model can help the mall to provide advice to customers and encourage product consumption.

Machine learning comes directly from the early days of artificial intelligence. Traditional algorithms include decision trees, clustering, Bayesian classification, support vector machines, EM, Adaboost, and so on. From the perspective of learning methods, machine learning algorithms can be divided into supervised learning (such as classification problems), unsupervised learning (such as clustering problems), semi-supervised learning, integrated learning, deep learning and reinforcement learning.

The traditional machine learning algorithms in the fields of fingerprint recognition, Haar-based face detection, HoG-based object detection and other fields have basically met the requirements of commercialization or the commercialization of specific scenes, but each step is extremely difficult until The emergence of deep learning algorithms.

Deep learning: a technology for machine learning

Deep learning is not an independent learning method. It itself uses supervised and unsupervised learning methods to train deep neural networks. However, due to the rapid development of this field in recent years, some unique learning methods have been proposed (such as the residual network), so more and more people regard it as a learning method.

The initial deep learning is a learning process that uses deep neural networks to solve feature representation. The deep neural network itself is not a completely new concept and can be roughly understood as a neural network structure containing multiple hidden layers. In order to improve the training effect of deep neural networks, people have made corresponding adjustments to the connection methods and activation functions of neurons. In fact, many ideas have been there in the early years, but due to insufficient training data and backward computing power, the final result is not satisfactory.

Deep learning has ruthlessly fulfilled various tasks, making it seem that all machine auxiliary functions are possible. Driverless cars, preventive health care, and even better movie recommendations are all in sight or nearing.

The difference and connection between the three

Machine learning is a way to implement artificial intelligence. Deep learning is a technology that implements machine learning. We use the simplest method - concentric circles to visually reveal the relationship between the three.

Figure 2 is a schematic diagram of the relationship between the three

At present, the industry has a more common sense of error, that is, " deep learning may eventually eliminate all other machine learning algorithms ." This kind of consciousness is mainly caused by the fact that the application of deep learning in the field of computer vision and natural language processing far exceeds the traditional machine learning method, and the media has made extensive and exaggerated reports on deep learning.

Deep learning, as the hottest machine learning method at present, does not mean the end of machine learning. At least the following problems exist:

1. The deep learning model requires a large amount of training data to show the magical effect, but in real life, it often encounters small sample problems. At this time, the deep learning method cannot be started, and the traditional machine learning method can handle it;

2. Some areas, using traditional simple machine learning methods, can be well solved, and there is no need to use complex deep learning methods;

3. The idea of ​​deep learning comes from the inspiration of the human brain, but it is by no means a simulation of the human brain. For example, after watching a bicycle for a child of three or four years old, seeing even a bicycle with a completely different appearance, the child also In all likelihood, it is a judgment of a bicycle. That is to say, the human learning process often does not require large-scale training data, and the current deep learning method is obviously not a simulation of the human brain.

Deep learning 佬Yoshua Bengio has a very good answer to a similar question on Quora. Here is a quote to answer the above question:

Science is NOT a battle, it is a collaboration. We all build on each other's ideas. Science is an act of love, not war. Love for the beauty in the world that surrounds us and love to share and build something together. Science a highly satisfying activity, emotionally speaking!

The general meaning of this passage is that science is not war but cooperation. The development of any discipline has never been a path to black, but peers learn from each other, learn from each other, learn from each other, complement each other, and stand on the shoulders of giants. Forward. The same is true for machine learning research. It is a cult to live and die, and openness and tolerance are the right way.

Combining the development of machine learning since 2000, and then taking a look at Bengio's words, I am deeply touched. In the 21st century, looking at the development of machine learning, research hotspots can be simply summarized as manifold learning from 2000 to 2006, sparse learning from 2006 to 2011, and deep learning from 2012 to the present. Which machine learning algorithm will become a hot spot in the future? Wu Enda, one of the three giants of deep learning, once said, "After deep learning, migration learning will lead the next wave of machine learning technology." But in the end, what is the next hot spot of machine learning, and who can say it.

Therefore, the current introduction to artificial intelligence requires not only mastering traditional machine learning algorithms, but also familiarizing with deep learning models. To this end, the young teachers of the Institute of Automation of the Chinese Academy of Sciences and the Deep Blue Institute jointly launched two online live courses , "Machine Learning: From Theory to Practice" and "Deep Learning: From Theory to Practice ." The curriculum system is fully integrated with theory and practice. PPT and code are provided to the students, and the WeChat group is established for the students to communicate with the lecturers.

Machine learning course content

1. Mathematical basis (PPT data)

1.1 Matrix theory, probability theory, optimization basics

2. Machine learning algorithm: from theory to practice (20 hours)

2.1 Machine Learning Overview

2.1.1 Classification of machine learning methods

2.1.2 Introduction to Basic Knowledge

2.2 KNN algorithm

2.2.1 Overview and theoretical explanation

2.2.2 K-nearest application case and code implementation

2.1 Machine Learning Overview

2.1.1 Classification of machine learning methods

2.1.2 Introduction to Basic Knowledge

2.2 KNN algorithm

2.2.1 Overview and theoretical explanation

2.2.2 K-nearest application case and code implementation

2.3 Bayesian classification

2.3.1 Naive Bayes

2.3.2 Bayesian Decision Theory

2.3.3 NB classification algorithm application case and code implementation

2.4 Regression and classification

2.4.1 Curve fitting

2.4.2 Linear regression

2.4.3 logistic regression

2.4.4 Related application cases and code implementation

2.5 support vector machine

2.5.1 Linear Support Vector Machine

2.5.2 Nonlinear Support Vector Machine

2.5.3 Nuclear method

2.5.4 SVM application case and code implementation

2.6 Clustering algorithm

2.6.1 K-means clustering

2.6.2 Hierarchical Clustering

2.6.3 Clustering algorithm application case and code implementation

2.7 Data dimensionality reduction

2.7.1 Linear dimensionality reduction

2.7.2 Nonlinear dimensionality reduction

2.7.3 Dimensionality reduction application case and code implementation

2.8 EM algorithm

2.8.1 EM algorithm basis

2.8.2 Multi-Gaussian parameter estimation

2.8.3 EM application case and code implementation

2.9 Adaboost algorithm

2.9.1 Algorithm-independent machine learning

2.9.2 Adaboost algorithm

2.9.3 Application case and code implementation

2.10 Hidden Markov Model

2.10.1 Markov

2.10.2 Hidden Markov Model

2.10.3 Application case and code implementation

Deep learning course content

1. Mathematical basis (PPT data)

1.1 Bayesian decision theory, parameters and nonparametric estimation

1.2 Regression and classification, gradient descent optimization

1.3 Information entropy

2. Deep learning theory (6 hours)

2.1 Feedforward Neural Networks (Overview, Single Layer Neural Networks, Multilayer Neural Networks)

2.2 Convolutional neural networks (basic concepts, development history, network characteristics, network settings, network training, and related applications)

2.3 feedback neural network (Hopfield network, Boltzmann machine, restricted Boltzmann machine)

3. Deep network common model and Keras combat (8 hours)

3.1 Keras and residual network (from LSTM to Highway network, from Highway network to residual network, Highway network based applications)

3.2 Automatic coding machine AE and generation of anti-network GAN (origin and variant of AE, generation of anti-network GAN, application based on GAN network)

3.3 Keres-based behavior recognition (introduction to behavioral identification problems, introduction to common models of behavior recognition based on deep learning, Keres-based behavior recognition practice)

3.4 Scene segmentation based on Keras (introduction to scene segmentation problem, introduction of common model for scene segmentation based on deep learning, scene segmentation based on Keras)

4. Deep learning framework (4 hours)

4.1 Introduction to Caffe (introduction, installation and configuration, advantages and limitations analysis, in-depth Caffe source code, Caffe debugging)

4.2 Caffe enhancement (based on Caffe's MINST handwriting recognition, Caffe's Python interface, Caffe modification and adding Layer, network training skills)

Lecturer team

Mr. Wang , a researcher and associate researcher at the Institute of Automation of the Chinese Academy of Sciences, has published more than 20 papers in the top conference journals ICCV, TNNLS, TIP, etc.; participated in the National Video Image Analysis Technology Challenge, and won the second place in the target detection and identification. Apply deep learning Keras framework and Caffe framework.

Ms. Gong , an algorithmic engineer at a well-known foreign enterprise research institute and a Ph.D. student at the Institute of Automation, Chinese Academy of Sciences, has nearly six years of research experience in the field of computer vision and artificial intelligence. During his Ph.D. in research, he focused on pattern recognition and image processing. He published papers in top international journals in the field of pattern recognition, and participated in the image segmentation competition organized by a well-known Internet company, winning the fourth place. Currently responsible for algorithm development in computer vision and artificial intelligence.

Mr. Shao , currently an associate professor and master tutor, graduated from the Institute of Automation of the Chinese Academy of Sciences, and has two years of teaching experience. His main research interests include machine learning, pattern recognition and image processing. As the project leader, he is responsible for the National Natural Science Foundation and publishes several papers in international mainstream journals and conferences.

Mr. Huo , a young scholar at the front line, graduated from Xi'an University of Electronic Science and Technology with a research position in image processing, presided over and participated in many national natural science funds, and published many papers in the journals and conferences of related fields by the first author.

Start time and form

1. The price of the in- depth study course is 499 yuan, only 90 places are reserved. From October 29th to November 26th, every Saturday and Sunday, 7:00-9:00, live broadcast is taught;

2. The price of the machine learning course is 499 yuan, and it will be broadcast live online from 7:00 to 9:00 on Thursdays, Saturdays and Sundays from November 30th to December 21st.

3. The course can be viewed in real time within one year of the course ;

4. The course PPT and source program will be disclosed to the students in advance;

5. WeChat group can answer questions before class, during class and after class.