The Fundamentals of

Machine Learning
Hichem Felouat
hichemfel@gmail.com
https://www.linkedin.com/in/hichemfelouat/
Hichem Felouat - hichemfel@gmail.com - Algeria 1
What Is Artificial Intelligence?
Artificial intelligence (AI) is an area of computer
science that emphasizes the creation of intelligent
machines that work and react like humans.

• AI is an interdisciplinary science with multiple

approaches.

• AI has become an essential part of the technology

industry.
Hichem Felouat - hichemfel@gmail.com - Algeria 2
Subdomains of Artificial Intelligence

Hichem Felouat - hichemfel@gmail.com - Algeria 3

Artificial Intelligence Applications

Hichem Felouat - hichemfel@gmail.com - Algeria 4

What Is Machine Learning?
• Machine Learning is the science (and
art) of programming computers so
they can learn from data.

• Machine Learning is the field of

study that gives computers the ability
to learn without being explicitly
programmed. - Arthur Samuel, 1959.

Hichem Felouat - hichemfel@gmail.com - Algeria 5

What Does Learning Mean?
A computer program is said to learn
from experience E with respect to some
task T and some performance measure P,
if its performance on T, as measured by
P, improves with experience E. —Tom
Mitchell, 1997
Hichem Felouat - hichemfel@gmail.com - Algeria 6
Timeline of Machine Learning

Hichem Felouat - hichemfel@gmail.com - Algeria 7

Why Use Machine Learning?

The traditional approach. If the problem is not trivial, your program will
likely become a long list of complex rules pretty hard to maintain.
Hichem Felouat - hichemfel@gmail.com - Algeria 8
Why Use Machine Learning?

Hichem Felouat - hichemfel@gmail.com - Algeria 9

Why Use Machine Learning?

Machine Learning approach. The program is much shorter, easier to

maintain, and most likely more accurate.
Hichem Felouat - hichemfel@gmail.com - Algeria 10
Why Use Machine Learning?

Machine Learning can help humans learn.

Hichem Felouat - hichemfel@gmail.com - Algeria 11

Why Use Machine Learning?

AI Index 2019 Annual Report.

Hichem Felouat - hichemfel@gmail.com - Algeria 12

Applications of Machine Learning

Hichem Felouat - hichemfel@gmail.com - Algeria 13

Applications of Machine Learning

To summarize, Machine Learning is great for:

• Problems for which existing solutions require a lot of hand-

tuning or long lists of rules: one Machine Learning algorithm
can often simplify code and perform better.

• Complex problems for which there is no good solution at all

using a traditional approach: the best Machine Learning
techniques can find a solution.

Hichem Felouat - hichemfel@gmail.com - Algeria 14

How to get started with ML
1) Mathematics: statistics, probability, and

linear algebra.(NumPy, SciPy)

2) Programming: data structures, OOP, and

parallel programming. (Python)

3) Databases: SQL and NOSQL.

4) ML algorithms: regression, classification,

and clustering.

5) ML Tools: Scikt learn, TensorFlow and

Keras.

Hichem Felouat - hichemfel@gmail.com - Algeria 15

How to get started with ML

Hichem Felouat - hichemfel@gmail.com - Algeria 16

Machine Learning Vocabulary 1
1) Examples: Items or instances of data used for learning or evaluation. In our
spam problem, these examples correspond to the collection of email
messages we will use for learning and testing.

2) Training sample: Examples used to train a learning algorithm. In

our spam problem, the training sample consists of a set of email
examples along with their associated labels.
3) Labels: Values or categories assigned to examples. In classification
problems, examples are assigned specific categories, for instance, the spam
and not-spam categories in our binary classification problem. In regression,
items are assigned real-valued labels.

Hichem Felouat - hichemfel@gmail.com - Algeria 17

Machine Learning Vocabulary 2
4) Features: The set of attributes, often represented as a vector, associated to
an example. In the case of email messages, some relevant features may
include the length of the message, the name of the sender, various
characteristics of the header, the presence of certain keywords in the body of
the message, and so on.
5) Test sample: Examples used to evaluate the performance of a learning algorithm. The test
sample is separate from the training and validation data and is not made available in the
learning stage. In the spam problem, the test sample consists of a collection of email
examples for which the learning algorithm must predict labels based on features. These
predictions are then compared with the labels of the test sample to measure the performance
of the algorithm.

6) Loss function: A function that measures the difference, or loss,

between a predicted label and a true label.
Hichem Felouat - hichemfel@gmail.com - Algeria 18
 Machine Learning Vocabulary
Features Label
Examples (Training sample + Test sample) = Dataset

One Example Hichem Felouat - hichemfel@gmail.com - Algeria 19

Types of Machine Learning Systems
There are so many different types of Machine Learning systems that it is
useful to classify them in broad categories based on:

• Whether or not they are trained with human supervision (supervised,

unsupervised, semisupervised, and Reinforcement Learning).

• Whether or not they can learn incrementally on the fly (online versus
batch learning).

• Whether they work by simply comparing new data points to known data
points, or instead detect patterns in the training data and build a
predictive model, much like scientists do (instance-based versus model-
based learning).
Hichem Felouat - hichemfel@gmail.com - Algeria 20
Types of Machine Learning Systems

Hichem Felouat - hichemfel@gmail.com - Algeria 21

Types of Machine Learning Systems
Supervised learning :

In supervised learning, the training data you feed to the

algorithm includes the desired solutions, called labels.

• When y is real, we talk about regression.

• When y is discrete, we talk about classification.

Hichem Felouat - hichemfel@gmail.com - Algeria 22

Types of Machine Learning Systems

A labeled training set for supervised learning.

Hichem Felouat - hichemfel@gmail.com - Algeria 23
Types of Machine Learning Systems
Here are some of the most important supervised
learning algorithms:

• k-Nearest Neighbors
• Linear Regression
• Logistic Regression
• Support Vector Machines (SVMs)
• Decision Trees and Random Forests
• Neural networks*
Hichem Felouat - hichemfel@gmail.com - Algeria 24
Types of Machine Learning Systems

Unsupervised Learning:
In unsupervised learning, as you might guess, the training
data is unlabeled. The system tries to learn without a
teacher.

No labels are given to the learning algorithm, leaving it on its own to

explore or find structure in the data.
Hichem Felouat - hichemfel@gmail.com - Algeria 25
Types of Machine Learning Systems

An unlabeled training set for unsupervised learning.

Hichem Felouat - hichemfel@gmail.com - Algeria 26

Types of Machine Learning Systems

Here are some of the most important

unsupervised learning algorithms:

• Clustering
• Visualization and dimensionality reduction

Hichem Felouat - hichemfel@gmail.com - Algeria 27

Types of Machine Learning Systems
Semi-Supervised Learning :

Some algorithms can deal with partially labeled

training data, usually a lot of unlabeled data and a little
bit of labeled data. This is called semi-supervised
learning.

Most semi-supervised learning algorithms are

combinations of unsupervised and supervised
algorithms.
Hichem Felouat - hichemfel@gmail.com - Algeria 28
Types of Machine Learning Systems
Reinforcement Learning :
• The learning system called an agent in this context.

• Can observe the environment, select and perform actions, and

get rewards in return (or penalties in the form of negative
rewards).

• It must then learn by itself what is the best strategy, called a

policy, to get the most reward over time.

• A policy defines what action the agent should choose when it

is in a given situation.
Hichem Felouat - hichemfel@gmail.com - Algeria 29
Types of Machine Learning Systems

Reinforcement Learning
Hichem Felouat - hichemfel@gmail.com - Algeria 30
Types of Machine Learning Systems

Batch learning:

In batch learning, the system is incapable of learning

incrementally: it must be trained using all the available
data. This will generally take a lot of time and computing
resources, so it is typically done offline. First, the system is
trained, and then it is launched into production and runs
without learning anymore; it just applies what it has learned.
This is called offline learning.

Hichem Felouat - hichemfel@gmail.com - Algeria 31

Types of Machine Learning Systems

On-line learning:
In online learning, you train the system
incrementally by feeding i t d at a i n st an ces
sequentially, either individually or by small groups
called mini batches. Each learning step is fast and
cheap, so the system can learn about new data on
the fly, as it arrives.
Hichem Felouat - hichemfel@gmail.com - Algeria 32
Types of Machine Learning Systems

Online learning
Hichem Felouat - hichemfel@gmail.com - Algeria 33
Instance-Based VS Model-Based Learning

Instance-based learning:
The system learns the examples by heart, then
generalizes to new cases using a similarity measure.

Hichem Felouat - hichemfel@gmail.com - Algeria 34

Instance-Based VS Model-Based Learning
Model-based learning:
Build a model of these examples, then use that model
to make predictions.

Hichem Felouat - hichemfel@gmail.com - Algeria 35

Active Learning vs Transfer Learning
• Active Learning is a case of semi-supervised machine
learning.
• For classification, you would want to sample a small
subset of examples and find those labels and use
these labeled examples as your training data for a
classifier.

Hichem Felouat - hichemfel@gmail.com - Algeria 36

Active Learning vs Transfer Learning
Transfer learning (TL) is a research problem in ML that focuses on
storing knowledge gained while solving one problem and applying it
to a different but related problem. For example, knowledge gained
while learning to recognize Cats could apply when trying to
recognize Tigers.

Hichem Felouat - hichemfel@gmail.com - Algeria 37

Loss Function

The loss function computes the error for a single training

example, while the cost function is the average of the loss
functions of the entire training set.
Hichem Felouat - hichemfel@gmail.com - Algeria 38
Machine Learning Vocabulary 3
• Hyperparameters : are configuration variables that are external to the model
and whose values cannot be estimated from data. That is to say, they can not
be learned directly from the data in standard model training. They are almost
always specified by the machine learning engineer prior to training.

• Regression: this is the problem of predicting a real value for each item.
Examples of regression include prediction of stock values or that of variations
of economic variables.

• Classification: this is the problem of assigning a category to each item.

• Clustering: this is the problem of partitioning a set of items into

homogeneous subsets.
Hichem Felouat - hichemfel@gmail.com - Algeria 39
Machine Learning Vocabulary 3
Simple Regression: If you want to predict a single
value (e.g., the price of a house, given many of its
features). The output is the predicted value.

Multivariate Regression (i.e., to predict multiple

values at once). For example, to locate the center of
an object in an image, you need to predict 2D
coordinates.
Hichem Felouat - hichemfel@gmail.com - Algeria 40
Machine Learning Vocabulary 3

Integers : Labels [2, 0, 1]

one-hot : Labels [[0 0 1], [1 0 0], [ 0 1 0]]
Hichem Felouat - hichemfel@gmail.com - Algeria 41
 In Summary
1) You studied the data.

2) You selected a model.

3) You trained it on the training data.

4) F i n a l l y , y o u a p p l i e d t h e m o d e l t o m a k e
predictions on new cases.
Hichem Felouat - hichemfel@gmail.com - Algeria 42
Thank you for your
attention
Hichem Felouat ...
Hichem Felouat - hichemfel@gmail.com - Algeria 43