ML UNIT-2 (1).pptx machine learning s fo

Unit II:
SUPERVISEDLEARNING(REGRESSION/CLASSIFICATION):
1) Basic Methods:
Distance based Methods,
Nearest Neighbours,
Decision Trees,
Naive Bayes,
2)Linear Models:
Linear Regression,
Logistic Regression,
Generalized Linear Models,
Support Vector Machines,
3) Binary Classification:
Multiclass/Structured outputs,
MNIST,
Ranking.

I) Basic Methods:
DISTANCE BASED METHODS:
 Distanced based algorithms are machine
learning algorithms that classify queries by
computing distances between these
points(queries).

1) Euclidean Distance
It is the most common use of distance.
Euclidean distance is also known as simply
distance.
When data is dense or continuous, this is the
best proximity measure.

The Euclidean distance between two points is
the length of the path connecting them.
The Pythagorean theory gives distance
between two points.

2) Manhattan Distance
If you want to find Manhattan distance
between two different points (x1, y1) and (x2,
y2).
• Manhattan distance = Ix2 – x1I + Iy2 – y1I

Diagrammatically, it would look like traversing
the path from point A to point B while walking
on the pink straight line.

3) Minkowski Distance
• The generalized form of the Euclidean and
Manhattan Distances is the Minkowski
Distance.
• You can express the Minkowski distance as

If h= 1, Manhattan Distance is represented.
If h=2 in the above formula, Euclidean
Distance is represented.

NEAREST NEIGHBOURS (KNN)
The abbreviation KNN stands for “K-Nearest
Neighbour”.
It is a supervised machine learning algorithm.
The algorithm can be used to solve both
classification and regression problem
statements.

The number of nearest neighbours to a new
unknown variable that has to be predicted or
classified is denoted by the symbol ‘K’.

KNN calculates the distance from all points in
the proximity of the unknown data and filters
out the ones with the shortest distances to it.
As a result, it’s often referred to as a distance-
based algorithm.

In order to correctly classify the results, we
must first determine the value of K (Number
of Nearest Neighbours).
• It is recommended to always select an odd
value of K

NAÏVE BAYES CLASSIFIER ALGORITHM
Naïve Bayes algorithm is a
supervised learning algorithm, which
is based on Bayes theorem and used
for solving classification problems.

• It is mainly used in text classification that
includes a high-dimensional training dataset.
• which helps in building the fast machine
learning models that can make quick
predictions.

It is a probabilistic classifier, which means it
predicts on the basis of the probability of an
object.

BAYES' THEOREM:
• Bayes' theorem is also known as Bayes'
Rule or Bayes' law, which is used to
determine the probability of a hypothesis with
prior knowledge. It depends on the
conditional probability.
• The formula for Bayes' theorem is given as:

• Where,
• P(A|B) is Posterior probability: Probability of
hypothesis A on the observed event B.
• P(B|A) is Likelihood probability: Probability of
the evidence given that the probability of a
hypothesis is true.

WORKING OF NAÏVE BAYES' CLASSIFIER:
Example:
Suppose we have a dataset of weather
conditions and corresponding target
variable "Play".
So using this dataset we need to
decide that” whether we should play
or not on a particular day according
to the weather conditions”.

NAÏVE BAYES ALGORITHM:
Step1: Convert the given dataset into
frequency tables.
Step2: Generate Likelihood table by
finding the probabilities of given
features.
Step3: Now, use Bayes theorem to
calculate the posterior probability

• Problem: If the weather is sunny, then the
Player should play or not?
• Solution: To solve this, first consider the below
dataset:

Outlook Play
1 Rainy Yes
2 Sunny Yes
3 Overcast Yes
4 Overcast Yes
5 Sunny No
6 Rainy Yes
7 Sunny Yes
8 Overcast Yes
9 Rainy No
10 Sunny No
11 Sunny Yes
12 Rainy No
13 Overcast Yes
14 Overcast Yes

Step-1
Frequency table for the Weather Conditions
WEATHER YES NO
Overcast 5 0
Rainy 2 2
Sunny 3 2
Total 10 5

Step-2:
Likelihood table weather condition:
WEATHER NO YES
Overcast 0 5 5/14= 0.35
Rainy 2 2 4/14=0.29
Sunny 2 3 5/14=0.35
All 4/14=0.29 10/14=0.71

APPLYING BAYES'THEOREM: “yes “ case
P(Yes|Sunny)= P(Sunny|Yes)*P(Yes)/P(Sunny)
P(Sunny|Yes)= 3/10= 0.3
P(Sunny)= 0.35
P(Yes)=0.71
So P(Yes|Sunny) = 0.3*0.71/0.35= 0.60

P(No|Sunny)= P(Sunny|No)*P(No)/P(Sunny)
P(Sunny|NO)= 2/4=0.5
P(No)= 0.29
P(Sunny)= 0.35
So P(No|Sunny)= 0.5*0.29/0.35 = 0.41

• So as we can see from the above calculation
that P(Yes|Sunny)>P(No|Sunny)
Hence on a Sunny day, Player can
play the game.

• DECISION TREE
• A decision tree is a structure that includes a
root node, branches, and leaf nodes.
• Each internal node denotes a test on an
attribute, each branch denotes the outcome
of a test.
• each leaf node holds a class label. The
topmost node in the tree is the root node.

• The following decision tree is for the concept
buy_computer that indicates whether a customer at
a company is likely to buy a computer or not.
• Each internal node represents a test on an attribute.
Each leaf node represents a class.

LINEAR MODELS:
 Linear Regression.
 Logistic Regression.
 Generalized Linear Models.
 Support Vector Machines.

What is Linear Regression?
“Linear regression is a statistical method that
is used to predict a continuous dependent
variable(target variable) based on one or more
independent variables”.

This technique assumes a linear relationship
between the dependent and independent
variables.
which implies that the dependent variable
changes proportionally with changes in the
independent variables.

Types of Linear Regression
There are two main types of linear regression:
1) Simple linear regression: This involves
predicting a dependent variable based on a
single independent variable.
2) Multiple linear regression: This involves
predicting a dependent variable based on
multiple independent variables.

LOGISTIC REGRESSION
Logistic regression is a supervised machine
learning algorithm mainly used for
binary classification where we use a logistic
function, also known as a sigmoid function
that takes input as independent variables and
produces a probability value between 0 and
1.

For example:
we have two classes Class 0 and Class 1.
 if the value of the logistic function for
an input is greater than 0.5 (threshold value)
then it belongs to Class 1
 otherwise it belongs to Class 0.
It is mainly used for classification problems.

Sr.
No Linear Regresssion Logistic Regression
1
Linear regression is used to predict the
continuous dependent variable using a
given set of independent variables.
Logistic regression is
used to predict the
categorical dependent
variable using a given
set of independent
variables.
2 Linear regression is used for solving
Regression problem.
It is used for solving
classification
problems.
3 In this we predict the value of continuous
variables
In this we predict
values of categorical
varibles
4 In this we find best fit line.
In this we find S-
Curve .

5 Least square estimation method is used
for estimation of accuracy.
Maximum likelihood
estimation method is
used for Estimation of
accuracy.
6 The output must be continuous value,such
as price,age,etc.
Output is must be
categorical value such
as 0 or 1, Yes or no,
etc.
7 It required linear relationship between
dependent and independent variables.
It not required linear
relationship.

GENERALIZED LINEAR MODELS
Generalized Linear Models (GLMs) are a class
of regression models that can be used to
model a wide range of relationships between
a response variable and one or more predictor
variables.

SUPPORT VECTOR MACHINE
ALGORITHM:
“Support Vector Machine or SVM is
one of the most popular Supervised
Learning algorithms, which is used
for Classification as well as
Regression problems”.

THE GOAL of the SVM algorithm is to
create the best line or decision boundary that
can segregate n-dimensional space into
classes so that we can easily put the new data
point in the correct category in the future.
This best decision boundary is called a hyper
plane.

SVM chooses the extreme points/vectors that
help in creating the hyper plane.
These extreme cases are called as support
vectors, and hence algorithm is termed as
Support Vector Machine.

There are two different categories that are classified using a
decision boundary or hyper plane:

APPLICATIONS
SVM algorithm can be used for Face
detection, image classification, text
categorization, etc.

TYPES OF SVM:
1) Linear SVM: if a dataset can be
classified into two classes by
using a single straight line.
2) Non-linear SVM:
if a dataset cannot be classified
by using a straight line, then such
data is termed as non-linear data.

Linear SVM:
Suppose we have a dataset that has two tags (green and
blue), and the dataset has two features x1 and x2. We
want a classifier that can classify the pair(x1, x2) of
coordinates in either green or blue.

So as it is 2-d space so by just using a straight line, we
can easily separate these two classes. But there can
be multiple lines that can separate these classes.

The SVM algorithm helps to find the best line
or decision boundary; this best boundary or
region is called as a hyperplane.
SVM algorithm finds the closest point of the
lines from both the classes.
These points are called support vectors.

The distance between the vectors and the
hyperplane is called as margin.
And the goal of SVM is to maximize this
margin.
The hyperplane with maximum margin is
called the optimal hyperplane.

2) Non-Linear SVM:
 If data is linearly arranged, then we can separate it by
using a straight line, but for non-linear data, we cannot
draw a single straight line.

So to separate these data points, we need to
add one more dimension.
For linear data, we have used two dimensions
x and y, so for non-linear data, we will add a
third dimension z. It can be calculated as
z=x2
+y2

By adding the third dimension, the sample space
will become as below image:

So now, SVM will divide the datasets into classes
in the following way. Consider the below image:

Binary Classification:
Multiclass/Structured outputs

Definition of Classification
“Classification is the process of assigning new
input variables (X) to the class they most likely
belong to, based on a classification model, as
constructed from previously labeled training
data”.

For example:
 if we are taking a dataset of scores of a
cricketer in the past few matches, along with
average, strike rate, not outs etc, we can
classify him as “in form” or “out of form”.

Types of Classification
There are two types of classifications;
1) Binary classification
2) Multi-class classification

Binary Classification
It is a process or task of classification, in which
a given data is being classified into two
classes. It’s basically a kind of prediction
about which of two groups the thing belongs
to.

EXAMPLE:
Let us suppose, two emails are sent to you,
one is sent by an insurance company that
keeps sending their ads, and the other is from
your bank regarding your credit card bill.
The email service provider will classify the two
emails, the first one will be sent to the spam
folder and the second one will be kept in the
primary one.

This process is known as binary classification,
as there are two discrete classes, one is spam
and the other is primary.

SOME OF THE MOST COMMON ALGORITHMS
USED BY BINARY CLASSIFICATION ARE .
• Logistic Regression
• k-Nearest Neighbors
• Decision Trees
• Support Vector Machine
• Naive Bayes

Multiclass Classification
• Multi-class classification is the task of
classifying elements into different classes.
Unlike binary, it doesn’t restrict itself to any
number of classes.

EXAMPLES OF MULTI-CLASS CLASSIFICATION
ARE :
1) classification of news in different categories,
2) classifying books according to the subject,
3) classifying students according to their streams
etc.

Binary vs Multiclass Classification
Parameters
Binary
classification
Multi-class
classification
No. of classes
It is a classification
of two groups, i.e.
classifies objects in
at most two classes.
There can be any
number of classes in
it, i.e., classifies the
object into more
than two classes.

Algorithms used
The most popular
algorithms used by
the binary
classification are-
• Logistic Regression
•k-Nearest
Neighbors
•Decision Trees
•Support Vector
Machine
•Naive Bayes
Popular algorithms
that can be used for
multi-class
classification
include:
•k-Nearest
Neighbors
•Decision Trees
•Naive Bayes
•Random Forest.
•Gradient Boosting

Examples
Examples of binary
classification
include-
•Email spam
detection (spam or
not).
•Churn prediction
(churn or not).
•Conversion
prediction (buy or
not).
Examples of multi-
class classification
include:
•Face classification.
•Plant species
classification.
•Optical character
recognition.

MNIST
The MNIST database (Modified National
Institute of Standards and Technology
database) is a large database.
The database is also widely used for training
and testing in the field of machine learning.

The MNIST database contains 60,000 training
images and 10,000 testing images.
Half of the training set and half of the test set
were taken from NIST's training dataset,
while the other half of the training set and the
other half of the test set were taken from
NIST's testing dataset.

RANKING
A binary classification system involves
a system that generates ratings for
each occurrence, which, by ordering
them, are turned into rankings, which
are then compared to a threshold..

Occurrences with rankings above the
threshold are declared positive, and
occurrences below the threshold are declared
negative

ML UNIT-2 (1).pptx machine learning s fo

ML UNIT-2 (1).pptx machine learning s fo

More Related Content

Similar to ML UNIT-2 (1).pptx machine learning s fo

Recently uploaded

ML UNIT-2 (1).pptx machine learning s fo