LOGISTIC REGRESSION
UNIT-III S3 SLO-2
INTRODUCTION TO LOGISTIC
REGRESSION
Logistic Regression is a statistical method for binary classification. It
predicts the probability that a given input belongs to a certain class.
Purpose: Used to model the relationship between a dependent binary
variable and one or more independent variables.
Output: Produces probabilities that are then converted to class labels
(e.g., yes/no, 0/1).
Type: Classification algorithm.
Output: Probability values between 0 and 1.
THE LOGISTIC FUNCTION (SIGMOID
FUNCTION)
Logistic Function: The core of logistic regression is the sigmoid function,
which maps any real-valued number into the range [0, 1].
Formula: σ(z)=11+e−z\sigma(z) = \frac{1}{1 + e^{-z}}σ(z)=1+e−z1where zzz is
the linear combination of inputs and weights.
Interpretation: The sigmoid function outputs a probability score which is then
used for classification.
Range: Outputs values between 0 and 1.
Purpose: Converts linear predictions into probabilities.
TRAINING AND PREDICTION WITH LOGISTIC
REGRESSION
Training: Uses Maximum Likelihood Estimation (MLE) to find the
best-fitting model parameters by maximizing the likelihood function.
Cost Function: The cost function used is the Log Loss or Binary Cross-
Entropy Loss.
Prediction: Once trained, the model predicts class probabilities by
applying the sigmoid function to the linear combination of inputs.
Training: Optimize model parameters.
Prediction: Classify based on probability thresholds (e.g., 0.5).
APPLICATIONS AND LIMITATIONS OF LOGISTIC
REGRESSION
Applications: Commonly used in binary classification problems like spam detection, medical diagnosis, and
customer churn prediction.
Advantages:
Simplicity and Interpretability: Easy to understand and interpret results.
Efficient: Works well with linearly separable data.
Limitations:
Linearity Assumption: Assumes a linear relationship between independent variables and the log-odds of
the dependent variable.
Not Suitable for Complex Relationships: May underperform with complex, non-linear data.
Strengths: Easy to implement and interpret.
Weaknesses: Limited to linear decision boundaries, can be outperformed by more complex models on non-linear
data.
