Machine Learning Class Notes: Introduction to Supervised Learning
Course: Introduction to Machine Learning
Date: August 2025
Instructor: Dr. Jane Carter
Institution: Virtual University of Technology
---
1. Overview of Machine Learning
Machine learning (ML) enables computers to learn from data without explicit
programming. Supervised learning, a core ML paradigm, involves training models on
labeled data to predict outcomes. These notes cover supervised learning
fundamentals, algorithms, and practical applications.
2. Key Concepts
- Labeled Data: Data with input-output pairs (e.g., house sizes and their prices).
- Features: Input variables (e.g., house size, number of bedrooms).
- Target: Output variable to predict (e.g., house price).
- Training: Adjusting model parameters to minimize prediction errors.
- Testing: Evaluating model performance on unseen data.
3. Types of Supervised Learning
- Regression: Predicts continuous values (e.g., predicting temperature).
- Classification: Predicts discrete categories (e.g., spam vs. not spam).
4. Common Algorithms
4.1 Linear Regression
- Purpose: Predict continuous outcomes.
- Model: y = w₀ + w₁x₁ + w₂x₂ + ... + wₙxₙ, where w₀ is the intercept, wᵢ are
weights, and xᵢ are features.
- Example: Predicting house prices based on size (x₁) and location score (x₂).
- Cost Function: Mean Squared Error (MSE) = (1/n)∑(y_pred - y_true)².
- Optimization: Gradient descent adjusts weights to minimize MSE.
4.2 Logistic Regression
- Purpose: Binary classification (e.g., pass/fail).
- Model: Uses the sigmoid function, σ(z) = 1/(1 + e⁻ᶻ), where z = w₀ + w₁x₁ + ... +
wₙxₙ.
- Example: Predicting if a student passes (1) or fails (0) based on study hours and
attendance.
- Cost Function: Log Loss (Binary Cross-Entropy).
- Optimization: Gradient descent.
4.3 Decision Trees
- Purpose: Classification or regression.
- Structure: A tree where nodes represent decisions based on feature values,
leading to a final prediction.
- Example: Classifying emails as spam based on word frequency and sender.
- Advantage: Interpretable, handles non-linear data.
- Disadvantage: Prone to overfitting without pruning.
5. Practical Example: Predicting Student Performance
- Dataset: 100 students, features (study hours, attendance %), target (exam score).
- Steps:
1. Preprocess: Normalize study hours (0–1 scale), handle missing attendance
values.
2. Split Data: 80% training, 20% testing.
3. Model: Train a linear regression model.
4. Evaluation: Calculate MSE on test set (e.g., MSE = 12.5 indicates average
squared error).
5. Interpretation: Study hours have a stronger impact (w₁ = 0.7) than attendance
(w₂ = 0.3).
6. Evaluation Metrics
- Regression: MSE, Root Mean Squared Error (RMSE), R² Score.
- Classification: Accuracy, Precision, Recall, F1 Score.
- Example: A model with 85% accuracy correctly classifies 85/100 samples.
7. Tools and Libraries
- Python: Use scikit-learn for model implementation (e.g., `from
sklearn.linear_model import LinearRegression`).
- Example Code:
```
from sklearn.linear_model import LinearRegression
import numpy as np
X = np.array([[2, 80], [4, 90], [6, 85]]) # Features: study hours, attendance
y = np.array([70, 85, 90]) # Target: exam scores
model = LinearRegression()
model.fit(X, y)
predictions = model.predict([[3, 82]]) # Predict score for 3 hours, 82% attendance
print(f"Predicted score: {predictions[0]:.2f}")
```
- Other Tools: TensorFlow, PyTorch for advanced models.
8. Challenges and Solutions
- Overfitting: Model performs well on training data but poorly on test data.
Solution: Regularization (e.g., Lasso, Ridge).
- Underfitting: Model is too simple. Solution: Increase model complexity or add
features.
- Data Quality: Missing or noisy data. Solution: Impute missing values, clean
outliers.
9. Applications
- Healthcare: Predicting patient outcomes based on medical history.
- Finance: Credit risk assessment using customer data.
- Marketing: Customer segmentation for targeted campaigns.
10. Further Study
- Read “Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow” by
Aurélien Géron.
- Explore Kaggle datasets for hands-on practice (e.g., Titanic dataset for
classification).
- Join ML communities on X or Reddit for discussions and updates.
---
These notes are intended for students new to machine learning. Practice
implementing these algorithms in Python and analyze real datasets to deepen your
understanding.
