1. What is Bayesian Classification?

Explain the basic concepts of

Bayesian Classification.

Definition:

Bayesian Classification is a supervised learning method that applies

Bayes' Theorem to predict the probability that a given data point
belongs to a particular class. It is widely used in statistics and machine
learning for classification tasks.

Basic Concepts in Bayesian Classification:

1. Prior Probability (P(C))

This is the initial belief about the probability of a class before any
evidence is taken into account.

2. Likelihood (P(X|C))

This is the probability of observing the features X given that the class is C.

3. Posterior Probability (P(C|X))

This is the updated probability of the class after observing the features. It
is the basis for decision-making in Bayesian classification.

4. Evidence (P(X))

This is the total probability of the data under all possible classes. It
normalizes the result and can often be omitted in comparisons.

2. What is meant by mutually exclusive events? Give some

examples.

Definition:

Mutually exclusive events are two or more events that cannot occur
at the same time. In other words, if one event happens, the other(s)
must not happen.

Mathematically, for two events A and B to be mutually exclusive:

P(A∩B)=0P(A \cap B) = 0P(A∩B)=0

This means the probability of both A and B occurring together is zero.

Explanation:

When events are mutually exclusive, the occurrence of one excludes the
possibility of the others occurring in the same trial.

For example, when tossing a coin, the result can be either Heads (H) or
Tails (T) — not both. So, "getting Heads" and "getting Tails" are mutually
exclusive events.

Examples:

1. Tossing a coin:

o Event A: Getting Heads

o Event B: Getting Tails

o A and B are mutually exclusive.

2. Rolling a die:

o Event A: Getting a 2

o Event B: Getting a 5

o A and B are mutually exclusive because a die shows only one

number at a time.

3. Drawing a card from a deck:

o Event A: Drawing a red card

o Event B: Drawing a black card

o These events are mutually exclusive.

4. Passing or Failing an Exam:

o A student cannot both pass and fail at the same time. So,
these events are mutually exclusive.

Key Point:

If two events are mutually exclusive, they cannot happen together,

and:

P(A or B)=P(A)+P(B)P(A \text{ or } B) = P(A) + P(B)P(A or B)=P(A)+P(B)

3. What is meant by joint probability? Explain with example.

Definition:

Joint probability refers to the probability of two or more events

happening at the same time. It is denoted as:

P(A∩B)P(A \cap B)P(A∩B)

This represents the probability that event A and event B both occur.

Explanation:

Joint probability helps us understand the likelihood of simultaneous

occurrences of multiple events.

 If A and B are independent events, then:

P(A∩B)=P(A)⋅P(B)P(A \cap B) = P(A) \cdot P(B)P(A∩B)=P(A)⋅P(B)

 If the events are dependent, then:

P(A∩B)=P(A)⋅P(B∣A)P(A \cap B) = P(A) \cdot P(B|A)P(A∩B)=P(A)⋅P(B∣A)

Example (Independent Events):

Suppose you roll a fair die and toss a fair coin.

 Event A: Rolling a 3 → P(A)=16P(A) = \frac{1}{6}P(A)=61

 Event B: Getting Heads → P(B)=12P(B) = \frac{1}{2}P(B)=21

Since rolling a die and tossing a coin are independent:

P(A∩B)=P(A)⋅P(B)=16⋅12=112P(A \cap B) = P(A) \cdot P(B) = \frac{1}

{6} \cdot \frac{1}{2} = \frac{1}{12}P(A∩B)=P(A)⋅P(B)=61⋅21=121

This means there’s a 1 in 12 chance of rolling a 3 and getting heads

simultaneously.

Example (Dependent Events):

Suppose you draw two cards from a deck without replacement.

 Event A: First card is an Ace → P(A)=452P(A) = \frac{4}

{52}P(A)=524
  Event B: Second card is also an Ace → P(B∣A)=351P(B|A) = \frac{3}
{51}P(B∣A)=513

So, joint probability:

P(A∩B)=452⋅351=122652≈0.0045P(A \cap B) = \frac{4}{52} \cdot \

frac{3}{51} = \frac{12}{2652} \approx 0.0045P(A∩B)=524⋅513=265212
≈0.0045

Use Cases:

 Risk analysis

 Weather forecasting

 Medical diagnosis

 Marketing (e.g., likelihood a customer buys two products)

4. Define Conditional Probability. Give an Example.

Definition:

Conditional probability is the probability of an event occurring given

that another event has already occurred. It is denoted as:

P(A∣B)=P(A∩B)P(B)(provided P(B)>0)P(A|B) = \frac{P(A \cap B)}{P(B)} \

quad \text{(provided } P(B) > 0\text{)}P(A∣B)=P(B)P(A∩B)
(provided P(B)>0)

Here,

 P(A|B) = Probability of event A given event B has occurred

 P(A ∩ B) = Probability of both A and B occurring

 P(B) = Probability of event B

Explanation:

Conditional probability refines the sample space to only include

outcomes where event B has occurred. It helps in updating
probabilities based on new information.

Example:
Problem: A card is drawn from a standard deck of 52 cards.
Let:

 Event A: The card is a king

 Event B: The card is a face card (Jack, Queen, King → 12 cards)

Now, what is the probability that the card is a king given that it's a face
card?

P(A∣B)=P(A∩B)P(B)=4/5212/52=412=13P(A|B) = \frac{P(A \cap B)}{P(B)}

= \frac{4/52}{12/52} = \frac{4}{12} = \frac{1}{3}P(A∣B)=P(B)P(A∩B)
=12/524/52=124=31

So, the conditional probability of drawing a king given it is a face card is

1/3.

Real-Life Examples:

1. Medical Diagnosis:

o Probability a person has a disease given they tested positive.

2. Weather Forecasting:

o Probability it rains today given it was cloudy yesterday.

3. Spam Detection:

o Probability an email is spam given it contains certain

keywords.

5. What is meant by Bayes' Theorem? Explain with example.

Definition:

Bayes’ Theorem describes the probability of an event based on prior

knowledge of conditions related to the event. It is used to update the
probability of a hypothesis as more evidence or information becomes
available.

The formula is:

P(A∣B)=P(B∣A)⋅P(A)P(B)P(A|B) = \frac{P(B|A) \cdot P(A)}

{P(B)}P(A∣B)=P(B)P(B∣A)⋅P(A)

Where:
  P(A|B) = Posterior probability (Probability of A given B)

 P(B|A) = Likelihood (Probability of B given A)

 P(A) = Prior probability of A

 P(B) = Marginal probability of B

Explanation:

Bayes' Theorem helps in reversing conditional probabilities. It allows

us to compute the probability of a cause (A) given an observed effect (B).
It is a core principle in Bayesian statistics and inference.

Example (Medical Diagnosis):

Suppose:

 1% of a population has a disease → P(D)=0.01P(D) = 0.01P(D)=0.01

 A test detects the disease 99% of the time → P(Pos∣D)=0.99P(Pos|D)

= 0.99P(Pos∣D)=0.99

 False positives occur 5% of the time → P(Pos∣¬D)=0.05P(Pos|\neg D)

= 0.05P(Pos∣¬D)=0.05

Now, what is the probability a person actually has the disease, given
they tested positive?
Use Bayes’ Theorem:

P(D∣Pos)=P(Pos∣D)⋅P(D)P(Pos∣D)⋅P(D)+P(Pos∣¬D)⋅P(¬D)P(D|Pos) = \
frac{P(Pos|D) \cdot P(D)}{P(Pos|D) \cdot P(D) + P(Pos|\neg D) \cdot P(\neg
D)}P(D∣Pos)=P(Pos∣D)⋅P(D)+P(Pos∣¬D)⋅P(¬D)P(Pos∣D)⋅P(D)
P(D∣Pos)=0.99⋅0.010.99⋅0.01+0.05⋅0.99=0.00990.0099+0.0495=0.00990.
0594≈0.1667P(D|Pos) = \frac{0.99 \cdot 0.01}{0.99 \cdot 0.01 + 0.05 \
cdot 0.99} = \frac{0.0099}{0.0099 + 0.0495} = \frac{0.0099}{0.0594} \
approx 0.1667P(D∣Pos)=0.99⋅0.01+0.05⋅0.990.99⋅0.01
=0.0099+0.04950.0099=0.05940.0099≈0.1667

So, the probability the person actually has the disease is about
16.67%, despite testing positive.

Applications of Bayes’ Theorem:

 Spam filters
  Medical diagnosis

 Machine learning (Naive Bayes classifier)

 Risk assessment

 Forensic science

6. Define Prior and Posterior Probabilities with an Example.

1. Prior Probability (P(A)):

Prior probability represents our initial belief about the probability of an

event before new evidence is taken into account.
It reflects existing knowledge or assumptions.

Example:
If 1% of people in a city have a certain disease, then:
P(Disease) = 0.01 is the prior probability.

2. Posterior Probability (P(A|B)):

Posterior probability is the updated probability of an event after

considering new evidence or data.
It is calculated using Bayes’ Theorem:

P(A∣B)=P(B∣A)⋅P(A)P(B)P(A|B) = \frac{P(B|A) \cdot P(A)}

{P(B)}P(A∣B)=P(B)P(B∣A)⋅P(A)

Example (continued):
If a person tests positive, and the test is 99% accurate, we now update
the probability that the person has the disease using Bayes’ Theorem.
This updated probability is called the posterior probability.

P(Disease∣Positive)≈16.67%(as shown earlier)P(Disease|Positive) \approx

16.67\% \quad \text{(as shown earlier)}P(Disease∣Positive)≈16.67%
(as shown earlier)

Key Difference:

Aspect Prior Probability Posterior Probability

Based
Initial knowledge Updated with new evidence
on

Used in Before After considering

Aspect Prior Probability Posterior Probability

experiment/data experiment/data

Input to Bayes’
Part of Output of Bayes’ Theorem
Theorem

Real-Life Applications:

 Medical testing: Updating risk based on test results

 Spam detection: Revising spam probability with email content

 Weather prediction: Adjusting forecasts based on current data

7. Describe the naive Bayes classification technique with an

illustrative example.

Naive Bayes Classification Technique:

Naive Bayes is a probabilistic classifier based on Bayes' theorem,

assuming independence among features. Despite this "naive" assumption,
it performs well in many real-world situations.

Key idea:
It calculates the probability of each class given the input features and
predicts the class with the highest posterior probability.

Mathematically:
For a class CCC and features x1,x2,...,xnx_1, x_2, ..., x_nx1,x2,...,xn,

P(C∣x1,x2,...,xn)=P(C)×∏i=1nP(xi∣C)P(x1,x2,...,xn)P(C|x_1, x_2, ..., x_n) = \

frac{P(C) \times \prod_{i=1}^{n} P(x_i|C)}{P(x_1, x_2, ..., x_n)}P(C∣x1,x2
,...,xn)=P(x1,x2,...,xn)P(C)×∏i=1nP(xi∣C)

Since P(x1,x2,...,xn)P(x_1, x_2, ..., x_n)P(x1,x2,...,xn) is constant for all

classes, it can be ignored for classification.

Illustrative Example:

Suppose an email can be classified as Spam or Not Spam based on

words it contains.

 Features: presence of words like "offer", "free", "win".

 We calculate the probabilities P(Spam)P(\text{Spam})P(Spam),

P(Not Spam)P(\text{Not Spam})P(Not Spam), and conditional
 probabilities like P("offer"∣Spam)P(\text{"offer"}|\
text{Spam})P("offer"∣Spam).

If the product of probabilities for Spam is greater than that for Not Spam,
classify the email as Spam; otherwise, Not Spam.

8. Explain the algorithm for naive Bayes classifier.

Naive Bayes Classifier Algorithm:

1. Prepare the Training Data:

Collect a labeled dataset where each instance is associated with a
class label

2. P(Ci)=Number of instances in class CiTotal number of instancesP(C_i

) = \frac{\text{Number of instances in class } C_i}{\text{Total
number of instances}}P(Ci
)=Total number of instancesNumber of instances in class Ci

3. Calculate Likelihood:
For each feature, calculate the conditional probability of the feature
value given the class. Assume features are independent (Naive
assumption):

P(xj∣Ci)P(x_j | C_i)P(xj∣Ci)

4. Apply Bayes Theorem:

For a new instance with features x=(x1,x2,...,xn)x = (x_1, x_2, ...,
x_n)x=(x1,x2,...,xn), calculate the posterior probability for each
class:

P(Ci∣x)=P(Ci)∏j=1nP(xj∣Ci)P(x)P(C_i | x) = \frac{P(C_i) \prod_{j=1}^{n}

P(x_j | C_i)}{P(x)}P(Ci∣x)=P(x)P(Ci)∏j=1nP(xj∣Ci)

Since P(x)P(x)P(x) is the same for all classes, it can be ignored for
comparison purposes.

5. Classify:
Assign the new instance to the class with the highest posterior
probability:

C^=arg⁡max⁡CiP(Ci)∏j=1nP(xj∣Ci)\hat{C} = \arg\max_{C_i} P(C_i) \

prod_{j=1}^{n} P(x_j | C_i)C^=argCimaxP(Ci)j=1∏nP(xj∣Ci)

6. Output the Class:

The class C^\hat{C}C^ is the predicted class for the instance.