Naive Bayes Classifier

●Naive Bayes is a supervised learning algorithm, based on Bayes theorem and used to solve classification problems.

●It is a probabilistic classifier, which means it predicts on the basis of the probability of an object.

Types of Probabilities

● Prior Probability also called as Unconditional Probability

E.g. P(cavity) = 0.1, ○Probability of a patient having a cavity is 0.1 in absence of any other information

● Posterior probability also called as Conditional Probability

E.g. P(cavity/toothache) = 0.8, Probability of a patient having a cavity is 0.8 given that he has toothache

●Total probability Theorem:

●Bayes’ Theorem:

○Let X be a data sample (“evidence”): class label is unknown

○Let H be a hypothesis that X belongs to class C

○Classification is to determine P(H|X), (i.e., posteriori probability):  the probability that the hypothesis holds given the observed data sample X

Working of Naive Bayes Algorithm

1.Convert the data set into a frequency table.

2.Create Likelihood table by finding the probabilities.

3.Use Naive Bayesian equation to calculate the posterior probability

argmax chooses the value of y that gives the maximum value of the expression

E.g.

P(Spam∣X)=0.72

P(NotSpam∣X)=0.28, here X is the data sample

P(y) is the prior probability of the class.

$\pi means Product, P(x_i​∣y)$mean Probability of feature xi​, given that the class is y.

e.g. P(Outlook=Sunny∣Play=Yes)

Solved Problem:

Consider Play Tennis Data set

Classify the tuple <Outlook=Sunny, Temperature=Cool, Humidity=high, Wind=strong>

For classification, calculate

1.Prior probability- “probability of yes” and “probability of no”

2.Conditional probability

Prior probability

●“Yes” is appearing 9 times

●“No” is appearing 5 times

●P(Play_tennis=yes)= 9/14

●P(Play_tennis=no)= 5/14

The weather data, with counts and probabilities

Outlook

Temperature

Humidity

Wind

Play Tennis

Yes

No

Yes

No

Yes

No

Yes

No

Yes

No

Sunny

2

3

Hot

2

2

High

3

4

False

6

2

9

5

Overcast

4

0

Mild

4

2

Normal

6

1

True

3

3

Rain

3

2

Cool

3

1

Likelihood table

Sunny

2/9

3/5

Hot

2/9

2/5

High

3/9

4/5

False

6/9

2/5

9/14

5/14

Overcast

4/9

0/5

Mild

4/9

2/5

Normal

6/9

1/5

True

3/9

3/5

Rain

3/9

2/5

Cool

3/9

1/5

How the Frequency table is created?

Count the number of “Sunny” and “Yes”, there are 2 (“Sunny” and “Yes”), Similarly there are 3 (“Sunny” and “No”), So write 2 and 3 at respective place.

In the likelihood table, convert these frequencies into probabilities. So for Sunny and Yes column it will be 2/9 as total yes are 9 and sunny and yes are 2. Similarly for sunny and no it will be 3/9.

Likewise write for all the values.

Y_nb(yes) = p(yes)*( P(sunny/yes) * P(cool/yes) * P(high/yes) * P(strong/yes))

Y_nb(no) = p(no)*( P(sunny/no) * P(cool/no) * P(high/no) * P(strong/no))

Y_predict = max(Y_nb(yes) , Y_nb(no) )

Maximum value is that of No, therefore Naive Bayes Classifier will predict a “No”, So Play Tennis is “No” for the given tuple.