Naive Bayes Classifier | Part 7 | Mathematics behind Naive Bayes Algorithm — Transcript

Full transcript

1. 0:02So now let's do the mathematics part of
2. 0:04this algorithm.
3. 0:06Uh to be honest uh
4. 0:08which points to that
5. 0:09but other last video says it follow
6. 0:11here.
7. 0:12So what if it didn't work here? So
8. 0:15what we are going to do in this video is
9. 0:17we are going to derive the formula.
10. 0:19And to be last thing we used here was
11. 0:21mathematically derived from here.
12. 0:23Or along the way seeking it key in life
13. 0:26assumption here life is here. So let's
14. 0:29start.
15. 0:30Uh so let's start with this
16. 0:34uh vector.
17. 0:36Let's say
18. 0:37X1 X2 X3
19. 0:42XN.
20. 0:44And uh
21. 0:47uh this
22. 0:49uh
23. 0:51target variable C1 C2 C3
24. 0:54up till CK.
25. 0:57This is capital K. This is small K. Now
26. 1:00I think it's
27. 1:00difficult to see
28. 1:02it's a very simple thing.
29. 1:09So there was toss.
30. 1:11Uh there was venue.
31. 1:13And there was outlook.
32. 1:16Right? And there was result.
33. 1:18Or last time it was a two class
34. 1:20classification if you remember. Yes. I'm
35. 1:22sorry.
36. 1:23Won
37. 1:25or lost. Basically this is what we have
38. 1:27to do here. I don't know if this model
39. 1:29will follow X1
40. 1:31X2 X3.
41. 1:33So you have three things only. Okay?
42. 1:35Or similarly yeah C1
43. 1:38or won.
44. 1:39And C2
45. 1:41lost.
46. 1:42Yeah.
47. 1:43Okay. So we have a general case like
48. 1:45that. You want to make N columns. So my
49. 1:48X1 X2 X3 XN
50. 1:50or you may
51. 1:52there are this is a multi-class label or
52. 1:55a
53. 1:55problem. Okay? This is a binary problem,
54. 1:58binary class classification. So, there
55. 2:00was there is only C1 and C2. You have to
56. 2:02have what? K classes. Okay? So, there
57. 2:04are
58. 2:04K classes. It's a K class
59. 2:07classification.
60. 2:08So, I guess you all are pointing here.
61. 2:10Okay? So,
62. 2:12I'll remove this.
63. 2:18Now,
64. 2:19uh
65. 2:21you remember this, right? What we have
66. 2:23to do is
67. 2:25for every C, C1, C2, C3, I have to find
68. 2:28out this, probability of
69. 2:32C
70. 2:33K, but you have to pay. One is the
71. 2:35probability of the general K given
72. 2:39X.
73. 2:41You can do it.
74. 2:44So, probability of C K given X.
75. 2:54C2.
76. 2:56Okay? Uh
77. 3:02So, it's going to be
78. 3:04I'm just writing it this way.
79. 3:06So, we need to get back to that. Okay?
80. 3:08And now, according to Bayes' theorem, we
81. 3:11already know this that that this would
82. 3:13become
83. 3:14X given
84. 3:16C K multiplied by probability of
85. 3:19C K divided by probability of
86. 3:23X.
87. 3:24Right?
88. 3:25Or
89. 3:31This term is going to be common
90. 3:34everywhere. So,
91. 3:38So, what we will do is we'll remove
92. 3:40this.
93. 3:42So, we are left with this term.
94. 3:44Okay?
95. 3:47Now,
96. 3:48if you remember conditional probability,
97. 3:50according to conditional probability,
98. 4:01right?
99. 4:02So,
100. 4:06probability of A given B multiplied by
101. 4:08probability of B, this will be
102. 4:10probability of A intersection B, right?
103. 4:15So,
104. 4:16what we will do is we'll use this logic
105. 4:19over here.
106. 4:22I will write
107. 4:24probability of
108. 4:26CK happening given X will actually be
109. 4:30equal to probability of
110. 4:33X
111. 4:34intersection
112. 4:36CK.
113. 4:37This is basic
114. 4:39conditional probability.
115. 4:47Uh you already know this that you can
116. 4:49replace this with a comma,
117. 4:51right? So,
118. 4:54I'm going to go ahead and instead of
119. 4:55writing this, we are writing this.
120. 4:58X comma
121. 5:00CK.
122. 5:04Now,
123. 5:05equal to
124. 5:08Don't you think I can break down X into
125. 5:11X1 {comma} X2 {comma} X3 {comma}
126. 5:15XN {comma}
127. 5:17CK?
128. 5:28X is actually equal to
129. 5:30X1 intersection X2 intersection X3
130. 5:34intersection XN.
131. 5:49Right?
132. 5:55What we are doing is come is done cool.
133. 5:58X1 cool.
134. 5:59A minor.
135. 6:01Or is for a damn cool.
136. 6:06B minor.
137. 6:07Okay, so in a way in between we got it.
138. 6:10This.
139. 6:11Up again.
140. 6:13By.
141. 6:15Conditional probability we know this.
142. 6:17This can be replaced.
143. 6:21With this.
144. 6:23This is conditional probability.
145. 6:29Probability of. A given B. So this is A.
146. 6:34X1.
147. 6:35Given.
148. 6:37X2 {comma} X3.
149. 6:40XN {comma}
150. 6:42CK.
151. 6:48I just want to know
152. 6:49to get.
153. 6:50Now you want to
154. 6:52probability of B. So probability of B
155. 6:55would become this.
156. 6:59X2 {comma} X3 {dot} {dot} {dot} {comma}
157. 7:03XN {comma}
158. 7:06CK.
159. 7:07I guess it's not so much
160. 7:08to get.
161. 7:09Up a counter.
162. 7:11This
163. 7:12is not going to be too much let's say
164. 7:14it's going to be cool.
165. 7:16You get more than let's call it A.
166. 7:19Okay, so you can actually write.
167. 7:22Probability.
168. 7:24Of. CK. Given X is equal to A multiplied
169. 7:31by.
170. 7:32Probability of X2.
171. 7:35X3.
172. 7:37XN.
173. 7:38CK.
174. 7:40Right.
175. 7:43Up cool social is done cool cool.
176. 7:47Probability of
177. 7:49X1, sorry, not one. Two is the start of
178. 7:52the year. X3
179. 7:54XN {comma} CK.
180. 7:58Don't you think you have to be the same
181. 7:59assumption applied to the system? What I
182. 8:01can do is I can assume this thing to be
183. 8:04equal to A and this entire thing
184. 8:07to be equal to
185. 8:09B.
186. 8:10Or you have to first see the conditional
187. 8:11probability apply to the rule.
188. 8:13So, I can write this again as So, I make
189. 8:16it equal to the conditional probability.
190. 8:17So, I will write equal to
191. 8:19A, which is this big term
192. 8:22multiplied by I will write probability
193. 8:25of
194. 8:27X2
195. 8:28given
196. 8:30X3 {comma} X4
197. 8:33XN {comma} CK.
198. 8:36Or you have to write down the
199. 8:37probability of you know B. So, we will
200. 8:39write
201. 8:41X3
202. 8:43X4
203. 8:44XN {comma}
204. 8:47CK.
205. 8:48Right? I don't know what you think this
206. 8:49term will become. I can say something.
207. 8:51Let's call it B. So, you put all these
208. 8:53together and you get A.
209. 8:54AB
210. 8:56probability of
211. 8:58X3 X4
212. 9:01XN {comma} CK.
213. 9:10Um
214. 9:12chain
215. 9:14rule
216. 9:16for conditional probability.
217. 9:18A term here mathematical
218. 9:21to be honest, machine learning with my
219. 9:22company I get it down. But you have to
220. 9:24keep learning and understand what I
221. 9:26mean. So, this is known as chain rule
222. 9:28for conditional probability. I will
223. 9:29probably break it down. Okay? So, don't
224. 9:31you think there will be a time when
225. 9:33eventually you will get it down? So, you
226. 9:35have to get it down.
227. 9:37So, there will be this term multiplied
228. 9:38by this term and then I will say will be
229. 9:40this
230. 9:40multiplied by this term
231. 9:48probability of
232. 9:50X N given C K
233. 9:54or B K C K
234. 9:59C K
235. 10:02X C
236. 10:03X
237. 10:04X N C K B and I got X N C A 1 J X N B 1
238. 10:08J C
239. 10:10This will become This is This will
240. 10:12become the last A and this will become
241. 10:16the last B
242. 10:18So, many of you last minute condition
243. 10:19probability that I got
244. 10:21So, and then
245. 10:24space for a comment, but I'll try. So,
246. 10:29It's a busy day.
247. 10:31So, this entire term will be simplified
248. 10:32as this. P of X 1
249. 10:35given
250. 10:36X 2 {comma} X 3
251. 10:39X N
252. 10:41{comma} C K
253. 10:43multiplied by
254. 10:46P of
255. 10:48X 2
256. 10:50given X 3 X 4
257. 10:54X N C K
258. 10:57multiplied by
259. 10:59P of X 3
260. 11:01X 4
261. 11:03X 5
262. 11:04C K or something
263. 11:09P of X N minus 1
264. 11:13X N {comma} C K
265. 11:18probability of X N given C K or last
266. 11:23minute probability of
267. 11:25C K So, this will be the entire term.
268. 11:28Okay. So,
269. 11:31but
270. 11:34If you you'll be able to understand
271. 11:36Now comes this part. John
272. 11:39naive assumption
273. 11:42simple assumption
274. 11:44So
275. 11:45Yeah
276. 11:46assumption
277. 11:47Yeah
278. 11:50Yeah
279. 11:52last terms problem
280. 11:55I'm
281. 11:55terms assumption
282. 11:57So assumption be this
283. 12:04So
284. 12:05This is like this. Imagine key
285. 12:09So
286. 12:10So
287. 12:11Let's say.
288. 12:15same problem
289. 12:17toss
290. 12:19Mumbai
291. 12:21or sunny
292. 12:24Okay. So Yeah
293. 12:27probability of
294. 12:30a toss being lost given
295. 12:35venue is Mumbai
296. 12:37a
297. 12:38outlook is sunny
298. 12:41or match
299. 12:42They
300. 12:43Similarly
301. 12:44Yeah So
302. 12:47probability of Mumbai given outlook
303. 12:51sunny or
304. 12:52match you got Okay. I
305. 12:55last notice
306. 12:56is probabilities about coming Yeah zero
307. 13:01You
308. 13:02probability about Okay. So what do we
309. 13:05take an assumption.
310. 13:06Okay. And that assumption is this. Key
311. 13:10Yeah
312. 13:11X3 Q depend
313. 13:16Yeah X1
314. 13:17X3 depend
315. 13:19Yeah
316. 13:20depend
317. 13:22Similarly
318. 13:23X2 will not depend X3 X4
319. 13:26It will only CK.
320. 13:29This n minus will not depend on Xn. It
321. 13:32will only depend on CK. This is the
322. 13:34assumption.
323. 13:40We call this assumption,
324. 13:42if I remember it correctly, it's called
325. 13:44conditional
326. 13:49conditional
327. 13:51independence.
328. 13:56Independent events, so independent
329. 13:58events, if you remember,
330. 14:03If you condition here, P of A given B is
331. 14:06equal to P of A, then these events are
332. 14:08known as
333. 14:09independent events. Conditional
334. 14:11independence here is given by the term
335. 14:13here.
336. 14:17This is comma, which means intersection.
337. 14:22You have conditionally
338. 14:24A
339. 14:25B C independent. So, we can write it
340. 14:27like this, P
341. 14:29A
342. 14:32C.
343. 14:40So, we are going to
344. 14:41we are going to
345. 14:43simplify this entire equation and we'll
346. 14:46write this. Now,
347. 14:47so this entire thing will become
348. 14:51this.
349. 14:52Probability of
350. 14:54X1
351. 14:56given CK
352. 14:58probability of X2
353. 15:01given CK probability of X3 given CK dot
354. 15:06dot dot probability of Xn given CK
355. 15:11multiplied by probability of
356. 15:13CK.
357. 15:14And if you remember, in the last video
358. 15:16we did that.
359. 15:17We did the probability of
360. 15:20toss given one, probability of windy
361. 15:22given one, probability of outlook given
362. 15:24one. Okay, I'll multiply
363. 15:27probability of winning ticket. So, yeah,
364. 15:29final formula ticket. Or this may humble
365. 15:33is called simplify
366. 15:35mathematically. We can write this down
367. 15:37as so I'm going to calculate you got to
368. 15:39get them.
369. 15:41Right, you got to get them.
370. 15:42So, we'll write it like this.
371. 15:44Probability of
372. 15:47CK
373. 15:49given a particular X is equal to
374. 15:53probability of
375. 15:56CK
376. 15:59product of
377. 16:01I is equal to
378. 16:03one say and up probability of
379. 16:07X of I
380. 16:09CK. Ticket. Summation means product is
381. 16:13going to inside the
382. 16:14product multiplication
383. 16:16ticket. So, this is the final formula.
384. 16:21Ticket.
385. 16:22So,
386. 16:23uh this is the final formula.
387. 16:25I was a classy
388. 16:27or we get it
389. 16:28to final decision making that
390. 16:30by the way
391. 16:32uh
392. 16:33I'll just write it down over here so
393. 16:35that clarify that.
394. 16:36So,
395. 16:38uh yeah.
396. 16:39This is the final formula that we have
397. 16:41established or derived.
398. 16:43Uh
399. 16:43I'll write it down again. So, this is
400. 16:45the final formula. Probability of
401. 16:48CK given
402. 16:50X is equal to
403. 16:54probability of CK
404. 16:59product
405. 17:00I one to N
406. 17:03probability of X of I given CK, right?
407. 17:08What I wanted to tell you is
408. 17:10yeah, actually
409. 17:12you will
410. 17:13be remember from the denominator
411. 17:16consider
412. 17:17so I will equal to
413. 17:19we have to do it
414. 17:47You actually use
415. 17:49arc max
416. 17:52for K in the range of 1 {comma} 2
417. 17:55{comma}
418. 17:56small K
419. 17:57and
420. 17:58it is
421. 18:00P of
422. 18:01C K
423. 18:03product of
424. 18:05I is equal to 1 to N
425. 18:08or probability of
426. 18:10X of I C of K
427. 18:27maximum
428. 18:32a posteriori rule
429. 18:37Yeah, map people this cool.
430. 18:40maximum a posteriori rule
431. 18:43Okay. So basically
432. 18:53I don't know if you know but I would
433. 18:55recommend
434. 18:58or
435. 18:59surely
436. 19:04So thank you for watching.