Markov Chaining (to generate Fake Alex Jones) — Transcript

Full transcript

1. 0:00all right welcome back everyone uh can i
2. 0:02treat it as a 2d array we can't treat it
3. 0:04like a vector
4. 0:05uh yeah so if you have an end pointer an
5. 0:08endpointer is simply a memory address
6. 0:10nothing more nothing less is it a 1d
7. 0:13array is it a 2d array who knows who
8. 0:15cares it's up to you to figure that that
9. 0:17stuff out
10. 0:18so if it's a 2d array that means it's
11. 0:20going to be laid out in some format you
12. 0:22know it's like you're going to have the
13. 0:23first row first followed by the second
14. 0:25row of all with it
15. 0:27but
16. 0:28it's kind of up to you right like
17. 0:30especially when you're processing image
18. 0:31data
19. 0:32image data a lot of times can be held
20. 0:35with a complete plane of red data
21. 0:37and then a complete plane of green and
22. 0:39then a complete plane of blue or you can
23. 0:41do what's called interleaved or
24. 0:42interlaced or something like that redo
25. 0:44the red the green the blue for pixel at
26. 0:47row zero column zero red green blue of
27. 0:50row zero column one red green blue row
28. 0:52zero column two so the format the layout
29. 0:55of where the data is when you have a
30. 0:56pointer like it doesn't contain any of
31. 0:58that information it's just where is the
32. 1:01spotted memory there's a spot in ram
33. 1:02right here and that's where the image
34. 1:04begins you know nothing else beyond that
35. 1:06so if you want to pass in
36. 1:08additional information like how big is a
37. 1:10row how big is a column then you pass
38. 1:13those in as separate parameters and
39. 1:14that's why the
40. 1:15code has a width and a height parameter
41. 1:18so it tells you how the
42. 1:20uh the thing is formatted the rows tell
43. 1:22you you know how long it goes until you
44. 1:24get down to the next row and or the
45. 1:27columns rather tell you how far
46. 1:30you go until you're down to the next row
47. 1:32and then the rows tell you how far it is
48. 1:33till the end of the image or the
49. 1:35into the elevation field
50. 1:37so i prefer using vector vectors because
51. 1:40it contains all that information
52. 1:42we may be getting matrices in c plus
53. 1:44plus 23.
54. 1:46don't hold your breath
55. 1:47um
56. 1:48the standards moves
57. 1:50not quickly but there is currently some
58. 1:54um
59. 1:57some indication we might get
60. 1:59matrices in
61. 2:02in 23
62. 2:03um
63. 2:05coin toss whether or not that actually
64. 2:06happens but that'd be nice because it's
65. 2:08like these kinds of like
66. 2:10things like come up you know like a 2d
67. 2:13matrix wow that's
68. 2:14it's crazy somebody would want that in
69. 2:16the standard
70. 2:21what kind of obscure use case do you
71. 2:22have for a 2d matrix man
72. 2:26it'd also be cool if you added in
73. 2:28support for numerical vectors cross
74. 2:30product dot product so that everybody
75. 2:32and their mother doesn't have to write
76. 2:34their own matrix library and their own
77. 2:35vertex library none of which are
78. 2:37compatible with each other that'd be
79. 2:38kind of cool too wouldn't it i don't
80. 2:40know
81. 2:41and then you could just multiply a
82. 2:43vector in a matrix and
83. 2:45a point in a
84. 2:47yeah okay
85. 2:48so our next topic is going to be
86. 2:53markov chains
87. 2:56markov
88. 2:58chain
89. 3:00so markov chain
90. 3:02is
91. 3:04pretty cool pretty cool stuff um
92. 3:08basically um
93. 3:10you can imagine it's a graph okay
94. 3:13stream oh i'm not streaming
95. 3:17there so
96. 3:19uh
97. 3:20markov
98. 3:21chains are graphs
99. 3:24and the edges indicate the probability
100. 3:27the transition from one graph to another
101. 3:30and so all the outgoing probabilities
102. 3:33must always
103. 3:35add up to one
104. 3:36right because one is 100
105. 3:39so if we're on the e node right here
106. 3:42we roll it i
107. 3:44inside a die if we get a
108. 3:46one two or three
109. 3:48we stay at e we print e again
110. 3:50and if we
111. 3:52get a four through ten
112. 3:54we print a instead and then we're at
113. 3:56eight
114. 3:57and then we roll another ten sided die
115. 3:58if we get let's say two we transition
116. 4:00back to e we roll a two we transition
117. 4:02back to e
118. 4:03roll five transition to a roll of five
119. 4:05we transition to a
120. 4:07and so this will randomly generate
121. 4:09strings of just e's and a's
122. 4:12not very useful
123. 4:14what is useful
124. 4:15is when you take
125. 4:18the english language
126. 4:20and feed it
127. 4:22into building a markov chain graph
128. 4:25okay
129. 4:27so for example if you wanted to make a
130. 4:30program that would generate
131. 4:31random names
132. 4:33for let's say a specific
133. 4:35country
134. 4:36what you could do is
135. 4:38grab census data for that country
136. 4:41and
137. 4:43grab just the first names let's say and
138. 4:45feed them into this
139. 4:46markov
140. 4:47chain generator and what you basically
141. 4:49do is you say all right
142. 4:53for every letter you read
143. 4:56what is the probability that it moves to
144. 4:58one of the other letters
145. 5:00and so there's 26 letters in the english
146. 5:02language so if i'm if i'm feeding in all
147. 5:04the names of
148. 5:06babies
149. 5:08um
150. 5:09you know
151. 5:10how often does the letter i
152. 5:12follow the letter b and you just count
153. 5:14you just count how many times each
154. 5:16letter follows another letter so you
155. 5:17have a 26 by 26 matrix
156. 5:20and it's
157. 5:21you know you just count every time a
158. 5:23letter follows another letter you count
159. 5:25and
160. 5:25then you have two special letters one is
161. 5:27start of word
162. 5:29and one is end of word and so
163. 5:32you can count how often a certain letter
164. 5:34starts a word for example you
165. 5:36probably does not start a word in the
166. 5:39english language very often
167. 5:41but like j
168. 5:42would even though j is not a very common
169. 5:44letter in the english
170. 5:45language overall
171. 5:47there's a lot of j names
172. 5:49so what you do then when you want to
173. 5:51generate a random
174. 5:52a random name is you start at the
175. 5:56it's a graph that you've essentially
176. 5:57created
177. 5:58um you normalize all those values down
178. 6:00to
179. 6:01you know
180. 6:02zero to one
181. 6:04and you basically start at the start
182. 6:06node start graph start vertex
183. 6:09and you roll die and
184. 6:11then you see okay uh i rolled a j okay
185. 6:14then you move to the j spot on the graph
186. 6:16and then j could be followed by o or u
187. 6:19or i or
188. 6:21um
189. 6:23a for james you know something like that
190. 6:25and you roll die you're like okay you
191. 6:26got an a so you move to a and you see
192. 6:28what what letters follow
193. 6:30a you know very often you roll and you
194. 6:32get a s
195. 6:34you know and so you sit there you move
196. 6:36around and the end word is one of the
197. 6:38possible moves you can make and
198. 6:40eventually it'll move around and
199. 6:41eventually it'll move on top of the end
200. 6:43node and then you stop and there's your
201. 6:45random there's your random name
202. 6:47so markov chains are great for learning
203. 6:49patterns of data and then you can
204. 6:51randomly generate new names based on the
205. 6:54patterns of the language or the country
206. 6:56or the county or the city that you're
207. 6:59trying to emulate
208. 7:02just pretty cool stuff
209. 7:04yeah
210. 7:05so i have a program on
211. 7:07the server called mc
212. 7:10andre because i think
213. 7:15yeah andrea is his name
214. 7:17so uh i have a program on my
215. 7:20server called mc andre that takes in rap
216. 7:23lyrics
217. 7:24and it and it randomly generates rap
218. 7:26lyrics and it has some additional
219. 7:28constraints on it to make sure that the
220. 7:31um
221. 7:32versus rhyme and things like that like
222. 7:34it actually
223. 7:35looks at the random
224. 7:37data coming out and rejects those that
225. 7:39don't sort of follow the
226. 7:41like if you're trying to
227. 7:42go with a certain you know beat or
228. 7:44number of syllables and rhyme pattern
229. 7:46and things like that
230. 7:47then it will reject
231. 7:49um
232. 7:50verses that don't follow that pattern
233. 7:52but the uh the upshot is that it can it
234. 7:55can randomly spit out you know okay at
235. 7:58least sounding wrap wrap lines
236. 8:02you should take note of the smart off
237. 8:03chain counts if you should because it's
238. 8:04your next homework assignment you're
239. 8:05gonna have to do this
240. 8:09okay
241. 8:10so
242. 8:12uh more graph theory yay
243. 8:16graph theory is fun a lot of uses okay
244. 8:19so let's see if we can have a picture
245. 8:21here
246. 8:47yeah like something like that okay
247. 8:50so let's say that you notice how the
248. 8:51outgoing probabilities always add up to
249. 8:53one right
250. 8:54so let's say you start at one
251. 8:57and so you roll a hundred sided die
252. 8:59and you get a 72
253. 9:01okay so
254. 9:03if it's if you got a zero to one to 23
255. 9:05you'd go to two but we didn't so we're
256. 9:07gonna go to four
257. 9:08so you output one you go to four you
258. 9:10roll a die
259. 9:12and i got a thirty okay so then i move
260. 9:15to five then i roll another thirty well
261. 9:17a hundred percent of the time
262. 9:19i move to six so it's going to print out
263. 9:20one four five six
264. 9:22do you see how this works it and it'll
265. 9:24just keep moving around
266. 9:25the system at every point you roll a die
267. 9:28and that randomly chooses
268. 9:30which direction you're gonna go in right
269. 9:32so if it's raining today
270. 9:34there's a 60 chance it'll rain tomorrow
271. 9:36it's a 10 chance it'll be sunny and a 30
272. 9:38chance it'll be cloudy so if you want to
273. 9:40generate fake weather forecasts for a
274. 9:42video game let's say
275. 9:44then um you know you're playing harvest
276. 9:46moon or something you probably want to
277. 9:47start the game in sunny because it's a
278. 9:49little weird to start again with it like
279. 9:51thunderstorming right
280. 9:53but it'll stay sunny probably for a
281. 9:55while and then oh it switches into
282. 9:57cloudy and then cloudy switches into
283. 9:59rain then it stays raining for a while
284. 10:01then it switches back into cloudy and
285. 10:02then switches back into raining and
286. 10:04switches into sunny
287. 10:06and so you can use this to generate
288. 10:07weather forecasts you know that so
289. 10:09rather than it going like sunny rainy
290. 10:11sunny rainy it's kind of weird you know
291. 10:13maybe
292. 10:14not too weird for fresno giving her our
293. 10:16weather the last couple of days
294. 10:19uh it was super windy yesterday now it's
295. 10:21just like
296. 10:22dead
297. 10:22right i was like all excited i got my
298. 10:24kites you know ready
299. 10:30i was gonna go to woodward park with my
300. 10:31kites and there was too much dust there
301. 10:33was so much dust flying up i was like
302. 10:36all right it's too windy for kite flying
303. 10:37i'll go tomorrow now it's there's like
304. 10:40it's completely dead
305. 10:41so
306. 10:44um
307. 10:48so this makes sense to you
308. 10:50and so you can use these things to
309. 10:51generate you know fairly realistic
310. 10:53looking and feeling
311. 10:56um
312. 10:58that's interesting
313. 11:01yeah that weather and
314. 11:03words and things like that so
315. 11:06well they're probably home yesterday now
316. 11:07you're back in front of where where is
317. 11:08home cohen
318. 11:17west coast washington yeah
319. 11:19yeah probably a little more dust storm
320. 11:21than usual
321. 11:23um
322. 11:24okay so let's talk about bridges markov
323. 11:28so bridges markov
324. 11:37we're going to be doing
325. 11:39we're not going to be doing word
326. 11:41generation we're going to be doing
327. 11:43sentence generation so what we're going
328. 11:45to do is we're going to feed in a corpus
329. 11:49a body of
330. 11:50work
331. 11:51and what we're going to do is write down
332. 11:52how often each word
333. 11:54follows another word
334. 11:56okay
335. 11:58so
336. 12:00for example i put in 1500 lines of
337. 12:03wu-tang clan lyrics
338. 12:05and then
339. 12:07once you build it
340. 12:09it it's basically just you know
341. 12:12kind of a hash table right you have a
342. 12:14hash table rehash based on the word
343. 12:17and then each each word has a
344. 12:20a vector
345. 12:21of possible words that it connects to
346. 12:23and how often those words happen
347. 12:26and then once you build the graph phase
348. 12:27one is building the graph then phase two
349. 12:31is generating new sentences
350. 12:33and you generate a new sentence by just
351. 12:35picking a word again that connects to
352. 12:37the start node
353. 12:39and then you roll a die you're like okay
354. 12:41well this one's 75 chance i rolled a 50
355. 12:44i'm gonna go there and you sit there and
356. 12:46you hop around on this graph until you
357. 12:48get to the end note and you stop
358. 12:51so for example um
359. 12:53generating random i've had 1500 lines of
360. 12:55wu-tangler it's my favorite wu-tang
361. 12:57wraps not all of them just my favorite
362. 12:59ones i put into this
363. 13:01beast of a
364. 13:03input fed it into the markov thing it
365. 13:06generated a graph and then i had it just
366. 13:08sit there spitting out wu-tang lyrics
367. 13:11and so i've got my glock like voltron
368. 13:13which i think is
369. 13:14like
370. 13:17it's
371. 13:18pretty good pretty aligned i don't know
372. 13:20i'm stealing my glock like voltron it's
373. 13:22the name of your new rap band
374. 13:29that's a bar my clock like voltron bring
375. 13:32to mother bring to mother bring it like
376. 13:33it was kicking rhymes like your meth
377. 13:35hidden caps on the gizza
378. 13:37diluted broken down you ruined smokey
379. 13:39bear when you don't need that wu-tang
380. 13:40style like it's actually like not all of
381. 13:42it's good like some of it's just kind of
382. 13:43like
383. 13:44like you know somebody's having a stroke
384. 13:46but like some of these are like actually
385. 13:49like pretty good lines you know
386. 13:51chopping through your problems the gods
387. 13:52are hotter than heavy metal frame like
388. 13:54you know
389. 13:56and if you uh
390. 13:58are a fan of the wu-tang clan you
391. 13:59probably recognize the wraps that some
392. 14:00of these things come from right
393. 14:03stop quick hold this so grab your whole
394. 14:05era like i don't know i don't know what
395. 14:06that means but it's it's provocative so
396. 14:10uh
397. 14:11testing on the declaration of
398. 14:12independence
399. 14:13i got this sentence he has endeavored to
400. 14:15the lives our separation and of english
401. 14:17laws of our trade with his assent to our
402. 14:19constitution and hold them to dissolve
403. 14:21the supreme judge of immediate and
404. 14:22payment of constant guanudi we therefore
405. 14:25the most wholesome and magnificent the
406. 14:26people in accordance with deriving their
407. 14:28country to the conditions of the right
408. 14:29yeah
409. 14:30the declaration of independence has some
410. 14:32long-ass sentences in it and so um
411. 14:35the odds of going to the end node
412. 14:37weren't aren't very high and so you tend
413. 14:39to get these very long run-on sentences
414. 14:41which actually do kind of
415. 14:42represent the declaration of
416. 14:44independence to attend to the
417. 14:45inhabitants of fatiguing them of an
418. 14:46inheritable jurisdiction foreign
419. 14:48mercenaries to all experience has shown
420. 14:50that governments
421. 14:53right but i mean this is
422. 14:55a random generated thing but it sounds
423. 14:57like
424. 14:58the dui you know what i mean so
425. 15:02um
426. 15:03rap music's your favorite
427. 15:06uh load data from disk three points
428. 15:09so uh
429. 15:10the this one is auto graded i believe
430. 15:13and so
431. 15:14um
432. 15:17when you run it you can just print out
433. 15:18your your graph
434. 15:20and quit and so three of the test cases
435. 15:22are based on that so
436. 15:24you
437. 15:25load the corpus
438. 15:27you make a graph based on it and then if
439. 15:29the user hits one it just prints it and
440. 15:31then it quits
441. 15:33three points
442. 15:35part two generate random sentences you
443. 15:37must use my random number generator you
444. 15:39cannot use your own because it's going
445. 15:41to be auto-grading you based on
446. 15:44the the random roles you get have to be
447. 15:46the same random roles i get
448. 15:49and then visualizing it on bridges is
449. 15:51three
450. 15:52points okay
451. 15:54so how do you load
452. 15:57how do you load the data let's first of
453. 15:59all take a look at this
454. 16:01m4
455. 16:03so this is hamlet's uh
456. 16:05the soliloquy from was it act two or act
457. 16:08three
458. 16:16it's low queen act two
459. 16:19uh no it's not it's
460. 16:34so to be here not to be that is a
461. 16:36question whether it is number in the
462. 16:37mind service things in the narratives of
463. 16:38our ages 4 channel take arms against
464. 16:40troubles by
465. 16:41opposing them to die to sleep to sleep
466. 16:43to dream if i sleep
467. 16:46tonight to sleep no more and if i sleep
468. 16:47say in the heart i can
469. 16:49so
470. 16:50that's the whole thing
471. 16:52and so
472. 16:53we've got
473. 16:55you can feed this in and then what
474. 16:56you're gonna do
475. 16:58is every single word here thousand and
476. 17:01life and love and after all these words
477. 17:05are going to go into a hashtag
478. 17:07and that hash table
479. 17:09because it's very easy to hash into it's
480. 17:11very easy to find the record for after
481. 17:14let's say so you go there
482. 17:16and it's going to have just basically a
483. 17:19vector of all the other words that come
484. 17:20after it or a set
485. 17:22of all the words that come after it
486. 17:24and every time a word comes after it
487. 17:26like death
488. 17:27you put
489. 17:28death into the set so you you find after
490. 17:31you get it set
491. 17:33and you stuff death into it and if it's
492. 17:35already in there you increase its count
493. 17:37by one
494. 17:39okay
495. 17:40that's it so what it looks like is this
496. 17:42so if we come up here come into bridges
497. 17:46markov
498. 17:48archive dot reference
499. 17:53so
500. 17:55print graph
501. 17:57so here is um here is the graph okay so
502. 18:01every word in
503. 18:03every word in the hamlet soliloquy goes
504. 18:06in here and it's sorted by count
505. 18:08okay
506. 18:09so two
507. 18:10is the word that appears the most in the
508. 18:12act three soliloquy from hamlet
509. 18:15happens
510. 18:1715 times
511. 18:19and
512. 18:21it starts the sentence three times it
513. 18:23ends a sentence zero times
514. 18:25as
515. 18:26once a comma following it
516. 18:29and it has
517. 18:3015 words coming out of it and it has 15
518. 18:32words coming out of it because there's
519. 18:34no
520. 18:36you know
521. 18:37it doesn't end the sentence ever so
522. 18:38every time the word two kio appears in
523. 18:41the soliloquy
524. 18:42it has a word following it what words
525. 18:44following it well
526. 18:45be follows it three times
527. 18:47suffer
528. 18:49to suffer no more
529. 18:52by silence
530. 18:53in the heartache and the thousand
531. 18:55natural shocks the flesh is there too
532. 18:56does the consummation devalue to be
533. 18:58wished
534. 19:00to die to sleep to sleep for a chance to
535. 19:02dream hi there's the rub
536. 19:05um
537. 19:07to die to sleep to sleep for a chance of
538. 19:09dreams so sleep would come up twice you
539. 19:11hear that
540. 19:12to die
541. 19:13to sleep
542. 19:14to sleep
543. 19:16for a chance to drink so sleep follows
544. 19:18two
545. 19:19twice in that little sentence that i
546. 19:20said there
547. 19:22loading alex jones rant oh
548. 19:24could you find one for me i'll do it
549. 19:26right now
550. 19:27get get like a i need a lot of text
551. 19:29because markov chains work best when you
552. 19:31get like i need like
553. 19:33i need like a kilobyte of alex jones
554. 19:35ranting and i will put it in here and i
555. 19:37will make a brand new alex jones rant
556. 19:39that nobody's ever seen before
557. 19:41thank you
558. 19:42so you guys understand what's going on
559. 19:43here
560. 19:44so this is all just using standard
561. 19:46library stuff
562. 19:48oh
563. 19:49just gonna be loading from a file one
564. 19:51word at a time
565. 19:52every time you load a word
566. 19:54you add it to the hash table
567. 19:56and every time a word follows a word
568. 19:59you add to the count and so later on
569. 20:01when you do the random lyrics generation
570. 20:05then let's say we were on the the edge
571. 20:08the graph vertex for two we roll a die
572. 20:11how many
573. 20:12we roll a 15 sided die
574. 20:14if we get a one two or a three we're
575. 20:16gonna output b
576. 20:17if we get a four we output suffer if we
577. 20:19get a five we have to take get a six or
578. 20:22seven die
579. 20:24eight nine or ten sleep do you
580. 20:25understand
581. 20:26and then we're now on sleep okay so
582. 20:29let's say we
583. 20:30did sleep uh
584. 20:32where are you if you were asleep
585. 20:49i know that happens what is it sorted on
586. 20:52i guess it's just on the order that yeah
587. 20:54i guess it's just the order they appear
588. 20:56in the text okay
589. 20:58question whether it's not really it's
590. 21:00not sorted
591. 21:07[Music]
592. 21:10all right to sleep
593. 21:13to dream perchance to dream right and so
594. 21:16whenever you get to the sleep
595. 21:18vertex
596. 21:20there are
597. 21:22three
598. 21:23so sometimes it ends the sentence so
599. 21:25twice
600. 21:26when we got to sleep it ends the
601. 21:28sentence
602. 21:29and then three times it printed out a
603. 21:31word instead to sleep to dream to sleep
604. 21:34for chance to dream sorry
605. 21:36okay
606. 21:37and that's sleep of death what dreams
607. 21:38may come when yeah so in that sleep of
608. 21:43okay so when we get to the sleep edge we
609. 21:45roll a five-sided die
610. 21:48and if we get a one or two we end the
611. 21:50sentence
612. 21:51if we get a three we print out two we
613. 21:53get a four we print out per chance if we
614. 21:55got a five we print out up
615. 21:59okay
616. 22:00gives a consummation to value to be
617. 22:02wished right uh consummation
618. 22:04and so what this is doing it's learning
619. 22:06the patterns of shakespeare okay
620. 22:10so uh you're counting how many times
621. 22:13words are appearing in a sentence
622. 22:15that's part of it but what we're
623. 22:17counting is how often a word follows
624. 22:20another word
625. 22:21so if if
626. 22:22hamlet says
627. 22:24in his speech the word of
628. 22:27what word follows of
629. 22:31okay we're not just making a random
630. 22:33count of like
631. 22:35how common different words are because
632. 22:37then if you try generating sentences
633. 22:39using that you're just going to get
634. 22:41nonsense
635. 22:42uh you're going to get like fortune
636. 22:43perchance of uh the the
637. 22:47outrageous fortune of the the the
638. 22:50like you just get you get garbage with a
639. 22:52markov chain it's about what word
640. 22:54follows what word with what
641. 22:56proportionality
642. 22:58that makes sense
643. 23:01[Music]
644. 23:03wiki quote alex jones all right
645. 23:07he's only three years older than me that
646. 23:08dude looks like he's
647. 23:10wow okay
648. 23:12dang
649. 23:14like i mean let's be honest here i've
650. 23:16aged better than this guy like
651. 23:18can we can we agree on this
652. 23:22is that guy
653. 23:24all right that's all it has this
654. 23:26i don't think
655. 23:37okay that's that's that's pretty good
656. 23:39okay let's do this copy
657. 23:41come into here
658. 23:43and
659. 23:44then alex jones
660. 23:47there
661. 23:51get rid of this kind of stuff all the
662. 23:54header and footer stuff and get rid of
663. 23:58quotes about jones we don't need
664. 24:01these are all
665. 24:02you've chosen the path of pain
666. 24:05um
667. 24:08citations we don't need
668. 24:15um
669. 24:17the alex jones show
670. 24:18i think we need to delete
671. 24:20all of these lines the alex jones show
672. 24:24ashcraft how would you do that
673. 24:26how would you delete all lines in vim
674. 24:29that
675. 24:30have alex jones show on them
676. 24:37sashcrafter
677. 24:39oh you cut out everything okay all right
678. 24:41all right cool cool
679. 24:42that's
680. 24:44even
681. 24:46faster i think
682. 24:51uh it's empty dude
683. 24:54cut out too much
684. 24:55[Laughter]
685. 25:03when he said i cut out everything you
686. 25:05you were not you're definitely not lying
687. 25:06okay
688. 25:07you're continuing donations help keep no
689. 25:09get rid of that all right there's a
690. 25:11warfrog
691. 25:20so just need to get rid of all lines
692. 25:22that begin with the alex jones show so
693. 25:25let's search for beginning with the no
694. 25:28it's got indentation just search for the
695. 25:30alex
696. 25:32jones
697. 25:33show
698. 25:35and we'll go dd and next a bunch of
699. 25:38times
700. 25:40record a macro next dd
701. 25:43stop recording the macro
702. 25:46replay the macro
703. 25:48and
704. 25:49we'll just do this
705. 25:52[Music]
706. 25:55okay fast enough
707. 25:57okay oh
708. 25:58it seems like it didn't get this one the
709. 26:00alex jones show
710. 26:10next
711. 26:22oh over deleted
712. 26:32and then anything that has 2000 in it
713. 26:39okay
714. 26:40dd
715. 26:41and next
716. 27:39all right good enough
717. 27:41all right
718. 27:43yeah that was my bad with the empty file
719. 27:46it was kind of funny though i
720. 27:47got you a new one though
721. 27:50uh yeah
722. 27:51all right all right
723. 27:52all right you got it uh did you is it
724. 27:54from the same source is it from a
725. 27:55different source
726. 27:57it's on the same okay let's take a look
727. 27:58here
728. 28:04alex
729. 28:15uh
730. 28:17okay
731. 28:17cool
732. 28:21okay so let's uh print the graph for
733. 28:24alex jones
734. 28:35let's let's generate some random
735. 28:37alex jones
736. 28:39generate random lyrics how many
737. 28:41sentences should we make what do you
738. 28:42guys think
739. 28:44how many alex jones rants should we make
740. 28:47george soros or seven okay
741. 28:50there you go please enter the random
742. 28:52seed zero
743. 28:54okay you will you don't think you're
744. 28:56part of white men for you and a gimmick
745. 28:59you know folks got a love play a little
746. 29:01bit wild today excuse me i couldn't put
747. 29:02on for no and remember that as if they
748. 29:04want to get back there nobody effing
749. 29:06moment of cowardice is there not even
750. 29:07run your fault you understand that gives
751. 29:09me so sweet hitler what i'm angry
752. 29:14angry he was thinking in fact
753. 29:17wow
754. 29:20ireland the most ancient egyptian
755. 29:22rituals some of liberty is what i'm not
756. 29:25saying veterans are terrorists under
757. 29:26rocks they reckon beat these insane
758. 29:29people and i want to have youtube
759. 29:30banning videos of that you this is the
760. 29:33public had a bunch of white people just
761. 29:34choking ah you're not the joker
762. 29:36experienced
763. 29:40[Laughter]
764. 29:49that is amazing
765. 29:50that is amazing ah
766. 29:54uh you're not the chill we're gonna
767. 29:56experience up but my testosterone is
768. 29:58going to lose most of women across
769. 29:59europe didn't happen oh my gosh
770. 30:02oh my gosh doing this you're going to
771. 30:04perception want to kill the same room
772. 30:06with me you know your life and my spirit
773. 30:08is cool you i can unlock a 55 year old
774. 30:11uh ate a pile of myself
775. 30:14from two other people that point the gun
776. 30:16you know there's real culprit suicide
777. 30:18bills alec baldwin thinks they're not
778. 30:20being up and we're a gd to be proud of
779. 30:23anti-human trash
780. 30:24the reason you sons of it and afromother
781. 30:27do whatever you do this isn't periable
782. 30:29you're so royal we're such a devil you
783. 30:31know
784. 30:32we're all over you when never when never
785. 30:34never said that attitude when they have
786. 30:35you you're ah i want to lick the guns
787. 30:38how much didn't happen
788. 30:41parasites will return
789. 30:49[Music]
790. 30:54i'm sorry this is this is too amazing
791. 30:56generate random lyrics we'll do seven
792. 30:57more with a random seed of one this time
793. 31:00okay
794. 31:01uh
795. 31:03uh is this same one nah all right yeah
796. 31:05okay
797. 31:06and alex
798. 31:12oh
799. 31:12dang it uh infowars.com liberties
800. 31:15they're clearly used to
801. 31:17your stinging guts well i've talked to
802. 31:19get free will and staring at her yeah
803. 31:20they're going to break your mommy and
804. 31:22praising satan we're about family name
805. 31:24in fact that you think i'd roll up
806. 31:25killing me a mustache you were looking
807. 31:27up against your wall i've told them
808. 31:28they're
809. 31:29i'm not their organs we're going to act
810. 31:31like they're like this country over us
811. 31:34i mean how i mean give up in this whole
812. 31:36life 150 year old uh swam two miles away
813. 31:38and harvest their heads disappear them
814. 31:40take that they're tough enough and pants
815. 31:42at the troops these people often just
816. 31:43can't find the stuff that's what i do
817. 31:45and you want to catch him calling dana
818. 31:47lewis a car a real culprit suicide pills
819. 31:49mass murder africa invasion force this
820. 31:51because they want to have the smiling
821. 31:53learning devil the ultimate buffoons the
822. 31:54good knock then come for me
823. 32:02what do you guys think pretty uh pretty
824. 32:04accurate i don't know
825. 32:06i got i gotta stab this one
826. 32:15i i i'm gonna i'm gonna have to write
827. 32:17this one down yeah
828. 32:18that's that's just too funny
829. 32:42some of these are even you know like
830. 32:44okay
831. 32:47can we have a s a copy of it of what of
832. 32:50the
833. 32:53uh what do you want a copy of ashcraft
834. 32:56the paragraph
835. 33:02[Music]
836. 33:10is
837. 33:20[Music]
838. 33:24all right
839. 33:26there you go
840. 33:27so that's uh peak uh computer science
841. 33:30for today i think okay let's talk about
842. 33:32how to do this right
843. 33:35um
844. 33:40so if you look at name dot cc
845. 33:45um we're gonna be using hash tables
846. 33:47which is an unordered map all right
847. 33:51and uh let's see do we have a hash table
848. 33:53hash tables how do we do it no
849. 33:55just have a vertex of edge okay
850. 33:58so a vector of edges so
851. 34:02probably could be improved with a set
852. 34:04but
853. 34:06okay
854. 34:07so basically every edge represents the
855. 34:10word that you're connected to and the
856. 34:12count of how often
857. 34:13you
858. 34:14um
859. 34:15go in that direction so if you've got a
860. 34:18a bunch of words
861. 34:21and for the word
862. 34:25joe
863. 34:27if 99 percent of the time the word that
864. 34:29follows it is rogan
865. 34:31then
866. 34:32joe would be a vertex
867. 34:34all right
868. 34:35and
869. 34:36it would hold the word joe there
870. 34:38count is how many times the word joe
871. 34:40appears in the entire corpus
872. 34:42and then it's going to have edges
873. 34:45to all of the words that follow it
874. 34:48so for example there's holloway joe and
875. 34:51there's joe rogan experience
876. 34:53so like maybe 99 of the time alex jones
877. 34:56says rogan following the word joe but
878. 34:58maybe one time he says hall after joe
879. 35:01then
880. 35:02your vector of edges would have two
881. 35:05elements in it the first would be
882. 35:07an edge
883. 35:10holding rogan
884. 35:11with a weight of however many times the
885. 35:14word rogan follow joe
886. 35:16and then you would have a second element
887. 35:18in the vector holding the word
888. 35:21hall you know for like holloway joe the
889. 35:23sea shanty force
890. 35:24and
891. 35:26then that would have a weight of one
892. 35:28because it appeared one time after
893. 35:31after joe
894. 35:33you guys understand
895. 35:34so this is the data structure we're
896. 35:35going to be using
897. 35:37so we're just going to have a hash table
898. 35:40uh
899. 35:41right i've already that's cool i've
900. 35:43already got a double left error operator
901. 35:45done for you so you don't have to even
902. 35:46format all that stuff it just prints out
903. 35:49all that stuff for you that's cool
904. 35:52functions uppercase 5 functions to
905. 35:54lowercase if i functions to
906. 35:57capitalize just the first letter
907. 36:01so when you generate random sentences
908. 36:03the first letter in a sentence should be
909. 36:04capitalized and the rest should be
910. 36:07lowercase
911. 36:08but the the graph itself will have
912. 36:10everything in uppercase
913. 36:14um
914. 36:15i wrote a function to strip brackets
915. 36:17because
916. 36:20a lot of the wu-tang
917. 36:21lyrics
918. 36:24would say like bracket you know method
919. 36:27or whatever
920. 36:28odb
921. 36:30and so this this function here strips
922. 36:32everything within square brackets also i
923. 36:34think for maybe hamlet and stuff like
924. 36:35that
925. 36:36it'll have like hamlet in brackets and
926. 36:38so this just strips everything out in
927. 36:40brackets in the input corpus
928. 36:43okay um yeah so here we go so here is
929. 36:46here is the data structure the data
930. 36:47structure and this is why i now require
931. 36:5041 to take this class because this class
932. 36:52is built on the success of 41.
933. 36:56if you haven't taken 41 with me then
934. 36:58i can go over how these things work but
935. 37:01it's basically a hash table if you know
936. 37:03what a hashtag works
937. 37:09the key that we're searching for is the
938. 37:12word like joe
939. 37:14and then it's gonna the hash table is
940. 37:17just a
941. 37:18it hashes between
942. 37:20joe
943. 37:21and the index
944. 37:23of the
945. 37:24graph
946. 37:25so the hash table just as a quick look
947. 37:27up in the graph so that we don't have to
948. 37:28search through the whole graph every
949. 37:31time
950. 37:32instead
951. 37:33if joe is in index 50 in the graph we
952. 37:36say hey where's joe index 50 cool and
953. 37:38then we jump to index 50 in the graph
954. 37:41here and this is gonna this hash table
955. 37:42here just makes looking up
956. 37:46looking up entries in the graph easy and
957. 37:48it's a vector vertices and each vertex
958. 37:50has a vector of
959. 37:53edges coming out of it
960. 37:55and each edge holds
961. 37:57a word which and if you want to find out
962. 37:59where that edge leads to
963. 38:01that word joe or whatever you can hash
964. 38:04that and it's like oh well that one's at
965. 38:05this index here
966. 38:07rogan it's at that index there and so
967. 38:09you can quickly look up
968. 38:11where these things are
969. 38:13in the graph
970. 38:25so the code to print the graph is
971. 38:30done
972. 38:31so for every vertex in the graph it
973. 38:33prints it
974. 38:35uh but you need to do the code to load
975. 38:38it
976. 38:40so you have to load
977. 38:42you have to load the file
978. 38:46so you're gonna read through this this
979. 38:48file that holds all of the you know
980. 38:53hello world okay
981. 38:57so
982. 38:58it's gonna read two words and your graph
983. 39:01will have two nodes in it one is hello
984. 39:04one is world
985. 39:06hello is a start node world is an ending
986. 39:10node in other words the only transitions
987. 39:12out of world art ending
988. 39:15so you can use that to test your code
989. 39:19toad style is immensely strong and
990. 39:21immune to nearly every weapon when
991. 39:23properly used it's almost invincible
992. 39:25that's from five deadly poisons i think
993. 39:27also quoted by wu-tang clown
994. 39:30okay there's a couple really short
995. 39:32corpuses and again you should test your
996. 39:34code
997. 39:35right like make a little thing like
998. 39:38um
999. 39:39[Music]
1000. 39:44and then your
1001. 39:46your markov graph will say 100 of the
1002. 39:48time
1003. 39:49b will be followed by an a
1004. 39:52okay
1005. 39:53for a's
1006. 39:54one two
1007. 39:56three four times a is followed by
1008. 39:58another a
1009. 39:59one time a is followed by a b one time a
1010. 40:01is followed by a c
1011. 40:03a starts a sentence one time
1012. 40:06c ends a sentence one time and that's
1013. 40:08the graph okay
1014. 40:11you guys will have nine days to do this
1015. 40:13it's a fun
1016. 40:15fun project
1017. 40:16because you can take anything you want
1018. 40:18you can take the entire bee movie if you
1019. 40:20want
1020. 40:21and
1021. 40:22generate new new sentences brand new
1022. 40:24sentences never seen before by humans
1023. 40:27that are in the style of the b movie or
1024. 40:30you can take alex jones and generate
1025. 40:33brand new alex jones sentences or wu
1026. 40:35tanks you know
1027. 40:36lyrics and things like that
1028. 40:38i'll be strictly using hack political
1029. 40:40pundits yeah we need to get like a
1030. 40:43like one of those text-to-speech engines
1031. 40:44you know
1032. 40:45and just uh
1033. 40:47wonder if that exists
1034. 40:50we're basically out of time for today
1035. 40:51aren't we i know if there's like a
1036. 40:54alex jones text
1037. 40:57speech
1038. 40:58somebody done that there's gotta be
1039. 41:04vocal synthesis okay
1040. 41:06what the [ __ ] did you just talking said
1041. 41:08about me little [ __ ]
1042. 41:10i'll have you know aggregated jump of my
1043. 41:13class in the navy settles and i've been
1044. 41:15involved in numerous sacred raids on
1045. 41:17altiga and i have over 300 confirmed
1046. 41:20channels
1047. 41:21that's not quite
1048. 41:23too
1049. 41:24uh accurate but
1050. 41:26uh
1051. 41:28huh
1052. 41:29yeah maybe maybe i can get maybe get
1053. 41:31that sentence
1054. 41:34uh text-to-speech in
1055. 41:37his voice that'd be pretty fun
1056. 41:42google translated then just have it read
1057. 41:43it
1058. 41:45but it doesn't have a it doesn't have an
1059. 41:47alex jones voice you know
1060. 41:49okay
1061. 41:52okay any questions about the homework
1062. 41:55any questions about the dijkstra's
1063. 41:56assignment
1064. 41:57i think what i showed you was pretty
1065. 41:59much most of the
1066. 42:01thing and i did turn on recordings
1067. 42:03you're you're free to
1068. 42:05to dig into the youtube recording of it
1069. 42:07and
1070. 42:08fix up your code
1071. 42:11so i want to see everybody getting 20
1072. 42:13out of 20 on this okay
1073. 42:17a lot of people didn't show up today
1074. 42:18probably because the assignment was
1075. 42:21tough or something
1076. 42:22okay
1077. 42:24all right so that's it for today guys uh
1078. 42:25i'm gonna sit here and enjoy
1079. 42:32my alex jones generator
1080. 42:37oh here's the b movie
1081. 42:42kind of a bee community
1082. 42:45back to change have some lights on
1083. 42:46steroids
1084. 42:48what were with a human race can she's
1085. 42:50human girlfriend tea time snack
1086. 42:52garnishments
1087. 42:54bees are cut flowers and be a fellow
1088. 42:59that's jones
1089. 43:04oh my gosh
1090. 43:06if suddenly it starts the foreign war
1091. 43:08powers act you
1092. 43:18black uniforms grabbing a jellyfish
1093. 43:20slacker who were building fema camps in
1094. 43:22history and a swam to the other do oh
1095. 43:24what if he was called a lot of a loser
1096. 43:26you filthy vampire like 11 people all
1097. 43:28over again put a nast
1098. 43:29[Laughter]
1099. 43:35how do you set up bridges on a local
1100. 43:36machine um you have to go to the
1101. 43:38bridges.uncc
1102. 43:40i don't know it's bridges.github.io
1103. 43:43it's got to download there
1104. 43:50bridgesuncc.github.io yeah it's got the
1105. 43:52downloads
1106. 43:53on there
1107. 43:57and then uh
1108. 43:59yeah
1109. 44:01that's it
1110. 44:03so if that was fun for you all today
1111. 44:05it's fun for me
1112. 44:07so max flow we'll do another um
1113. 44:10we'll do another minimal spanning tree
1114. 44:13we'll do max flow and uh i'll test your
1115. 44:15knowledge on markov on the quiz today
1116. 44:17okay
1117. 44:18all right see you guys on thursday and i
1118. 44:20hope you all get your bridges done by
1119. 44:23then
1120. 44:24peace out