Data Repositories : Overview Data Warehouses and Big Data. — Transcript

Full transcript

1. 0:00welcome to another day of accountability
2. 0:02where I talk about what I've learned in
3. 0:03terms of data analysis today I'd like to
4. 0:06talk about data
5. 0:08repositories what are data
6. 0:11repositories data repositories are
7. 0:15basically data that has been collected
8. 0:17organized and
9. 0:19isolated
10. 0:20for various
11. 0:23uses which could
12. 0:25include business intelligence data
13. 0:28analysis or you could just use it as an
14. 0:30archive
15. 0:33for your
16. 0:35data and there're different types of
17. 0:38data repositories whether it's your
18. 0:41normal databases which are relational
19. 0:43non-relational databases we have your
20. 0:46data Lakes we have your data warehouses
21. 0:50we have big data we have data pipelines
22. 0:53I believe the list goes on but
23. 0:54nonetheless today I'd like to talk about
24. 0:56data warehouses as well as big
25. 1:01data data warehouses are or is a large
26. 1:05accumulation right of a company's
27. 1:08data that'll be used
28. 1:12to help make business
29. 1:15decisions or for any other use case
30. 1:19right
31. 1:20and this can be gathered from different
32. 1:23data sources whether it's from your herb
33. 1:27system your sales system
34. 1:30or just your transactions anything
35. 1:34really that is data right you can get it
36. 1:38from all these different data sources
37. 1:40there's a process to towards attaining
38. 1:44this data right so let's just say you
39. 1:46have your data warehouse right here and
40. 1:49then you have your CRM your herb as well
41. 1:52as your online transactions
42. 1:55right all of these things get filtered
43. 1:58through right these are data sources all
44. 2:01of these things get filtered through you
45. 2:03collect it the process that's used in
46. 2:06order to collect this data is e which is
47. 2:09for extraction T which is for transform
48. 2:13and then L which is for load right so
49. 2:16you're extracting the data I believe
50. 2:19this is where R comes in or
51. 2:23Python and
52. 2:25then t comes in which is the
53. 2:27transformation which I believe is the
54. 2:28cleaning and the standing ization of the
55. 2:31data in order to make sure that it is
56. 2:34structured
57. 2:35and there's a specific format right and
58. 2:38then L is to load which is where you
59. 2:40load it into this
60. 2:43Warehouse hence what are you going to do
61. 2:46with this data warehouse well you could
62. 2:48just store it alternatively you could
63. 2:51use this for analysis you could also use
64. 2:54this for reports or you could use this
65. 2:57for any other reason
66. 3:01imaginable so there are governing
67. 3:04factors when it comes to the data that
68. 3:07you'll collect in order to store into a
69. 3:09database such as
70. 3:12latency types of data the structure of
71. 3:15the data the intended use of
72. 3:18data the transaction speed as well as
73. 3:22how it is that you're going to query the
74. 3:25data so those are things that should be
75. 3:28considered when it comes to
76. 3:31creating your database if it's one thing
77. 3:32that I've noticed right throughout my
78. 3:35little journey of Entrepreneurship is
79. 3:40that WhatsApp used to be a type of
80. 3:42database right especially when you've
81. 3:44accumulated a lot of numbers and you get
82. 3:47status views and you get returned
83. 3:50customers you get new customers and so
84. 3:52on so forth one thing
85. 3:55that I'm glad I know now especially
86. 3:58going forward is
87. 4:02that it would have been very valuable
88. 4:05right to actually export all the
89. 4:07contacts and one of my best friends Phil
90. 4:09actually mentioned that
91. 4:12hey we should store these WhatsApp
92. 4:14numbers these are a database this is
93. 4:17proof of concept right and yeah now that
94. 4:21I'm reading about this and I'm learning
95. 4:23about data analysis it actually goes to
96. 4:25show that you actually can get data from
97. 4:30social media you know WhatsApp is
98. 4:33actually a data source that could have
99. 4:34been used especially when it comes to
100. 4:36Facebook advertising getting those
101. 4:39people who would actually contact us
102. 4:41concerning that we could have just just
103. 4:44filtered that into a
104. 4:45database and known that okay cool then
105. 4:48this is how we're going to Define it we
106. 4:49have the numbers we have whether or not
107. 4:52this person just contacted us and
108. 4:54obviously we sent them the message but
109. 4:55they never responded so that could have
110. 4:57been a cold cold lead and then moving on
111. 5:01to per say besides for the cold lead
112. 5:04then their warm leads people are like
113. 5:06yeah we're going to buy and then the
114. 5:07people that are hot on the mark you know
115. 5:10that could have actually helped us in
116. 5:12terms
117. 5:13of not just analysis but
118. 5:17rather approaching as to how we can
119. 5:19Market or what it is that is valuable
120. 5:23and I feel like if a lot of people would
121. 5:25actually consider that right
122. 5:30as solo tropers
123. 5:35as Soul Proprietors they could actually
124. 5:39go get a lot further if they would to
125. 5:42take their database
126. 5:43seriously and their analytics I guess
127. 5:46but that's a personal Journey
128. 5:47nonetheless that's neither here nor
129. 5:49there the next
130. 5:51data repository that I'd like to talk
131. 5:54about is Big Data right the idea that I
132. 5:58get from Big Data reminds me of cloud
133. 6:01computing right where
134. 6:03basically because there's just so
135. 6:06many resources that a big company has
136. 6:10such as
137. 6:12Amazon they offer those resources to
138. 6:16people because obviously I'm going to
139. 6:18use this for my virtual machine I don't
140. 6:21need to buy a machine that'll be fully
141. 6:24dedicated to that I'm going to use this
142. 6:26less time less space less money was did
143. 6:30I'm just going to use it for this
144. 6:31specific purpose Big Data just gives me
145. 6:35that feeling of use case even though I
146. 6:39might be wrong so the definition is
147. 6:41distributed computational and storage
148. 6:44infrastructure to store scale and
149. 6:48process very large data sets I'm not
150. 6:51sure if big query is this but then one
151. 6:56thing that I get from Big Data right
152. 6:58besides for the cloud
153. 7:01computing perspective that I understand
154. 7:04from it is
155. 7:06that
156. 7:08big data is valuable in terms
157. 7:12of which I learned from the Google
158. 7:14analytics course is valuable in terms of
159. 7:17viewing things in a larger picture and a
160. 7:21larger sense of data that you might
161. 7:25not have because in this little
162. 7:28microcosm
163. 7:31of running anme or even just normal
164. 7:35businesses there's a large microcosm of
165. 7:38everything and how everything goes
166. 7:41especially when you consider Google and
167. 7:44yeah hey and the amount of queries that
168. 7:46are there which is basically people
169. 7:48trying to access the database and you
170. 7:52know perform what it is that they need
171. 7:54to do or analyze what it is that they
172. 7:56need to do gain insight to report what
173. 7:59it is
174. 8:00that they need to report yeah that's
175. 8:03actually
176. 8:04pretty amazing that we could possibly
177. 8:07gain access to that or that we can gain
178. 8:09access to that I just need to check that
179. 8:11out but nonetheless those are the two
180. 8:13types of data sources that I wanted to
181. 8:15talk about we'll talk about relational
182. 8:17databases as well as data legs data
183. 8:20pipelines and so on so forth but besides
184. 8:23for that thank you so much for watching
185. 8:25this video peace