Data analysis with AI — Transcript

Full transcript

1. 0:02[music]
2. 0:11In our last lesson, we dealt with data
3. 0:13privacy and security. What you
4. 0:15absolutely need to protect and how to do
5. 0:17it. So, now let's talk about the
6. 0:19question that's probably stopped you
7. 0:21from using AI for data analysis in the
8. 0:23first place. How can I trust the
9. 0:25results? Today's lesson is about the
10. 0:27delegation diligence loop. Specifically,
11. 0:30building confidence in AI's analytical
12. 0:32capabilities for your specific work by
13. 0:35systematically testing it against data
14. 0:37you already understand. By doing this,
15. 0:39you can better understand how AI will
16. 0:41support your specific circumstances.
17. 0:44The process starts with delegation.
18. 0:46Here's how this works. First, identify a
19. 0:49specific analytical task you do
20. 0:50regularly that you want to delegate to
21. 0:52AI. Find past data where you already
22. 0:55completed the analysis and then work
23. 0:57with AI to reproduce what you did,
24. 0:59evaluating what works and what doesn't.
25. 1:02Refine your approach and test again. If
26. 1:04AI can match your known results, you
27. 1:06know how to use it and trust it for
28. 1:07similar future tasks. And if not, you've
29. 1:10learned that this task is something you
30. 1:11shouldn't delegate. So, let me show you
31. 1:14what this looks like in practice, and
32. 1:15then we'll talk through what to do if
33. 1:16you're not that data savvy to begin
34. 1:18with.
35. 1:19Meet Rio, the program director at Valley
36. 1:22Veterans Services. Every quarter, he
37. 1:25analyzes program attendance alongside
38. 1:27employment outcomes, calculating
39. 1:29participation rates, tracking monthly
40. 1:32changes, and determining whether
41. 1:33attendance correlates with job placement
42. 1:36success. This analysis consistently
43. 1:38takes him hours. Considering delegation,
44. 1:41Rio knows he wants to continue using the
45. 1:43results of this analysis to improve his
46. 1:45program. He wants to interpret the
47. 1:47results himself, but he could do without
48. 1:49the data cleaning and formula mayhem he
49. 1:51usually finds himself in to do the
50. 1:53actual analysis. So, in order to test
51. 1:55whether AI is appropriate in this
52. 1:57scenario, he's going to evaluate it
53. 1:59using last quarter's data. He knows
54. 2:02exactly what this data showed after he
55. 2:04analyzed it without AI, and he has the
56. 2:06raw messy data from before he started.
57. 2:09This is his test case. Rio uploads the
58. 2:11data and starts to work with AI using
59. 2:13description and discernment to perform
60. 2:15his analysis. Only each time the AI
61. 2:18responds, Rio is going to check the
62. 2:20results against what he knows to be true
63. 2:22and jot down potential gaps in AI's
64. 2:24reasoning. Sometimes additional
65. 2:26description helps AI get the outcome
66. 2:28he's looking for. In these cases, Rio
67. 2:31knows he has to include that kind of
68. 2:32information for future data analysis
69. 2:34tasks. Other times, Rio might find
70. 2:37legitimate capability gaps. This is the
71. 2:39delegation diligence loop in action.
72. 2:42Rio's diligence to evaluate the model's
73. 2:44capabilities can change what he chooses
74. 2:46to delegate to AI in the future. His
75. 2:48first attempt might look like, I'm
76. 2:50sharing attendance data and employment
77. 2:52outcome data from our job training
78. 2:54program last quarter. Please analyze the
79. 2:56participation patterns across the three
80. 2:58months and graph the correlations
81. 3:00between attendance levels and employment
82. 3:02success. I'm particularly interested in
83. 3:05understanding whether consistent
84. 3:06attendance predicts better job placement
85. 3:08outcomes.
86. 3:10AI responds with a summary, but rather
87. 3:13than assuming this is fact, Rio checks
88. 3:15this against his records and notes
89. 3:18what's good and what's not. AI correctly
90. 3:21identified the correlation between
91. 3:23program attendance and job placement,
92. 3:25but it missed a critical insight around
93. 3:27the combined housing assistance and job
94. 3:29placement program. So Rio refineses his
95. 3:32description asking AI to try again but
96. 3:35pay special attention to the program
97. 3:37type.
98. 3:39This time AI catches its mistake. So Rio
99. 3:42notes that for future quarters he'll
100. 3:44need to specifically request the AI to
101. 3:45consider the program type when
102. 3:47performing its analysis.
103. 3:49Then he tests something harder. Can you
104. 3:51also look at this based on when
105. 3:53participants enrolled? AI responds as
106. 3:56Rio observes that despite not knowing
107. 3:58the enrollment data, AI could help
108. 4:00extract it. He makes a note to cross
109. 4:02reference these results later on.
110. 4:05By going through this process, Rio has
111. 4:07systematically validated what AI can and
112. 4:09can't do for his quarterly reporting.
113. 4:11He's learned that with the right
114. 4:12description, AI can accurately reproduce
115. 4:15the analysis he used to do manually. But
116. 4:17he's also identified clear limitations
117. 4:19and areas for follow-up. AI needs
118. 4:21enrollment dates in the data to do
119. 4:23cohort analysis. Otherwise, it'll try to
120. 4:26infer them, which he doesn't want. And
121. 4:28most importantly, Rio now has a tested
122. 4:31approach that he can confidently use
123. 4:32with this quarter's data and clear notes
124. 4:34about what information he needs to
125. 4:36include and what context he still needs
126. 4:38to add himself. When Rio uses this
127. 4:40validated approach with new data, his
128. 4:42diligence continues. He'll check whether
129. 4:45numbers make sense based on what he
130. 4:46knows about his programs. He'll take
131. 4:48accountability for the final report and
132. 4:50he'll be transparent about AI's role if
133. 4:52asked. But now he's working from
134. 4:55validated confidence, not guesswork. So
135. 4:58here's the framework. Identify a
136. 5:00specific analytical task that you want
137. 5:02to delegate. Be precise about what you
138. 5:05need. Then find past data where you
139. 5:08already completed that analysis. You
140. 5:10need the right answers to evaluate
141. 5:12whether AI can arrive at them.
142. 5:15Work with AI to reproduce your past
143. 5:17analysis and systematically evaluate the
144. 5:19results. What did AI produce? How did it
145. 5:22approach the task? How did it
146. 5:24communicate findings? Identify gaps,
147. 5:27refine your delegation, and then test
148. 5:29again. If you can validate that AI
149. 5:31produces correct results, you've built
150. 5:33an approach that you can confidently use
151. 5:34on new data. But if you can't get there
152. 5:37after several refinements, you've
153. 5:38learned that this isn't a task you
154. 5:40should delegate. So, this is all great,
155. 5:42but what if you're not very comfortable
156. 5:44with the data to begin with and wouldn't
157. 5:46be able to spot those process gaps
158. 5:48yourself? AI can also be a useful tool
159. 5:51to brainstorm and implement solutions
160. 5:52you might not have thought of on your
161. 5:54own. Because AI models are uniquely good
162. 5:56at coding, they can help with things
163. 5:58like writing Excel formulas,
164. 6:00reformatting messy data, and more.
165. 6:03In these cases, you can simply bring
166. 6:05your question or idea to AI and
167. 6:07specifically ask for help understanding
168. 6:09what a solution could look like, just
169. 6:11like how you would work with a data
170. 6:12analyst on your team. As you work with
171. 6:15AI, just keep asking for clarifications
172. 6:17and explanations so that you can follow
173. 6:19the process and understand the final
174. 6:21output. Just remember, validation builds
175. 6:23confidence, but it doesn't eliminate
176. 6:25responsibility. you're still accountable
177. 6:27for checking that these results make
178. 6:28sense and being transparent about AI's
179. 6:31role in your analysis process. This
180. 6:34testing works for any analytical task
181. 6:36you're considering. Donor analysis,
182. 6:38budget forecasting, survey synthesis,
183. 6:41outcome tracking. Test first, validate
184. 6:44what works, then apply with more
185. 6:46confidence, or learn what you shouldn't
186. 6:48delegate at all. In our next lesson,
187. 6:51we'll look at workflow augmentation and
188. 6:53how to apply these same principles when
189. 6:55AI handles routine tasks on your behalf.