YouTube2Text

Causal inference in econometrics — Transcript

by MetricsProf · 6,151 words · 1,055 segments · language en · Watch on YouTube

Full transcript

  1. 0:00welcome to this introductory lecture on
  2. 0:02causal inference in
  3. 0:04econometrics there are some big words in
  4. 0:06the title of this lecture that maybe
  5. 0:08deserve
  6. 0:09explanation in particular I want to
  7. 0:11explain what causal inference
  8. 0:14means causa means that we're interested
  9. 0:17in causality that is we believe that in
  10. 0:20our society in the economic environment
  11. 0:23that we live in there are relationships
  12. 0:25of cause and effect by identifying the
  13. 0:29causes and by manipulating them we as
  14. 0:31politicians managers or citizens can
  15. 0:35affect change and move our society in a
  16. 0:38different direction in a more preferable
  17. 0:40Direction hopefully inference means that
  18. 0:44we want to learn about these causal
  19. 0:45relationships using data of course
  20. 0:48learning from data is nothing special
  21. 0:50that's what we as humans do all the time
  22. 0:53every ray of light that enters your
  23. 0:56retina is data that you use to interpret
  24. 1:00the world around you that you use to
  25. 1:01learn about the world around you what is
  26. 1:04different in what we're going to
  27. 1:05accomplish here is that we will learn
  28. 1:07from the data using rigorous statistical
  29. 1:10methods from the toolbox of econometrics
  30. 1:14and they will be rigorous in two
  31. 1:16important
  32. 1:17ways first we will be very explicit
  33. 1:20about the assumptions that we make about
  34. 1:23the underlying structure of reality um
  35. 1:26or the world model if you
  36. 1:28will
  37. 1:30whenever we are interpreting data we are
  38. 1:32assuming some underlying structure that
  39. 1:35we a priori assume to be true that's
  40. 1:38what we call a model for example if you
  41. 1:41are seeing a person and then you turn
  42. 1:43around 360° and you see a person out
  43. 1:46that looks exactly the same as the
  44. 1:47person you just saw then you assume that
  45. 1:49it's the same person the model that you
  46. 1:51use to interpret the data that you're
  47. 1:54perceiving incorporates the Assumption
  48. 1:56of object
  49. 1:58permanence when we are interpreting data
  50. 2:01naively the model that we're using to
  51. 2:03interpret the data is often very
  52. 2:05implicit and we probably be hardpressed
  53. 2:07to Define exactly what it
  54. 2:10is econometrics is different we will
  55. 2:13always be very deliberate in our choice
  56. 2:15of model and we will always explicitly
  57. 2:17State the assumptions that the model
  58. 2:19makes about the structure of reality
  59. 2:22this will be very helpful for
  60. 2:24identifying the limits and weaknesses of
  61. 2:26a data
  62. 2:27analysis the second dimension along
  63. 2:30which we will be very rigorous is
  64. 2:33statistical error
  65. 2:35control statistical error control keeps
  66. 2:38us from overgeneralizing patterns that
  67. 2:40we see in our
  68. 2:42data when we look at a small number of
  69. 2:46data points and that we typically do
  70. 2:48it's perfectly possible that there are
  71. 2:50patterns that seem to suggest a cause
  72. 2:53relationship that doesn't really exist
  73. 2:56these patterns can randomly occur
  74. 2:58through our choice of data points that
  75. 3:00we're looking at and they wouldn't
  76. 3:01persist in a different choice of data
  77. 3:04sample to give an example of a CA
  78. 3:07relationship consider the effect of
  79. 3:09classroom size on student
  80. 3:16learning now you may be wondering why
  81. 3:19are economists interested in education
  82. 3:22well we as economists consider education
  83. 3:24as knowledge
  84. 3:27production education us uses scarce
  85. 3:31resources such as teacher time or study
  86. 3:34materials to produce
  87. 3:36knowledge many parents and policy makers
  88. 3:39believe that there's a causal
  89. 3:40relationship between classroom size and
  90. 3:42student learning specifically they
  91. 3:45believe that smaller classes lead to
  92. 3:47Better Learning outcomes for students
  93. 3:50presumably because in a smaller
  94. 3:52classroom the teacher has more time to
  95. 3:55focus on the individual student let's
  96. 3:57picture a classroom at a middle school
  97. 4:00we have our
  98. 4:03teacher and we have a bunch of
  99. 4:06students if there are only a few
  100. 4:08students then this is a small classroom
  101. 4:10if there are many students then this is
  102. 4:12a large
  103. 4:13classroom to explain how economists
  104. 4:16think about cause relationships we have
  105. 4:18to introduce three important Concepts
  106. 4:21two of which are very obvious so we
  107. 4:23start with a less obvious one and this
  108. 4:25one addresses the question of where we
  109. 4:27are measuring the cause and effect so
  110. 4:29we're asking who is
  111. 4:33affected for our example here there are
  112. 4:36different choices that would work for
  113. 4:37example we could say it is the
  114. 4:39individual student that is affected and
  115. 4:42we're measuring everything at the
  116. 4:43student level what here we're making a
  117. 4:46different choice we're measuring
  118. 4:47everything at the classroom
  119. 4:49level in econometric language we say the
  120. 4:52classroom is our unit of
  121. 4:56observation now we get to the two
  122. 4:58obvious Concepts when we talk about
  123. 5:00cause of relationship the first
  124. 5:02corresponds to the question what is the
  125. 5:07CA and the second one corresponds to the
  126. 5:10question what is
  127. 5:14caused what is the cause in our example
  128. 5:18let's write it's classroom
  129. 5:21size the fancy econometric word for
  130. 5:24cause is
  131. 5:26treatment this term is derived from the
  132. 5:28medical literature where we try to use
  133. 5:31data to evaluate the efficiency of a
  134. 5:34medical treatment for economists a
  135. 5:37treatment doesn't have to be a medical
  136. 5:39treatment it can be anything that can be
  137. 5:42manipulated if it's something that
  138. 5:44cannot be manipulated then as you'll see
  139. 5:46in a second we will run into some
  140. 5:48philosophical issues of defining just
  141. 5:51what a causal effect is more
  142. 5:53pragmatically why would we be interested
  143. 5:55in learning the effect of something that
  144. 5:57we can never change anyway
  145. 6:00what is cause in our example yes it's
  146. 6:03student learning let's say student
  147. 6:05learning is reflected by test
  148. 6:08scores economists call the variable that
  149. 6:11is affected by the cause the
  150. 6:15outcome as you will notice repeatedly
  151. 6:17throughout this class economists are
  152. 6:19very lazy often instead of using
  153. 6:22complicated words we like to use Simple
  154. 6:25symbols so instead of talking about
  155. 6:27classroom size or using the more general
  156. 6:29term treatment we will simply use the
  157. 6:31symbol capital D and instead of
  158. 6:35referring to a test score or the outcome
  159. 6:39we simply use the symbol capital
  160. 6:42Y these symbols give us a really compact
  161. 6:45way of representing the causal
  162. 6:46relationship using a causal diagram in a
  163. 6:50causal diagram we use arrows to
  164. 6:52represent causal relationships in our
  165. 6:55case we believe that
  166. 6:58d effects
  167. 7:01why the error here represents the causal
  168. 7:05effect that the treatment has on the
  169. 7:08outcome we haven't defined exactly what
  170. 7:10a causal effect is and it is the most
  171. 7:13exciting concept that we cover today in
  172. 7:15fact everything I've done so far was
  173. 7:17just building towards being able to
  174. 7:19explain what the causal effect
  175. 7:22is in the following I want you to think
  176. 7:24of this classroom as a real classroom
  177. 7:27that exists somewhere in the world since
  178. 7:29it's a real classroom it is observable
  179. 7:32and I know what its classroom size is
  180. 7:35and what its test score is I've only
  181. 7:38drawn a few students here let's assume
  182. 7:40that there are more than three students
  183. 7:41in this class let's assume that there
  184. 7:44are 35 students in the
  185. 7:47class and let's say that the test score
  186. 7:51is
  187. 7:5366 to emphasize that this could be a
  188. 7:56real world classroom I've picked the
  189. 7:58classroom characteristics from a real
  190. 8:00world data
  191. 8:02set this data is from a famous research
  192. 8:05paper by authors angrist and
  193. 8:10Levy I've plotted their data class size
  194. 8:14against test
  195. 8:16score every Point here represents an
  196. 8:19observed classroom and look here's our
  197. 8:22classroom it sits right here it has a
  198. 8:25class size of 35 and a test score of
  199. 8:3166 these are exactly the numbers that I
  200. 8:34gave
  201. 8:35here suppose that we are able to
  202. 8:37manipulate classroom size for this
  203. 8:39classroom this means we can take this
  204. 8:45classroom and put an exact copy of this
  205. 8:48classroom down here then we pick a new
  206. 8:52student little Jimmy down
  207. 8:55here and we move Jimmy into the class so
  208. 9:00now there will be 36
  209. 9:02students this is the only change we make
  210. 9:05in this classroom everything else we
  211. 9:08keep
  212. 9:11equal the Latin term for everything else
  213. 9:13equal is set is
  214. 9:17pares this is a term that we will use a
  215. 9:21lot we imagine this hypothetical
  216. 9:24classroom in order to define a causal
  217. 9:27effect it doesn't actually factually
  218. 9:31exist we call it the counterfactual
  219. 9:37classroom it is tempting to call the
  220. 9:40real classroom the factual classroom but
  221. 9:43no one ever does that what are the
  222. 9:45characteristics of the counterfactual
  223. 9:48classroom well it has a class size of
  224. 9:5136 now I don't want to write D for the
  225. 9:55class size here because I want to
  226. 9:57reserve capital letters
  227. 9:59for variables that represent real values
  228. 10:03not hypothetical ones so instead I write
  229. 10:07lower case D equal to
  230. 10:1036 what is the outcome for the
  231. 10:13counterfactual
  232. 10:14classroom our hypothesize cause a
  233. 10:17relationship suggests that the outcome
  234. 10:19should depend on the treatment let's
  235. 10:21make this explicit by writing
  236. 10:25y as a function of
  237. 10:29treatment the treatment lower case D can
  238. 10:33be any value that represents a valid
  239. 10:35classroom size therefore y of lower case
  240. 10:38D represents hypothetical outcomes under
  241. 10:41a certain assigned classroom size in
  242. 10:45particular if we put
  243. 10:48in the real observe classroom size we'll
  244. 10:52get the observed
  245. 10:54outcome the Y is a function of an
  246. 10:57hypothetical treatment is called called
  247. 10:59a potential
  248. 11:02outcome the outcome under the
  249. 11:05counterfactual is therefore the
  250. 11:07potential
  251. 11:09outcome
  252. 11:11at a classroom size of 36 since this
  253. 11:15number only ever becomes relevant in a
  254. 11:17hypothetical scenario we don't know what
  255. 11:20it is if it is true that smaller
  256. 11:22classrooms are good for student learning
  257. 11:24then this number should be smaller than
  258. 11:2766 Maybe
  259. 11:3065 but we don't even know if that
  260. 11:32conjecture is correct so for all we know
  261. 11:34this could also be 70 well we don't know
  262. 11:38what it is regardless of what this
  263. 11:40number is at least now we have all the
  264. 11:43tools that we need for a clean
  265. 11:45definition of the treatment
  266. 11:48effect the treatment
  267. 11:51effect is equal to the outcome in the
  268. 11:54counterfactual scenario this is the
  269. 11:56outcome where we have added one student
  270. 12:00to the class size minus the outcome in
  271. 12:04The observed scenario that is the
  272. 12:06outcome where class size is equal to
  273. 12:10Capital
  274. 12:12D in other words we Define the treatment
  275. 12:15effect as a simple before after
  276. 12:20comparison except that the after never
  277. 12:24really happens it only happens in our
  278. 12:26head in our thought experiment where we
  279. 12:28could consider the
  280. 12:30counterfactual in our example this here
  281. 12:33works out to be potential outcome at
  282. 12:3936 minus we don't have to speculate
  283. 12:42about the observed outcome it's
  284. 12:4666 the ceterus parabus treatment effect
  285. 12:50that we've defined here is the causal
  286. 12:53effect that we interested
  287. 12:55in so now we've explained what causality
  288. 12:58means means to us we can move on to
  289. 13:02inference how can we learn the value of
  290. 13:04this treatment effect clearly since we
  291. 13:07know the observed outcome the difficult
  292. 13:11part is figuring out the counterfactual
  293. 13:14outcome this is called the
  294. 13:16identification
  295. 13:20problem recall our purple classroom from
  296. 13:24before it has a class size of 35
  297. 13:27students
  298. 13:29to figure out the treatment effect we
  299. 13:31have to consider the counterfactual of a
  300. 13:34classroom size of 36
  301. 13:36students I said previously that we don't
  302. 13:39observe the counterfactual
  303. 13:41outcome let's think little bit about why
  304. 13:44that is
  305. 13:48true why can't we just change the
  306. 13:50classroom size from 35 to 36 and then
  307. 13:54observe our counterfactual
  308. 13:56outcome well first of we as researchers
  309. 13:59usually don't have the ability to change
  310. 14:01the classroom size that decision is up
  311. 14:04to the school principal the school board
  312. 14:06politicians or whoever decides on
  313. 14:08classroom size it's usually not
  314. 14:11researchers but suppose that we have
  315. 14:14this extraordinary power we can just
  316. 14:16change the classroom size any way we
  317. 14:18want it's still not really enough why is
  318. 14:22that well we haven't really talked about
  319. 14:24what our outcome is we said test score
  320. 14:26let's be a little bit more specific
  321. 14:28let's say this test score at the end of
  322. 14:31grade seven so we are looking at
  323. 14:34learning outcome during grade seven
  324. 14:36right that makes sense so we've observed
  325. 14:39that for this
  326. 14:41class now let's
  327. 14:44change the class size 236 let's suppose
  328. 14:48we're able to do that how do we get the
  329. 14:51test score from grade 7 the students
  330. 14:54have already done grade seven once well
  331. 14:58suppose we have the power to make them
  332. 15:00take grade seven again so that we can
  333. 15:04observe a new learning outcome for grade
  334. 15:06seven a second learning
  335. 15:09outcome the problem is that the learning
  336. 15:11outcome for a classroom size of 36 is
  337. 15:14not recorded under CIS parab
  338. 15:17circumstances we have changed a lot of
  339. 15:19things in addition to classroom size for
  340. 15:22example since the students have already
  341. 15:25taken grade seven before they've already
  342. 15:27studied a lot of topics so they have
  343. 15:30entered grade seven with a different set
  344. 15:33of prior knowledge also all the students
  345. 15:35are older and more
  346. 15:38mature as you can see we've changed all
  347. 15:40of these things in addition to changing
  348. 15:42classroom size this is not what cetus
  349. 15:45parabus
  350. 15:47means the only way that I can think of
  351. 15:50to really record the cetus parabus
  352. 15:52counterfactual outcome is if we have
  353. 15:55time travel after the students have
  354. 15:58taken grade seven the first time around
  355. 16:01and we've recorded their Baseline
  356. 16:03learning outcome we travel back in
  357. 16:07time wipe their memories and make them
  358. 16:10retake grade seven at the end of the
  359. 16:13year we record the counterfactual
  360. 16:15learning outcome yeah that's the only
  361. 16:18plan I have and it's not really feasible
  362. 16:20because I haven't really figured out
  363. 16:21time travel no good news on that
  364. 16:24front I hope that you're convinced now
  365. 16:27that we can't observe the counterfactual
  366. 16:30outcome let's focus on things that we
  367. 16:32can actually observe for this classroom
  368. 16:35we don't observe a class size of 36 but
  369. 16:38there may be a different classroom where
  370. 16:40we actually have a class size of 36
  371. 16:43let's say this is the green classroom it
  372. 16:46also has lots of
  373. 16:49students has a teacher a different
  374. 16:51teacher
  375. 16:53obviously and this class has little
  376. 16:56Jimmy in it it has a class class size of
  377. 17:0136 recall that class size is our
  378. 17:04treatment so this classroom receives
  379. 17:07more treatment than this
  380. 17:10classroom we say it receives higher
  381. 17:13treatment intensity so this is the high
  382. 17:16treatment intensity classroom and this
  383. 17:20is the low treatment intensity
  384. 17:27classroom in a lot of treatment effect
  385. 17:30settings in medicine for example
  386. 17:32treatment is binary that means you
  387. 17:35either get the full treatment or you
  388. 17:36don't get any treatment at all for
  389. 17:39example you would be in the high
  390. 17:42intensity group if you get all three
  391. 17:44doses of a three dose vaccine and you
  392. 17:46would be in the low treatment intensity
  393. 17:48group if you don't get the vaccine at
  394. 17:50all in such a setting we say that a high
  395. 17:53treatment intensity unit is
  396. 17:57treated and and that a low treatment
  397. 17:59intensity unit is a control
  398. 18:02unit let's use these terms also
  399. 18:05here for both of these classrooms we
  400. 18:08observe learning
  401. 18:12outcomes so how do we get at the
  402. 18:16treatment
  403. 18:20effect a natural approach is to subtract
  404. 18:23the control outcome from the treated
  405. 18:27outcome
  406. 18:29this is not exactly equal to the
  407. 18:30definition of the cause effect so we
  408. 18:32shouldn't put an equal sign here we just
  409. 18:35want to indicate that this is a guess
  410. 18:37based on our data of the treatment
  411. 18:40effect a guess based on data is called
  412. 18:42an
  413. 18:44estimate to indicate that a value
  414. 18:47depends on data we often put a head on
  415. 18:50it to check whether this is really
  416. 18:53estimating the treatment effect let's
  417. 18:55make it look a little more like our
  418. 18:56definition of the treatment effect fact
  419. 18:59that is let's make it look more like
  420. 19:02this the treated outcome is the
  421. 19:05potential
  422. 19:07outcome at a class size of
  423. 19:1136 the control outcome is the potential
  424. 19:14outcome at a class size of
  425. 19:2135 36 is equal to the Baseline class
  426. 19:25size +
  427. 19:27one
  428. 19:30and 35 obviously is just equal to the
  429. 19:32Baseline class
  430. 19:34size now does this look
  431. 19:38like
  432. 19:40this almost but not quite can you see
  433. 19:44what's gone wrong yes exactly here we
  434. 19:47have the potential outcome function for
  435. 19:49the green class and not for the purple
  436. 19:53class the potential outcome functions
  437. 19:56don't necessarily have to be the same
  438. 19:59to explain why let's think more
  439. 20:01carefully about the economic environment
  440. 20:03that we are studying and let's do this
  441. 20:05using a caus a
  442. 20:09diagram we are studying test
  443. 20:14scores this is our outcome
  444. 20:17variable we're interested in the effect
  445. 20:19of class
  446. 20:22size this is our
  447. 20:24treatment our conjecture is that there
  448. 20:27is a cause of relationship between these
  449. 20:29two and we indicate the cause
  450. 20:31relationship by an
  451. 20:34error clearly class size is not the only
  452. 20:38factor that has an effect on test
  453. 20:41score consider for example the socio
  454. 20:43economic background of the students in
  455. 20:45the
  456. 20:47classroom for example you may imagine
  457. 20:50that a student who has Highly Educated
  458. 20:53parents is more likely than other
  459. 20:55students to receive help with homework
  460. 20:59we expect socio economic background to
  461. 21:01also have causal effect on the test
  462. 21:05score social economic background has
  463. 21:07many facets in the data that I showed
  464. 21:10you earlier it is measured by a variable
  465. 21:12called PD
  466. 21:15Index this stands for percentage
  467. 21:19disadvantaged and gives the percentage
  468. 21:22of disadvantaged students where
  469. 21:23disadvantage is defined by several
  470. 21:25characteristics of the parents
  471. 21:28the composition of socio economic
  472. 21:30backgrounds in the classroom is
  473. 21:32obviously determined by The Neighborhood
  474. 21:34that the school is
  475. 21:38in neighborhood may also affect class
  476. 21:41size for example if the school is
  477. 21:43located in a rural sparsely populated
  478. 21:46area there may be only a few students
  479. 21:49entering School each year only enough
  480. 21:51students to have one small class so
  481. 21:54neighborhood can affect the size of the
  482. 21:56student cohort
  483. 22:00which in turn affects class
  484. 22:04size the PD index is also observed by
  485. 22:07politicians who may use it and actually
  486. 22:10do use it to allocate more resources to
  487. 22:12schools with high percentage of
  488. 22:14disadvantaged students for example they
  489. 22:17may have smaller classes in such
  490. 22:21areas I hope that I've convinced you
  491. 22:24that small classrooms and large
  492. 22:26classrooms may be very different in in
  493. 22:28terms of student composition and that
  494. 22:30these students May react very
  495. 22:31differently to differently sized
  496. 22:34classrooms therefore we shouldn't expect
  497. 22:36all classrooms to share the same
  498. 22:39potential outcome
  499. 22:41function there may be systematic
  500. 22:44differences in potential outcome
  501. 22:45functions or well unsystematic
  502. 22:48ones let's start with the systematic
  503. 22:51ones assume that smaller classes tend to
  504. 22:56face a different PD and index than laric
  505. 23:00classes we've already discussed that the
  506. 23:03PD index has a connection to the kind of
  507. 23:08potential outcome function that we're
  508. 23:10assuming this creates a systematic
  509. 23:13relationship between the class size that
  510. 23:15we are seeing and the kind of potential
  511. 23:18outcome function that we should be
  512. 23:22expecting a variable like PD index that
  513. 23:25is lurking in the background and the
  514. 23:27systematic changing the potential
  515. 23:29outcome functions is called a
  516. 23:34confounder if we observe this confounder
  517. 23:37then we should be able to deal with it
  518. 23:39by just slightly changing the potential
  519. 23:42outcome
  520. 23:46framework let's quickly recap the
  521. 23:49variables that we
  522. 23:52observe we observe treatment
  523. 23:56D this is class
  524. 23:59size and we observe outcome
  525. 24:02y this is the test
  526. 24:06score now we've added a new observable
  527. 24:08variable the PD
  528. 24:11index and this is the confounder in our
  529. 24:15example and we abbreviate it by the
  530. 24:17symbol
  531. 24:19X the potential outcome defines what the
  532. 24:23outcome will be under a hypothetical
  533. 24:26treatment assignment of small
  534. 24:29D so this is a classroom size that's
  535. 24:32potentially different from The observed
  536. 24:34classroom size for every possible
  537. 24:37treatment intensity we assign a
  538. 24:39potential
  539. 24:42outcome we previously called this y of
  540. 24:46D now we've added the confounder X and
  541. 24:49we treat X similar to D that is we're
  542. 24:53thinking about different levels of X
  543. 24:56that are conceivable
  544. 24:58confounder level lowercase
  545. 25:02x we argued that the confounder should
  546. 25:06have an impact on the potential outcome
  547. 25:08this not apparent in this notation so
  548. 25:10let's change that and we're letting the
  549. 25:13potential outcome depend on D as well as
  550. 25:18X there you go that wasn't too hard now
  551. 25:20we've Incorporated confounders in the
  552. 25:23potential outcome
  553. 25:25framework recall our perp classroom it
  554. 25:28was our baseline or control
  555. 25:32classroom it has a classroom size of
  556. 25:3635 it has to have a value of the PD
  557. 25:38index let's just say that is 20 and it
  558. 25:43has an outcome which is the potential
  559. 25:47outcome evaluated
  560. 25:50at 35 classroom size and 20 PD index the
  561. 25:56blue classroom was our treated
  562. 26:01classroom it had one more student it
  563. 26:04certainly has a PD value let's just say
  564. 26:07um
  565. 26:0935 and an outcome of the potential
  566. 26:13outcome evaluated at 36 and
  567. 26:1835 now we see clearly when we're
  568. 26:20comparing the two we're not just
  569. 26:24comparing different classroom sizes we
  570. 26:27also comparing different PD
  571. 26:30indices that's no good from this
  572. 26:33comparison we don't expect to learn much
  573. 26:34about the pure effect of a larger
  574. 26:38classroom but don't despair this seems
  575. 26:41easy to fix we've just chosen the wrong
  576. 26:44blue classroom recall that the blue
  577. 26:46classroom was supposed to be a stand in
  578. 26:49for the counterfactual purple classro
  579. 26:52that we don't
  580. 26:53observe but the counterfactual purle
  581. 26:55classroom is a cus par
  582. 26:58classroom where the X shouldn't be
  583. 27:01changed but here actually we see
  584. 27:04different axes so why don't we choose a
  585. 27:07blue classroom that has the same a as
  586. 27:09the purple
  587. 27:13classroom there we go this is already
  588. 27:15looking much
  589. 27:17better now our treatment effect
  590. 27:22estimate is treated y minus control y
  591. 27:28so is equal to potential outcome Y at
  592. 27:3536 20 minus potential outcome at
  593. 27:4235
  594. 27:4420 and we can rewrite this as
  595. 27:51before and key is of course that now
  596. 27:55here we have the same X let's let's try
  597. 27:57to implement this strategy for
  598. 27:59estimating the treatment effect using
  599. 28:01our
  600. 28:03data here I've plotted our data this
  601. 28:06time against the dimensions of percent
  602. 28:08disadvantaged so this is our
  603. 28:12confounder and class size this is our
  604. 28:18treatment here's our example classroom
  605. 28:21it's still here going
  606. 28:23strong and now we're looking for
  607. 28:26potential in Sample
  608. 28:28counterfactuals so our classroom has a
  609. 28:30class size of
  610. 28:3235 so we want to look at classrooms that
  611. 28:34have a class size of 36 so all of
  612. 28:38these also they should have a similar
  613. 28:41percentage
  614. 28:43disadvantage so I'm looking in this
  615. 28:46range this gives us candidates for the
  616. 28:49blue classroom that are
  617. 28:51located
  618. 28:53here that's great if everything works
  619. 28:56out as we anticipate we should be able
  620. 28:58to choose any of these classrooms and
  621. 29:00then just use their y as the blue
  622. 29:04Y and then we can estimate the treatment
  623. 29:07effect where we are controlling for the
  624. 29:10effect of the
  625. 29:12compounder so let's zoom in on these
  626. 29:15classrooms I have them all
  627. 29:18here they have percentage disadvantage
  628. 29:21between 17 and
  629. 29:2323 they all have a classroom size of of
  630. 29:2836 here's our classroom for comparison
  631. 29:32now here those are the test scores and
  632. 29:35oh no there's a problem they all have
  633. 29:40different test
  634. 29:42scores even those that have exactly the
  635. 29:45same percentage disadvantaged have
  636. 29:49different test
  637. 29:51scores so what should we choose as the
  638. 29:54blue y our choice will have huge cont
  639. 29:57consequences if we choose one of these
  640. 30:00for example we'll estimate a positive
  641. 30:02treatment effect if we choose one of
  642. 30:04these we will estimate a negative
  643. 30:06treatment effect how do we make the
  644. 30:09right
  645. 30:10choice one thing that we have definitely
  646. 30:13learned is
  647. 30:15that treatment and confounder don't
  648. 30:19fully determine the outcome there are
  649. 30:21other things that matter and that shift
  650. 30:24around the potential outcomes now this
  651. 30:26could be an additional confounder that
  652. 30:29we haven't taken into
  653. 30:30considerations but I want to propose
  654. 30:32something else you remember when I said
  655. 30:36there are systematic and unsystematic
  656. 30:38differences in potential outcomes where
  657. 30:41the confounder cost differences were the
  658. 30:44systematic ones now let's look at the
  659. 30:47unsystematic
  660. 30:48ones to talk about unsystematic
  661. 30:50differences we have to recall a very old
  662. 30:53concept from statistics the concept of
  663. 30:56an error term
  664. 31:00we are now about to impose some
  665. 31:02restrictions on how we think test scores
  666. 31:05are produced that is we're going to make
  667. 31:07some model
  668. 31:08assumptions as I promised you before
  669. 31:11whenever we make assumptions we are very
  670. 31:13explicit about
  671. 31:16that oh yeah now I see this was supposed
  672. 31:19to be word model we're making model
  673. 31:22assumptions let me just fix that real
  674. 31:26quick
  675. 31:27here we
  676. 31:28go so what kind of assumptions are we
  677. 31:31imposing as before we assume that
  678. 31:34outcome is given by the potential
  679. 31:37outcome evaluated at The observed
  680. 31:40treatment and observed confounder this
  681. 31:43in itself is not restrictive at all but
  682. 31:47now we going to make some additional
  683. 31:48assumptions we assume that this is equal
  684. 31:50to some function M of
  685. 31:55treatment and conf
  686. 31:58founder and this function is the same
  687. 32:01for all
  688. 32:03classrooms plus and aror term
  689. 32:08U once we assume that this part is the
  690. 32:11same for all classrooms you see
  691. 32:13immediately that we need an additional
  692. 32:15component to explain that all of these
  693. 32:19here don't have the same test score in
  694. 32:22our new model of test score
  695. 32:24production these differences are
  696. 32:27explained by these classrooms having
  697. 32:30different error
  698. 32:33terms let's just note that U is our
  699. 32:37error
  700. 32:39term and the idea of an error term is
  701. 32:42that it is completely
  702. 32:43unsystematic statisticians say it is
  703. 32:47random
  704. 32:51noise so for some classrooms this will
  705. 32:53be positive for others it will be
  706. 32:56negative and if we average over
  707. 32:58classrooms then the average will be
  708. 33:00close to
  709. 33:02zero this gives us a completely new idea
  710. 33:05for choosing the blue classroom the blue
  711. 33:07classroom shouldn't correspond to any
  712. 33:09individual classroom it should be an
  713. 33:12average of appropriate comparison
  714. 33:16classrooms in the average the effect of
  715. 33:18the error term will wash
  716. 33:21out for example we could average over
  717. 33:24all of these classrooms they seem to be
  718. 33:26good compar in
  719. 33:28classrooms in preparation for defining
  720. 33:30the average let's label them let's call
  721. 33:33this the I equal 1 classroom the I equal
  722. 33:362 classroom I equal 3 classroom and so
  723. 33:43on to identify values corresponding to a
  724. 33:46classroom we give all values in I
  725. 33:48subscript for
  726. 33:52example we would denote the test scorer
  727. 33:55in the first classroom here here as y
  728. 33:59subscript
  729. 34:021 we're counting 1 2 3 four five
  730. 34:05candidates for the blue classroom let's
  731. 34:09write this as blue n equal
  732. 34:125 now we have enough notation to
  733. 34:14describe our idea that the average error
  734. 34:16term is
  735. 34:17zero to formally write down the average
  736. 34:20over the error terms we
  737. 34:24sum over all of the five classrooms
  738. 34:28starting with the first classroom up to
  739. 34:31the fifth classroom right this was our
  740. 34:35blue
  741. 34:36n we sum the error
  742. 34:39terms and now we divide by the number of
  743. 34:44classrooms blue
  744. 34:46n we are hoping that this will be
  745. 34:49approximately equal to
  746. 34:52zero before we move on I have to correct
  747. 34:55some unfortunate notation
  748. 34:57I chose blue here as a color for the
  749. 35:00error term but this is not meant to
  750. 35:02indicate that the error term is only
  751. 35:03relevant for the blue classrooms so let
  752. 35:06me change the color of the error
  753. 35:12term so this is already much
  754. 35:16better when is our hope that this
  755. 35:18average is zero
  756. 35:21Justified while we're averaging over
  757. 35:24values in our sample so we have to ask
  758. 35:27where our sample is coming
  759. 35:29from we think of our sample as coming
  760. 35:32from a large collection of
  761. 35:34classrooms that is called the
  762. 35:39population the population contains all
  763. 35:42the classrooms that we think follow the
  764. 35:44same economic rules of knowledge
  765. 35:46production for example this could be all
  766. 35:48the classrooms in the same country or in
  767. 35:50the same educational
  768. 35:52system we collect our sample by
  769. 35:54selecting classrooms from the population
  770. 35:59and most of the time we do so randomly
  771. 36:01so this is called random
  772. 36:05sampling let's assume that we sample 1 2
  773. 36:10up
  774. 36:11to n
  775. 36:14classrooms we observe these classrooms
  776. 36:16and can record their values of test
  777. 36:20score class size and PD
  778. 36:25index
  779. 36:27this is data that we can say for example
  780. 36:30in an Excel
  781. 36:31sheet under what circumstances can we
  782. 36:34expect averages on the sample to be
  783. 36:36equal for example to
  784. 36:39zero the law of large numbers from
  785. 36:44statistics says that population
  786. 36:50averages are close to sample
  787. 36:55averages at least if the sample size n
  788. 36:59is
  789. 37:03large for the law of large numbers to
  790. 37:05hold it's also important that we sample
  791. 37:09randomly we can have a reasonable
  792. 37:12expectation that the sample average is
  793. 37:13close to zero if the corresponding
  794. 37:16population average is close to
  795. 37:18zero population averages can be written
  796. 37:21as expectations we'll explain why at a
  797. 37:24later point so the corresponding
  798. 37:27population average can be written as the
  799. 37:33expectation of the error term in the
  800. 37:36subpopulation of classrooms that have a
  801. 37:39class
  802. 37:40size equal to
  803. 37:4336 and face a PD
  804. 37:47index equal to
  805. 37:5020 let's assume that this population
  806. 37:53average is equal to zero
  807. 37:58by the law of large numbers the sample
  808. 38:00average inherits this Zero from the
  809. 38:04population
  810. 38:06average this seems like a good
  811. 38:08assumption now but as you'll see in a
  812. 38:10second we need something a little bit
  813. 38:11more General so for now let's assume
  814. 38:14that it doesn't hold for this
  815. 38:16subpopulation but for all
  816. 38:20subpopulations it holds for arbitrary
  817. 38:23lowercase D and arbitrary lowercase x
  818. 38:27later we would call this kind of
  819. 38:28assumption an exogeneity
  820. 38:34assumption another good name for it
  821. 38:36would be a balancing assumption because
  822. 38:38it assumes that the error terms in the
  823. 38:42subpopulations are well balanced in the
  824. 38:44sense that they average out to
  825. 38:52zero previously we estimated the
  826. 38:54treatment effect using a single blue
  827. 38:58classroom and its observed learning
  828. 39:02outcome now we're going to modify this
  829. 39:04identification strategy instead of using
  830. 39:07a single classroom we're using an
  831. 39:09average over
  832. 39:12classrooms the average sums over all
  833. 39:16suitable
  834. 39:19candidates takes the observed test score
  835. 39:22and then we divide by the number of
  836. 39:25observations
  837. 39:28before we can discuss whether our
  838. 39:29estimation strategy based on this
  839. 39:31quantity actually works we have to think
  840. 39:34about how to interpret causal effects
  841. 39:37now that we have an error
  842. 39:40term when we consider a set's parabus
  843. 39:43change what happens to the error term
  844. 39:46what is the error term anyway in
  845. 39:48economic
  846. 39:49terms for every given classroom U
  847. 39:52describes the effect of factors other
  848. 39:55than D and acts that are present in the
  849. 39:57classroom and that affect knowledge
  850. 39:59production in that classroom for example
  851. 40:02if a class has a particularly clever
  852. 40:04student that would enter through
  853. 40:08you we said you is like random noise but
  854. 40:11that's not very helpful for
  855. 40:12understanding you economically for every
  856. 40:15given classroom you is not random at all
  857. 40:18it's just one facet that describes the
  858. 40:20circumstances in the classroom random
  859. 40:23noise just means when we compare the
  860. 40:25circumstances between classrooms they
  861. 40:27seem
  862. 40:29random when we think about a setus parus
  863. 40:32change we only change in D we're not
  864. 40:35changing X and we're not changing
  865. 40:38U what does this mean for our treatment
  866. 40:47effect the treatment
  867. 40:50effect compares the outcome in the
  868. 40:53counterfactual
  869. 40:55word
  870. 40:57to the actually observed
  871. 41:00outcome we can now plug in our model for
  872. 41:02the potential
  873. 41:04outcomes so we have M of D +
  874. 41:101 and
  875. 41:13X plus our error term
  876. 41:16U and now we subtracting this part
  877. 41:21minus there's M this time evaluated at D
  878. 41:25and X
  879. 41:29plus the error
  880. 41:31term the error term shows up here and
  881. 41:34here so we'll just cancel and we're left
  882. 41:37with the difference between these M
  883. 41:48functions this is the treatment effect
  884. 41:50that we're trying to
  885. 41:52estimate now let's move on to treatment
  886. 41:55effect estimation
  887. 42:02let's start by considering what Y Bar is
  888. 42:05under our model
  889. 42:07assumptions Y Bar is the average over
  890. 42:11all the blue
  891. 42:15classrooms so y goes from one to the
  892. 42:19blue
  893. 42:20n and we're dividing this sum by the
  894. 42:24blue n
  895. 42:27so this will be the
  896. 42:32average over and
  897. 42:35now I'm plugging in the model here first
  898. 42:39we have the M function evaluated at the
  899. 42:45treatment and the
  900. 42:49confounder plus the arrow
  901. 42:54term now we've chosen the blue
  902. 42:56classrooms so that the treatment is
  903. 43:00always
  904. 43:0236 and the confounder is always
  905. 43:0720 therefore this is a fixed number and
  906. 43:09we'll just shift the average by this
  907. 43:12amount we can move this outside of the
  908. 43:19average and here get rid of the
  909. 43:25parenthesis
  910. 43:26now this here will be approximately Zero
  911. 43:29by the law of large
  912. 43:32numbers therefore the whole thing will
  913. 43:35be
  914. 43:37approximately the M function at 36 and
  915. 43:4120 previously we estimated the treatment
  916. 43:47effect by the difference between the
  917. 43:50blue
  918. 43:51classroom and the purple classroom now
  919. 43:54our plan was to modify this edification
  920. 43:56strategy by replacing the single
  921. 43:59classroom by the average of classrooms
  922. 44:03here what will this get us well this
  923. 44:05will get us
  924. 44:10approximately M at
  925. 44:1336 20 minus and now I'm plugging in the
  926. 44:18model for this
  927. 44:19y the m at
  928. 44:2435 20
  929. 44:28plus the error term in the purple
  930. 44:32classroom this doesn't give us the
  931. 44:34treatment
  932. 44:35effect that we figured out here because
  933. 44:39we end up with this additional error
  934. 44:42term how do we get rid of the error term
  935. 44:46well the same way that we got rid of the
  936. 44:47error term for the blue classrooms
  937. 44:50instead of using a single control
  938. 44:52classroom we're using an average of
  939. 44:55several control classrooms that have
  940. 44:57similar level of classroom size and
  941. 45:07confer suppose that we can find purple n
  942. 45:10such
  943. 45:16classrooms we can define a similar
  944. 45:18average as we had here and then apply
  945. 45:22very similar arguments to show that this
  946. 45:24is equal to
  947. 45:26the M function but this time at 35 the
  948. 45:31smaller class size and confounder level
  949. 45:3520 plus an average of error
  950. 45:44terms this again will be close to Zero
  951. 45:47by the law of large
  952. 45:49numbers therefore this will be
  953. 45:51approximately equal to M at 35
  954. 45:5720 now we can replace the single
  955. 46:00classroom by this
  956. 46:02average we get rid of this
  957. 46:07U now we have this difference of M
  958. 46:11functions and just to be clear the first
  959. 46:14M
  960. 46:15function is evaluated at per D + 1 and
  961. 46:21confounded level equal to what we see in
  962. 46:24the purple class
  963. 46:27and then the second M function is at
  964. 46:31D comma
  965. 46:34X this is exactly the expression that we
  966. 46:37figured out for the treatment
  967. 46:44effect finally we have figured out a
  968. 46:47viable identification
  969. 46:49strategy our estimate is close to the
  970. 46:52thing that we're trying to
  971. 46:54estimate let's take this edification
  972. 46:56strategy to the
  973. 46:57data first we need all the purple
  974. 47:00classrooms that is the control
  975. 47:05classrooms they have a class size of 35
  976. 47:09and a PD
  977. 47:10index of
  978. 47:1420 here we go we also need all the blue
  979. 47:19classrooms that is the treated
  980. 47:21classrooms they have a class size of 36
  981. 47:24and also a p of
  982. 47:2720 so now we have all the data that we
  983. 47:30need to compute our treatment effect
  984. 47:32estimate I'll leave the computation up
  985. 47:34to
  986. 47:35you before I finish this video I want to
  987. 47:37point out two issues with our estimation
  988. 47:40strategy
  989. 47:42here the first issue is what if we have
  990. 47:45an additional confounder maybe let's
  991. 47:47call it
  992. 47:49X2 where is the effect of X2 modeled in
  993. 47:52this model well it's not here it's not
  994. 47:54here so it's got to be here so the
  995. 47:58confounder X2 would enter the model
  996. 48:00through U but we've assumed that U is
  997. 48:04balanced in the sense of satisfying this
  998. 48:06Exogen
  999. 48:08assumption now the effect of confound us
  1000. 48:10is typically not balanced so in this
  1001. 48:12case we would have trouble justifying
  1002. 48:15the Assumption of course what we can do
  1003. 48:17if we observe X2 at least is ADD X2 as
  1004. 48:21an additional term in this mamp function
  1005. 48:23in this case we say that we control for
  1006. 48:25X2
  1007. 48:27however this may lead to additional
  1008. 48:29complications but let's first talk about
  1009. 48:31issue
  1010. 48:32two our argument relied on the law of
  1011. 48:35large numbers we need population
  1012. 48:38averages to be reflected in Sample
  1013. 48:40averages this works only if n is large
  1014. 48:43that is if we are taking a sample
  1015. 48:45average over many
  1016. 48:49values here we are taking averages over
  1017. 48:52only very few
  1018. 48:54observations for our control group we
  1019. 48:56have 1 2 3 observations and for the
  1020. 48:59treatment group we have four
  1021. 49:01observations these numbers are way too
  1022. 49:03small to ensure that the law of large
  1023. 49:04numbers
  1024. 49:05works now think back to issue one if we
  1025. 49:09Define the control and treatment groups
  1026. 49:11even more narrowly by requiring that
  1027. 49:14they not only share the same PD index
  1028. 49:16but also the value of an additional
  1029. 49:18confounder we make these groups even
  1030. 49:20smaller and take averages over even
  1031. 49:22fewer
  1032. 49:23observations this picture is a great
  1033. 49:25illustration of our problem we initially
  1034. 49:28start out with a lot of data but end up
  1035. 49:30using only a small fraction of it we
  1036. 49:32using observations located here for our
  1037. 49:35treatment group and observations located
  1038. 49:37here for our control group all other
  1039. 49:40observations are
  1040. 49:42ignored the problem becomes even more
  1041. 49:44severe if we want to control for more
  1042. 49:46confounders then the treatment and
  1043. 49:48control groups will become even
  1044. 49:51smaller adding an additional control is
  1045. 49:53like adding in another dimension that
  1046. 49:55want to model and the problem that
  1047. 49:58modeling more Dimensions is hard in
  1048. 50:00terms of diminishing the size of the
  1049. 50:02sample that you can actually use is
  1050. 50:04called curse of
  1051. 50:08dimensionality if you want to move
  1052. 50:10forward we have to overcome the curs of
  1053. 50:13dimensionality but that's a topic for
  1054. 50:15another video thanks for
  1055. 50:17watching

About this transcript

This page contains the full transcript of Causal inference in econometrics by MetricsProf, generated from the public captions YouTube serves with the video. The transcript has 6,151 words across 1,055 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.