Causal inference in econometrics — Transcript
Full transcript
- 0:00welcome to this introductory lecture on
- 0:02causal inference in
- 0:04econometrics there are some big words in
- 0:06the title of this lecture that maybe
- 0:08deserve
- 0:09explanation in particular I want to
- 0:11explain what causal inference
- 0:14means causa means that we're interested
- 0:17in causality that is we believe that in
- 0:20our society in the economic environment
- 0:23that we live in there are relationships
- 0:25of cause and effect by identifying the
- 0:29causes and by manipulating them we as
- 0:31politicians managers or citizens can
- 0:35affect change and move our society in a
- 0:38different direction in a more preferable
- 0:40Direction hopefully inference means that
- 0:44we want to learn about these causal
- 0:45relationships using data of course
- 0:48learning from data is nothing special
- 0:50that's what we as humans do all the time
- 0:53every ray of light that enters your
- 0:56retina is data that you use to interpret
- 1:00the world around you that you use to
- 1:01learn about the world around you what is
- 1:04different in what we're going to
- 1:05accomplish here is that we will learn
- 1:07from the data using rigorous statistical
- 1:10methods from the toolbox of econometrics
- 1:14and they will be rigorous in two
- 1:16important
- 1:17ways first we will be very explicit
- 1:20about the assumptions that we make about
- 1:23the underlying structure of reality um
- 1:26or the world model if you
- 1:28will
- 1:30whenever we are interpreting data we are
- 1:32assuming some underlying structure that
- 1:35we a priori assume to be true that's
- 1:38what we call a model for example if you
- 1:41are seeing a person and then you turn
- 1:43around 360° and you see a person out
- 1:46that looks exactly the same as the
- 1:47person you just saw then you assume that
- 1:49it's the same person the model that you
- 1:51use to interpret the data that you're
- 1:54perceiving incorporates the Assumption
- 1:56of object
- 1:58permanence when we are interpreting data
- 2:01naively the model that we're using to
- 2:03interpret the data is often very
- 2:05implicit and we probably be hardpressed
- 2:07to Define exactly what it
- 2:10is econometrics is different we will
- 2:13always be very deliberate in our choice
- 2:15of model and we will always explicitly
- 2:17State the assumptions that the model
- 2:19makes about the structure of reality
- 2:22this will be very helpful for
- 2:24identifying the limits and weaknesses of
- 2:26a data
- 2:27analysis the second dimension along
- 2:30which we will be very rigorous is
- 2:33statistical error
- 2:35control statistical error control keeps
- 2:38us from overgeneralizing patterns that
- 2:40we see in our
- 2:42data when we look at a small number of
- 2:46data points and that we typically do
- 2:48it's perfectly possible that there are
- 2:50patterns that seem to suggest a cause
- 2:53relationship that doesn't really exist
- 2:56these patterns can randomly occur
- 2:58through our choice of data points that
- 3:00we're looking at and they wouldn't
- 3:01persist in a different choice of data
- 3:04sample to give an example of a CA
- 3:07relationship consider the effect of
- 3:09classroom size on student
- 3:16learning now you may be wondering why
- 3:19are economists interested in education
- 3:22well we as economists consider education
- 3:24as knowledge
- 3:27production education us uses scarce
- 3:31resources such as teacher time or study
- 3:34materials to produce
- 3:36knowledge many parents and policy makers
- 3:39believe that there's a causal
- 3:40relationship between classroom size and
- 3:42student learning specifically they
- 3:45believe that smaller classes lead to
- 3:47Better Learning outcomes for students
- 3:50presumably because in a smaller
- 3:52classroom the teacher has more time to
- 3:55focus on the individual student let's
- 3:57picture a classroom at a middle school
- 4:00we have our
- 4:03teacher and we have a bunch of
- 4:06students if there are only a few
- 4:08students then this is a small classroom
- 4:10if there are many students then this is
- 4:12a large
- 4:13classroom to explain how economists
- 4:16think about cause relationships we have
- 4:18to introduce three important Concepts
- 4:21two of which are very obvious so we
- 4:23start with a less obvious one and this
- 4:25one addresses the question of where we
- 4:27are measuring the cause and effect so
- 4:29we're asking who is
- 4:33affected for our example here there are
- 4:36different choices that would work for
- 4:37example we could say it is the
- 4:39individual student that is affected and
- 4:42we're measuring everything at the
- 4:43student level what here we're making a
- 4:46different choice we're measuring
- 4:47everything at the classroom
- 4:49level in econometric language we say the
- 4:52classroom is our unit of
- 4:56observation now we get to the two
- 4:58obvious Concepts when we talk about
- 5:00cause of relationship the first
- 5:02corresponds to the question what is the
- 5:07CA and the second one corresponds to the
- 5:10question what is
- 5:14caused what is the cause in our example
- 5:18let's write it's classroom
- 5:21size the fancy econometric word for
- 5:24cause is
- 5:26treatment this term is derived from the
- 5:28medical literature where we try to use
- 5:31data to evaluate the efficiency of a
- 5:34medical treatment for economists a
- 5:37treatment doesn't have to be a medical
- 5:39treatment it can be anything that can be
- 5:42manipulated if it's something that
- 5:44cannot be manipulated then as you'll see
- 5:46in a second we will run into some
- 5:48philosophical issues of defining just
- 5:51what a causal effect is more
- 5:53pragmatically why would we be interested
- 5:55in learning the effect of something that
- 5:57we can never change anyway
- 6:00what is cause in our example yes it's
- 6:03student learning let's say student
- 6:05learning is reflected by test
- 6:08scores economists call the variable that
- 6:11is affected by the cause the
- 6:15outcome as you will notice repeatedly
- 6:17throughout this class economists are
- 6:19very lazy often instead of using
- 6:22complicated words we like to use Simple
- 6:25symbols so instead of talking about
- 6:27classroom size or using the more general
- 6:29term treatment we will simply use the
- 6:31symbol capital D and instead of
- 6:35referring to a test score or the outcome
- 6:39we simply use the symbol capital
- 6:42Y these symbols give us a really compact
- 6:45way of representing the causal
- 6:46relationship using a causal diagram in a
- 6:50causal diagram we use arrows to
- 6:52represent causal relationships in our
- 6:55case we believe that
- 6:58d effects
- 7:01why the error here represents the causal
- 7:05effect that the treatment has on the
- 7:08outcome we haven't defined exactly what
- 7:10a causal effect is and it is the most
- 7:13exciting concept that we cover today in
- 7:15fact everything I've done so far was
- 7:17just building towards being able to
- 7:19explain what the causal effect
- 7:22is in the following I want you to think
- 7:24of this classroom as a real classroom
- 7:27that exists somewhere in the world since
- 7:29it's a real classroom it is observable
- 7:32and I know what its classroom size is
- 7:35and what its test score is I've only
- 7:38drawn a few students here let's assume
- 7:40that there are more than three students
- 7:41in this class let's assume that there
- 7:44are 35 students in the
- 7:47class and let's say that the test score
- 7:51is
- 7:5366 to emphasize that this could be a
- 7:56real world classroom I've picked the
- 7:58classroom characteristics from a real
- 8:00world data
- 8:02set this data is from a famous research
- 8:05paper by authors angrist and
- 8:10Levy I've plotted their data class size
- 8:14against test
- 8:16score every Point here represents an
- 8:19observed classroom and look here's our
- 8:22classroom it sits right here it has a
- 8:25class size of 35 and a test score of
- 8:3166 these are exactly the numbers that I
- 8:34gave
- 8:35here suppose that we are able to
- 8:37manipulate classroom size for this
- 8:39classroom this means we can take this
- 8:45classroom and put an exact copy of this
- 8:48classroom down here then we pick a new
- 8:52student little Jimmy down
- 8:55here and we move Jimmy into the class so
- 9:00now there will be 36
- 9:02students this is the only change we make
- 9:05in this classroom everything else we
- 9:08keep
- 9:11equal the Latin term for everything else
- 9:13equal is set is
- 9:17pares this is a term that we will use a
- 9:21lot we imagine this hypothetical
- 9:24classroom in order to define a causal
- 9:27effect it doesn't actually factually
- 9:31exist we call it the counterfactual
- 9:37classroom it is tempting to call the
- 9:40real classroom the factual classroom but
- 9:43no one ever does that what are the
- 9:45characteristics of the counterfactual
- 9:48classroom well it has a class size of
- 9:5136 now I don't want to write D for the
- 9:55class size here because I want to
- 9:57reserve capital letters
- 9:59for variables that represent real values
- 10:03not hypothetical ones so instead I write
- 10:07lower case D equal to
- 10:1036 what is the outcome for the
- 10:13counterfactual
- 10:14classroom our hypothesize cause a
- 10:17relationship suggests that the outcome
- 10:19should depend on the treatment let's
- 10:21make this explicit by writing
- 10:25y as a function of
- 10:29treatment the treatment lower case D can
- 10:33be any value that represents a valid
- 10:35classroom size therefore y of lower case
- 10:38D represents hypothetical outcomes under
- 10:41a certain assigned classroom size in
- 10:45particular if we put
- 10:48in the real observe classroom size we'll
- 10:52get the observed
- 10:54outcome the Y is a function of an
- 10:57hypothetical treatment is called called
- 10:59a potential
- 11:02outcome the outcome under the
- 11:05counterfactual is therefore the
- 11:07potential
- 11:09outcome
- 11:11at a classroom size of 36 since this
- 11:15number only ever becomes relevant in a
- 11:17hypothetical scenario we don't know what
- 11:20it is if it is true that smaller
- 11:22classrooms are good for student learning
- 11:24then this number should be smaller than
- 11:2766 Maybe
- 11:3065 but we don't even know if that
- 11:32conjecture is correct so for all we know
- 11:34this could also be 70 well we don't know
- 11:38what it is regardless of what this
- 11:40number is at least now we have all the
- 11:43tools that we need for a clean
- 11:45definition of the treatment
- 11:48effect the treatment
- 11:51effect is equal to the outcome in the
- 11:54counterfactual scenario this is the
- 11:56outcome where we have added one student
- 12:00to the class size minus the outcome in
- 12:04The observed scenario that is the
- 12:06outcome where class size is equal to
- 12:10Capital
- 12:12D in other words we Define the treatment
- 12:15effect as a simple before after
- 12:20comparison except that the after never
- 12:24really happens it only happens in our
- 12:26head in our thought experiment where we
- 12:28could consider the
- 12:30counterfactual in our example this here
- 12:33works out to be potential outcome at
- 12:3936 minus we don't have to speculate
- 12:42about the observed outcome it's
- 12:4666 the ceterus parabus treatment effect
- 12:50that we've defined here is the causal
- 12:53effect that we interested
- 12:55in so now we've explained what causality
- 12:58means means to us we can move on to
- 13:02inference how can we learn the value of
- 13:04this treatment effect clearly since we
- 13:07know the observed outcome the difficult
- 13:11part is figuring out the counterfactual
- 13:14outcome this is called the
- 13:16identification
- 13:20problem recall our purple classroom from
- 13:24before it has a class size of 35
- 13:27students
- 13:29to figure out the treatment effect we
- 13:31have to consider the counterfactual of a
- 13:34classroom size of 36
- 13:36students I said previously that we don't
- 13:39observe the counterfactual
- 13:41outcome let's think little bit about why
- 13:44that is
- 13:48true why can't we just change the
- 13:50classroom size from 35 to 36 and then
- 13:54observe our counterfactual
- 13:56outcome well first of we as researchers
- 13:59usually don't have the ability to change
- 14:01the classroom size that decision is up
- 14:04to the school principal the school board
- 14:06politicians or whoever decides on
- 14:08classroom size it's usually not
- 14:11researchers but suppose that we have
- 14:14this extraordinary power we can just
- 14:16change the classroom size any way we
- 14:18want it's still not really enough why is
- 14:22that well we haven't really talked about
- 14:24what our outcome is we said test score
- 14:26let's be a little bit more specific
- 14:28let's say this test score at the end of
- 14:31grade seven so we are looking at
- 14:34learning outcome during grade seven
- 14:36right that makes sense so we've observed
- 14:39that for this
- 14:41class now let's
- 14:44change the class size 236 let's suppose
- 14:48we're able to do that how do we get the
- 14:51test score from grade 7 the students
- 14:54have already done grade seven once well
- 14:58suppose we have the power to make them
- 15:00take grade seven again so that we can
- 15:04observe a new learning outcome for grade
- 15:06seven a second learning
- 15:09outcome the problem is that the learning
- 15:11outcome for a classroom size of 36 is
- 15:14not recorded under CIS parab
- 15:17circumstances we have changed a lot of
- 15:19things in addition to classroom size for
- 15:22example since the students have already
- 15:25taken grade seven before they've already
- 15:27studied a lot of topics so they have
- 15:30entered grade seven with a different set
- 15:33of prior knowledge also all the students
- 15:35are older and more
- 15:38mature as you can see we've changed all
- 15:40of these things in addition to changing
- 15:42classroom size this is not what cetus
- 15:45parabus
- 15:47means the only way that I can think of
- 15:50to really record the cetus parabus
- 15:52counterfactual outcome is if we have
- 15:55time travel after the students have
- 15:58taken grade seven the first time around
- 16:01and we've recorded their Baseline
- 16:03learning outcome we travel back in
- 16:07time wipe their memories and make them
- 16:10retake grade seven at the end of the
- 16:13year we record the counterfactual
- 16:15learning outcome yeah that's the only
- 16:18plan I have and it's not really feasible
- 16:20because I haven't really figured out
- 16:21time travel no good news on that
- 16:24front I hope that you're convinced now
- 16:27that we can't observe the counterfactual
- 16:30outcome let's focus on things that we
- 16:32can actually observe for this classroom
- 16:35we don't observe a class size of 36 but
- 16:38there may be a different classroom where
- 16:40we actually have a class size of 36
- 16:43let's say this is the green classroom it
- 16:46also has lots of
- 16:49students has a teacher a different
- 16:51teacher
- 16:53obviously and this class has little
- 16:56Jimmy in it it has a class class size of
- 17:0136 recall that class size is our
- 17:04treatment so this classroom receives
- 17:07more treatment than this
- 17:10classroom we say it receives higher
- 17:13treatment intensity so this is the high
- 17:16treatment intensity classroom and this
- 17:20is the low treatment intensity
- 17:27classroom in a lot of treatment effect
- 17:30settings in medicine for example
- 17:32treatment is binary that means you
- 17:35either get the full treatment or you
- 17:36don't get any treatment at all for
- 17:39example you would be in the high
- 17:42intensity group if you get all three
- 17:44doses of a three dose vaccine and you
- 17:46would be in the low treatment intensity
- 17:48group if you don't get the vaccine at
- 17:50all in such a setting we say that a high
- 17:53treatment intensity unit is
- 17:57treated and and that a low treatment
- 17:59intensity unit is a control
- 18:02unit let's use these terms also
- 18:05here for both of these classrooms we
- 18:08observe learning
- 18:12outcomes so how do we get at the
- 18:16treatment
- 18:20effect a natural approach is to subtract
- 18:23the control outcome from the treated
- 18:27outcome
- 18:29this is not exactly equal to the
- 18:30definition of the cause effect so we
- 18:32shouldn't put an equal sign here we just
- 18:35want to indicate that this is a guess
- 18:37based on our data of the treatment
- 18:40effect a guess based on data is called
- 18:42an
- 18:44estimate to indicate that a value
- 18:47depends on data we often put a head on
- 18:50it to check whether this is really
- 18:53estimating the treatment effect let's
- 18:55make it look a little more like our
- 18:56definition of the treatment effect fact
- 18:59that is let's make it look more like
- 19:02this the treated outcome is the
- 19:05potential
- 19:07outcome at a class size of
- 19:1136 the control outcome is the potential
- 19:14outcome at a class size of
- 19:2135 36 is equal to the Baseline class
- 19:25size +
- 19:27one
- 19:30and 35 obviously is just equal to the
- 19:32Baseline class
- 19:34size now does this look
- 19:38like
- 19:40this almost but not quite can you see
- 19:44what's gone wrong yes exactly here we
- 19:47have the potential outcome function for
- 19:49the green class and not for the purple
- 19:53class the potential outcome functions
- 19:56don't necessarily have to be the same
- 19:59to explain why let's think more
- 20:01carefully about the economic environment
- 20:03that we are studying and let's do this
- 20:05using a caus a
- 20:09diagram we are studying test
- 20:14scores this is our outcome
- 20:17variable we're interested in the effect
- 20:19of class
- 20:22size this is our
- 20:24treatment our conjecture is that there
- 20:27is a cause of relationship between these
- 20:29two and we indicate the cause
- 20:31relationship by an
- 20:34error clearly class size is not the only
- 20:38factor that has an effect on test
- 20:41score consider for example the socio
- 20:43economic background of the students in
- 20:45the
- 20:47classroom for example you may imagine
- 20:50that a student who has Highly Educated
- 20:53parents is more likely than other
- 20:55students to receive help with homework
- 20:59we expect socio economic background to
- 21:01also have causal effect on the test
- 21:05score social economic background has
- 21:07many facets in the data that I showed
- 21:10you earlier it is measured by a variable
- 21:12called PD
- 21:15Index this stands for percentage
- 21:19disadvantaged and gives the percentage
- 21:22of disadvantaged students where
- 21:23disadvantage is defined by several
- 21:25characteristics of the parents
- 21:28the composition of socio economic
- 21:30backgrounds in the classroom is
- 21:32obviously determined by The Neighborhood
- 21:34that the school is
- 21:38in neighborhood may also affect class
- 21:41size for example if the school is
- 21:43located in a rural sparsely populated
- 21:46area there may be only a few students
- 21:49entering School each year only enough
- 21:51students to have one small class so
- 21:54neighborhood can affect the size of the
- 21:56student cohort
- 22:00which in turn affects class
- 22:04size the PD index is also observed by
- 22:07politicians who may use it and actually
- 22:10do use it to allocate more resources to
- 22:12schools with high percentage of
- 22:14disadvantaged students for example they
- 22:17may have smaller classes in such
- 22:21areas I hope that I've convinced you
- 22:24that small classrooms and large
- 22:26classrooms may be very different in in
- 22:28terms of student composition and that
- 22:30these students May react very
- 22:31differently to differently sized
- 22:34classrooms therefore we shouldn't expect
- 22:36all classrooms to share the same
- 22:39potential outcome
- 22:41function there may be systematic
- 22:44differences in potential outcome
- 22:45functions or well unsystematic
- 22:48ones let's start with the systematic
- 22:51ones assume that smaller classes tend to
- 22:56face a different PD and index than laric
- 23:00classes we've already discussed that the
- 23:03PD index has a connection to the kind of
- 23:08potential outcome function that we're
- 23:10assuming this creates a systematic
- 23:13relationship between the class size that
- 23:15we are seeing and the kind of potential
- 23:18outcome function that we should be
- 23:22expecting a variable like PD index that
- 23:25is lurking in the background and the
- 23:27systematic changing the potential
- 23:29outcome functions is called a
- 23:34confounder if we observe this confounder
- 23:37then we should be able to deal with it
- 23:39by just slightly changing the potential
- 23:42outcome
- 23:46framework let's quickly recap the
- 23:49variables that we
- 23:52observe we observe treatment
- 23:56D this is class
- 23:59size and we observe outcome
- 24:02y this is the test
- 24:06score now we've added a new observable
- 24:08variable the PD
- 24:11index and this is the confounder in our
- 24:15example and we abbreviate it by the
- 24:17symbol
- 24:19X the potential outcome defines what the
- 24:23outcome will be under a hypothetical
- 24:26treatment assignment of small
- 24:29D so this is a classroom size that's
- 24:32potentially different from The observed
- 24:34classroom size for every possible
- 24:37treatment intensity we assign a
- 24:39potential
- 24:42outcome we previously called this y of
- 24:46D now we've added the confounder X and
- 24:49we treat X similar to D that is we're
- 24:53thinking about different levels of X
- 24:56that are conceivable
- 24:58confounder level lowercase
- 25:02x we argued that the confounder should
- 25:06have an impact on the potential outcome
- 25:08this not apparent in this notation so
- 25:10let's change that and we're letting the
- 25:13potential outcome depend on D as well as
- 25:18X there you go that wasn't too hard now
- 25:20we've Incorporated confounders in the
- 25:23potential outcome
- 25:25framework recall our perp classroom it
- 25:28was our baseline or control
- 25:32classroom it has a classroom size of
- 25:3635 it has to have a value of the PD
- 25:38index let's just say that is 20 and it
- 25:43has an outcome which is the potential
- 25:47outcome evaluated
- 25:50at 35 classroom size and 20 PD index the
- 25:56blue classroom was our treated
- 26:01classroom it had one more student it
- 26:04certainly has a PD value let's just say
- 26:07um
- 26:0935 and an outcome of the potential
- 26:13outcome evaluated at 36 and
- 26:1835 now we see clearly when we're
- 26:20comparing the two we're not just
- 26:24comparing different classroom sizes we
- 26:27also comparing different PD
- 26:30indices that's no good from this
- 26:33comparison we don't expect to learn much
- 26:34about the pure effect of a larger
- 26:38classroom but don't despair this seems
- 26:41easy to fix we've just chosen the wrong
- 26:44blue classroom recall that the blue
- 26:46classroom was supposed to be a stand in
- 26:49for the counterfactual purple classro
- 26:52that we don't
- 26:53observe but the counterfactual purle
- 26:55classroom is a cus par
- 26:58classroom where the X shouldn't be
- 27:01changed but here actually we see
- 27:04different axes so why don't we choose a
- 27:07blue classroom that has the same a as
- 27:09the purple
- 27:13classroom there we go this is already
- 27:15looking much
- 27:17better now our treatment effect
- 27:22estimate is treated y minus control y
- 27:28so is equal to potential outcome Y at
- 27:3536 20 minus potential outcome at
- 27:4235
- 27:4420 and we can rewrite this as
- 27:51before and key is of course that now
- 27:55here we have the same X let's let's try
- 27:57to implement this strategy for
- 27:59estimating the treatment effect using
- 28:01our
- 28:03data here I've plotted our data this
- 28:06time against the dimensions of percent
- 28:08disadvantaged so this is our
- 28:12confounder and class size this is our
- 28:18treatment here's our example classroom
- 28:21it's still here going
- 28:23strong and now we're looking for
- 28:26potential in Sample
- 28:28counterfactuals so our classroom has a
- 28:30class size of
- 28:3235 so we want to look at classrooms that
- 28:34have a class size of 36 so all of
- 28:38these also they should have a similar
- 28:41percentage
- 28:43disadvantage so I'm looking in this
- 28:46range this gives us candidates for the
- 28:49blue classroom that are
- 28:51located
- 28:53here that's great if everything works
- 28:56out as we anticipate we should be able
- 28:58to choose any of these classrooms and
- 29:00then just use their y as the blue
- 29:04Y and then we can estimate the treatment
- 29:07effect where we are controlling for the
- 29:10effect of the
- 29:12compounder so let's zoom in on these
- 29:15classrooms I have them all
- 29:18here they have percentage disadvantage
- 29:21between 17 and
- 29:2323 they all have a classroom size of of
- 29:2836 here's our classroom for comparison
- 29:32now here those are the test scores and
- 29:35oh no there's a problem they all have
- 29:40different test
- 29:42scores even those that have exactly the
- 29:45same percentage disadvantaged have
- 29:49different test
- 29:51scores so what should we choose as the
- 29:54blue y our choice will have huge cont
- 29:57consequences if we choose one of these
- 30:00for example we'll estimate a positive
- 30:02treatment effect if we choose one of
- 30:04these we will estimate a negative
- 30:06treatment effect how do we make the
- 30:09right
- 30:10choice one thing that we have definitely
- 30:13learned is
- 30:15that treatment and confounder don't
- 30:19fully determine the outcome there are
- 30:21other things that matter and that shift
- 30:24around the potential outcomes now this
- 30:26could be an additional confounder that
- 30:29we haven't taken into
- 30:30considerations but I want to propose
- 30:32something else you remember when I said
- 30:36there are systematic and unsystematic
- 30:38differences in potential outcomes where
- 30:41the confounder cost differences were the
- 30:44systematic ones now let's look at the
- 30:47unsystematic
- 30:48ones to talk about unsystematic
- 30:50differences we have to recall a very old
- 30:53concept from statistics the concept of
- 30:56an error term
- 31:00we are now about to impose some
- 31:02restrictions on how we think test scores
- 31:05are produced that is we're going to make
- 31:07some model
- 31:08assumptions as I promised you before
- 31:11whenever we make assumptions we are very
- 31:13explicit about
- 31:16that oh yeah now I see this was supposed
- 31:19to be word model we're making model
- 31:22assumptions let me just fix that real
- 31:26quick
- 31:27here we
- 31:28go so what kind of assumptions are we
- 31:31imposing as before we assume that
- 31:34outcome is given by the potential
- 31:37outcome evaluated at The observed
- 31:40treatment and observed confounder this
- 31:43in itself is not restrictive at all but
- 31:47now we going to make some additional
- 31:48assumptions we assume that this is equal
- 31:50to some function M of
- 31:55treatment and conf
- 31:58founder and this function is the same
- 32:01for all
- 32:03classrooms plus and aror term
- 32:08U once we assume that this part is the
- 32:11same for all classrooms you see
- 32:13immediately that we need an additional
- 32:15component to explain that all of these
- 32:19here don't have the same test score in
- 32:22our new model of test score
- 32:24production these differences are
- 32:27explained by these classrooms having
- 32:30different error
- 32:33terms let's just note that U is our
- 32:37error
- 32:39term and the idea of an error term is
- 32:42that it is completely
- 32:43unsystematic statisticians say it is
- 32:47random
- 32:51noise so for some classrooms this will
- 32:53be positive for others it will be
- 32:56negative and if we average over
- 32:58classrooms then the average will be
- 33:00close to
- 33:02zero this gives us a completely new idea
- 33:05for choosing the blue classroom the blue
- 33:07classroom shouldn't correspond to any
- 33:09individual classroom it should be an
- 33:12average of appropriate comparison
- 33:16classrooms in the average the effect of
- 33:18the error term will wash
- 33:21out for example we could average over
- 33:24all of these classrooms they seem to be
- 33:26good compar in
- 33:28classrooms in preparation for defining
- 33:30the average let's label them let's call
- 33:33this the I equal 1 classroom the I equal
- 33:362 classroom I equal 3 classroom and so
- 33:43on to identify values corresponding to a
- 33:46classroom we give all values in I
- 33:48subscript for
- 33:52example we would denote the test scorer
- 33:55in the first classroom here here as y
- 33:59subscript
- 34:021 we're counting 1 2 3 four five
- 34:05candidates for the blue classroom let's
- 34:09write this as blue n equal
- 34:125 now we have enough notation to
- 34:14describe our idea that the average error
- 34:16term is
- 34:17zero to formally write down the average
- 34:20over the error terms we
- 34:24sum over all of the five classrooms
- 34:28starting with the first classroom up to
- 34:31the fifth classroom right this was our
- 34:35blue
- 34:36n we sum the error
- 34:39terms and now we divide by the number of
- 34:44classrooms blue
- 34:46n we are hoping that this will be
- 34:49approximately equal to
- 34:52zero before we move on I have to correct
- 34:55some unfortunate notation
- 34:57I chose blue here as a color for the
- 35:00error term but this is not meant to
- 35:02indicate that the error term is only
- 35:03relevant for the blue classrooms so let
- 35:06me change the color of the error
- 35:12term so this is already much
- 35:16better when is our hope that this
- 35:18average is zero
- 35:21Justified while we're averaging over
- 35:24values in our sample so we have to ask
- 35:27where our sample is coming
- 35:29from we think of our sample as coming
- 35:32from a large collection of
- 35:34classrooms that is called the
- 35:39population the population contains all
- 35:42the classrooms that we think follow the
- 35:44same economic rules of knowledge
- 35:46production for example this could be all
- 35:48the classrooms in the same country or in
- 35:50the same educational
- 35:52system we collect our sample by
- 35:54selecting classrooms from the population
- 35:59and most of the time we do so randomly
- 36:01so this is called random
- 36:05sampling let's assume that we sample 1 2
- 36:10up
- 36:11to n
- 36:14classrooms we observe these classrooms
- 36:16and can record their values of test
- 36:20score class size and PD
- 36:25index
- 36:27this is data that we can say for example
- 36:30in an Excel
- 36:31sheet under what circumstances can we
- 36:34expect averages on the sample to be
- 36:36equal for example to
- 36:39zero the law of large numbers from
- 36:44statistics says that population
- 36:50averages are close to sample
- 36:55averages at least if the sample size n
- 36:59is
- 37:03large for the law of large numbers to
- 37:05hold it's also important that we sample
- 37:09randomly we can have a reasonable
- 37:12expectation that the sample average is
- 37:13close to zero if the corresponding
- 37:16population average is close to
- 37:18zero population averages can be written
- 37:21as expectations we'll explain why at a
- 37:24later point so the corresponding
- 37:27population average can be written as the
- 37:33expectation of the error term in the
- 37:36subpopulation of classrooms that have a
- 37:39class
- 37:40size equal to
- 37:4336 and face a PD
- 37:47index equal to
- 37:5020 let's assume that this population
- 37:53average is equal to zero
- 37:58by the law of large numbers the sample
- 38:00average inherits this Zero from the
- 38:04population
- 38:06average this seems like a good
- 38:08assumption now but as you'll see in a
- 38:10second we need something a little bit
- 38:11more General so for now let's assume
- 38:14that it doesn't hold for this
- 38:16subpopulation but for all
- 38:20subpopulations it holds for arbitrary
- 38:23lowercase D and arbitrary lowercase x
- 38:27later we would call this kind of
- 38:28assumption an exogeneity
- 38:34assumption another good name for it
- 38:36would be a balancing assumption because
- 38:38it assumes that the error terms in the
- 38:42subpopulations are well balanced in the
- 38:44sense that they average out to
- 38:52zero previously we estimated the
- 38:54treatment effect using a single blue
- 38:58classroom and its observed learning
- 39:02outcome now we're going to modify this
- 39:04identification strategy instead of using
- 39:07a single classroom we're using an
- 39:09average over
- 39:12classrooms the average sums over all
- 39:16suitable
- 39:19candidates takes the observed test score
- 39:22and then we divide by the number of
- 39:25observations
- 39:28before we can discuss whether our
- 39:29estimation strategy based on this
- 39:31quantity actually works we have to think
- 39:34about how to interpret causal effects
- 39:37now that we have an error
- 39:40term when we consider a set's parabus
- 39:43change what happens to the error term
- 39:46what is the error term anyway in
- 39:48economic
- 39:49terms for every given classroom U
- 39:52describes the effect of factors other
- 39:55than D and acts that are present in the
- 39:57classroom and that affect knowledge
- 39:59production in that classroom for example
- 40:02if a class has a particularly clever
- 40:04student that would enter through
- 40:08you we said you is like random noise but
- 40:11that's not very helpful for
- 40:12understanding you economically for every
- 40:15given classroom you is not random at all
- 40:18it's just one facet that describes the
- 40:20circumstances in the classroom random
- 40:23noise just means when we compare the
- 40:25circumstances between classrooms they
- 40:27seem
- 40:29random when we think about a setus parus
- 40:32change we only change in D we're not
- 40:35changing X and we're not changing
- 40:38U what does this mean for our treatment
- 40:47effect the treatment
- 40:50effect compares the outcome in the
- 40:53counterfactual
- 40:55word
- 40:57to the actually observed
- 41:00outcome we can now plug in our model for
- 41:02the potential
- 41:04outcomes so we have M of D +
- 41:101 and
- 41:13X plus our error term
- 41:16U and now we subtracting this part
- 41:21minus there's M this time evaluated at D
- 41:25and X
- 41:29plus the error
- 41:31term the error term shows up here and
- 41:34here so we'll just cancel and we're left
- 41:37with the difference between these M
- 41:48functions this is the treatment effect
- 41:50that we're trying to
- 41:52estimate now let's move on to treatment
- 41:55effect estimation
- 42:02let's start by considering what Y Bar is
- 42:05under our model
- 42:07assumptions Y Bar is the average over
- 42:11all the blue
- 42:15classrooms so y goes from one to the
- 42:19blue
- 42:20n and we're dividing this sum by the
- 42:24blue n
- 42:27so this will be the
- 42:32average over and
- 42:35now I'm plugging in the model here first
- 42:39we have the M function evaluated at the
- 42:45treatment and the
- 42:49confounder plus the arrow
- 42:54term now we've chosen the blue
- 42:56classrooms so that the treatment is
- 43:00always
- 43:0236 and the confounder is always
- 43:0720 therefore this is a fixed number and
- 43:09we'll just shift the average by this
- 43:12amount we can move this outside of the
- 43:19average and here get rid of the
- 43:25parenthesis
- 43:26now this here will be approximately Zero
- 43:29by the law of large
- 43:32numbers therefore the whole thing will
- 43:35be
- 43:37approximately the M function at 36 and
- 43:4120 previously we estimated the treatment
- 43:47effect by the difference between the
- 43:50blue
- 43:51classroom and the purple classroom now
- 43:54our plan was to modify this edification
- 43:56strategy by replacing the single
- 43:59classroom by the average of classrooms
- 44:03here what will this get us well this
- 44:05will get us
- 44:10approximately M at
- 44:1336 20 minus and now I'm plugging in the
- 44:18model for this
- 44:19y the m at
- 44:2435 20
- 44:28plus the error term in the purple
- 44:32classroom this doesn't give us the
- 44:34treatment
- 44:35effect that we figured out here because
- 44:39we end up with this additional error
- 44:42term how do we get rid of the error term
- 44:46well the same way that we got rid of the
- 44:47error term for the blue classrooms
- 44:50instead of using a single control
- 44:52classroom we're using an average of
- 44:55several control classrooms that have
- 44:57similar level of classroom size and
- 45:07confer suppose that we can find purple n
- 45:10such
- 45:16classrooms we can define a similar
- 45:18average as we had here and then apply
- 45:22very similar arguments to show that this
- 45:24is equal to
- 45:26the M function but this time at 35 the
- 45:31smaller class size and confounder level
- 45:3520 plus an average of error
- 45:44terms this again will be close to Zero
- 45:47by the law of large
- 45:49numbers therefore this will be
- 45:51approximately equal to M at 35
- 45:5720 now we can replace the single
- 46:00classroom by this
- 46:02average we get rid of this
- 46:07U now we have this difference of M
- 46:11functions and just to be clear the first
- 46:14M
- 46:15function is evaluated at per D + 1 and
- 46:21confounded level equal to what we see in
- 46:24the purple class
- 46:27and then the second M function is at
- 46:31D comma
- 46:34X this is exactly the expression that we
- 46:37figured out for the treatment
- 46:44effect finally we have figured out a
- 46:47viable identification
- 46:49strategy our estimate is close to the
- 46:52thing that we're trying to
- 46:54estimate let's take this edification
- 46:56strategy to the
- 46:57data first we need all the purple
- 47:00classrooms that is the control
- 47:05classrooms they have a class size of 35
- 47:09and a PD
- 47:10index of
- 47:1420 here we go we also need all the blue
- 47:19classrooms that is the treated
- 47:21classrooms they have a class size of 36
- 47:24and also a p of
- 47:2720 so now we have all the data that we
- 47:30need to compute our treatment effect
- 47:32estimate I'll leave the computation up
- 47:34to
- 47:35you before I finish this video I want to
- 47:37point out two issues with our estimation
- 47:40strategy
- 47:42here the first issue is what if we have
- 47:45an additional confounder maybe let's
- 47:47call it
- 47:49X2 where is the effect of X2 modeled in
- 47:52this model well it's not here it's not
- 47:54here so it's got to be here so the
- 47:58confounder X2 would enter the model
- 48:00through U but we've assumed that U is
- 48:04balanced in the sense of satisfying this
- 48:06Exogen
- 48:08assumption now the effect of confound us
- 48:10is typically not balanced so in this
- 48:12case we would have trouble justifying
- 48:15the Assumption of course what we can do
- 48:17if we observe X2 at least is ADD X2 as
- 48:21an additional term in this mamp function
- 48:23in this case we say that we control for
- 48:25X2
- 48:27however this may lead to additional
- 48:29complications but let's first talk about
- 48:31issue
- 48:32two our argument relied on the law of
- 48:35large numbers we need population
- 48:38averages to be reflected in Sample
- 48:40averages this works only if n is large
- 48:43that is if we are taking a sample
- 48:45average over many
- 48:49values here we are taking averages over
- 48:52only very few
- 48:54observations for our control group we
- 48:56have 1 2 3 observations and for the
- 48:59treatment group we have four
- 49:01observations these numbers are way too
- 49:03small to ensure that the law of large
- 49:04numbers
- 49:05works now think back to issue one if we
- 49:09Define the control and treatment groups
- 49:11even more narrowly by requiring that
- 49:14they not only share the same PD index
- 49:16but also the value of an additional
- 49:18confounder we make these groups even
- 49:20smaller and take averages over even
- 49:22fewer
- 49:23observations this picture is a great
- 49:25illustration of our problem we initially
- 49:28start out with a lot of data but end up
- 49:30using only a small fraction of it we
- 49:32using observations located here for our
- 49:35treatment group and observations located
- 49:37here for our control group all other
- 49:40observations are
- 49:42ignored the problem becomes even more
- 49:44severe if we want to control for more
- 49:46confounders then the treatment and
- 49:48control groups will become even
- 49:51smaller adding an additional control is
- 49:53like adding in another dimension that
- 49:55want to model and the problem that
- 49:58modeling more Dimensions is hard in
- 50:00terms of diminishing the size of the
- 50:02sample that you can actually use is
- 50:04called curse of
- 50:08dimensionality if you want to move
- 50:10forward we have to overcome the curs of
- 50:13dimensionality but that's a topic for
- 50:15another video thanks for
- 50:17watching
About this transcript
This page contains the full transcript of Causal inference in econometrics by MetricsProf, generated from the public captions YouTube serves with the video. The transcript has 6,151 words across 1,055 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.
What you can do with it
Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.
Free YouTube transcript tool
YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.