YT YouTube2Text

YouTube2Text → Transcripts → Klinsky Lecture: Eric Lander

Klinsky Lecture: Eric Lander — Transcript

by Harvard Law School · 9,220 words · 1,356 segments · language en · Watch on YouTube

Open in transcriber Summarize with AI Download SRT/VTT

Full transcript

0:03okay welcome welcome everybody I think
0:07people will continue to filter in but I
0:08think it's time to start so that we can
0:10get the full hour so in to 2013 Steven
0:14Klinsky from the class of 81 and his
0:17wife Maureen Klinsky endowed the Steven
0:19and Maureen Klinsky professorship of
0:21practice for leadership and progress the
0:24first endowed professorship of practice
0:27at Harvard Law School it brings visiting
0:29leaders from a wide range of fields
0:30beyond the law to our campus to teach to
0:33inspire and to broaden perspectives both
0:35at HLS and in the wider University his
0:39eminence Timothy cardinal Dolan the
0:41Archbishop of New York delivered the
0:43first lecture under the auspices of the
0:45percept professorship and previous
0:47holders of the Klinsky professorship
0:49include Julius Genachowski class of 91
0:52the former chair of the FCC former
0:55congresswoman Jane Harman class of 69
0:57and Chris Kelley class of 97 the first
1:00chief privacy officer general counsel
1:02and head of global public policy for
1:04Facebook so it's now my great pleasure
1:08to introduce dr. Eric Lander our Steven
1:12and Maureen Klinsky visiting professor
1:14of practice for leadership and progress
1:16this year this semester he is teaching
1:19the reading group entitled science and
1:21law in the federal government issues in
1:23challenges dr. Lander is president and
1:27founding director of the Broad Institute
1:29at MIT and Harvard a geneticist
1:33molecular biologist and mathematician he
1:36has played a pirate pioneering role in
1:38the reading understanding and biomedical
1:40application of the human genome he was
1:42also principal leader of the human
1:45genome project dr. Lander is a professor
1:49of biology at MIT and a professor of
1:51systems biology at Harvard Medical
1:53School from 2009 to 2017 he served as
1:57co-chair of President the President's
1:59Council of Advisors on science and
2:01technology for President Obama
2:04dr. Saunders awards and honors are so
2:07numerous that we are going
2:11to acknowledge that they are too
2:14numerous to read out without cutting
2:17deeply into our question and answer time
2:19and today we will hear from him about
2:22science and law conversations we're not
2:25having so without further delay let's
2:27give an enthusiastic welcome well thank
2:35you very much Dean Manning it is a
2:37pleasure to be here
2:40there we go let me start with the thanks
2:43first Stephen and Maureen Klinsky for
2:47endowing this wonderful professorship of
2:51the practice it's really wonderful to
2:55create a role for people who are
2:57completely unqualified to teach at the
2:59law school to come to each of the law
3:00school but the requirement is it's four
3:02fields other than the law and I am
3:06absolutely tickled to be doing this and
3:09to be here and I am teaching a reading
3:11group and we've had five of our six
3:13meetings I'll tell you a little bit more
3:15about that in a moment but it has been
3:17tremendous to do that I want to thank
3:19our former dean Martha Minow because
3:23Martha corralled me one day and
3:27high-rise on Concord Ave and we were
3:32both on line and she turns I just I was
3:34thinking about have you come teach at
3:35the law school and I said I accept and
3:37so that's how this that's how this arose
3:41and to Dean Manning for having me here
3:44and welcome me welcoming me and just you
3:48know all of all of the warm welcome I've
3:51received for everybody here at the law
3:53school so I suppose that in addition to
3:58teaching the reading group the purpose
4:00of giving the Klinsky lecture is to give
4:03some accounting of yourself like who are
4:06you why are you here and and what do you
4:08what do you expect to be doing here and
4:10so I will start with a very brief
4:12introduction to me I indeed have no
4:16formal connection to law but we'll see
4:19in a moment I come from Brooklyn New
4:21York
4:23I went to Stuyvesant High School was an
4:25undergraduate at Princeton and then did
4:27my PhD in pure mathematics at Oxford
4:30having decided not to do pure
4:33mathematics as a career I somehow talked
4:36my way into teaching on the faculty of
4:38the Harvard Business School where I
4:40taught managerial economics for a number
4:42of years and having decided that manager
4:44your economics was not the career I
4:47wished either in the business school
4:49being very tolerant if you talk to your
4:51classes well I used the time to also
4:53pick up molecular biology on street
4:55corners this being a town with very good
4:58street corners for picking up molecular
5:00biology on and so drifted into molecular
5:04genetics over the course of the 1980s
5:07eventually got an appointment at the
5:09Whitehead Institute and MIT where we
5:13built out what became the first of the
5:15Centers under the Human Genome Project
5:16and during 1990s worked on the Human
5:18Genome Project and I'm proud to say that
5:21you know here in Cambridge we made the
5:23leading contribution to that great
5:25international project and after that was
5:28done and recognizing that a fantastic
5:30collaboration had grown up unofficially
5:33in the course of the Human Genome
5:34Project between Harvard and MIT and five
5:36teaching hospitals we somehow managed to
5:39create the Broad Institute of MIT and
5:41Harvard devoted to genomic medicine and
5:44it involves about 4,400 people across
5:47all of those different institutions in a
5:50variety of different ways it's a semi
5:52porous membrane with people coming in
5:54and out of it and it's a it's a really
5:57fun place when you're in Kendall Square
5:58it's right opposite legals come stop in
6:01sometime now I have no formal connection
6:05to law but I have a deep interest and
6:09affection for law in addition to my
6:14parents having been lawyers my wife
6:16having been a lawyer and and all I also
6:19have somehow stumbled into law on
6:22various occasions in the late 1980s I
6:25got involved within what turned out to
6:27be the first serious case in introducing
6:30DNA fingerprinting into criminal courts
6:33in the United States and
6:35not not coincidentally the first case in
6:38which DNA fingerprinting was rejected as
6:40being unacceptably performed it's a
6:44famous case it's it's the people of New
6:47York V Castro and it was a real great
6:50introduction to me for many things about
6:54how science can play powerful roles in
6:58the legal system and how if powerful
7:01roles are entailed to high quality
7:02standards they can do more harm than
7:04good
7:05and so that's turned out well DNA
7:07fingerprinting has turned out to be
7:08after a whole bunch of hiccups a very
7:11powerful and reliable tool in the courts
7:13and in the course of that some years
7:16later I also joined the board of the
7:18Innocence Project which emerged from
7:20that particular case I also from time to
7:24time am known to write amicus briefs in
7:27Supreme Court cases or at least two of
7:28them and we'll touch on both of them
7:30here but I enjoy you know dipping in
7:34occasionally where it seems like science
7:36might have something useful to say and
7:38then as Dean Manning said probably my my
7:42deepest involvement in all of these
7:44things not law per se but the touch on
7:46law was my work for eight years together
7:49with that 20-some odd other amazing
7:52people including John Holdren who is
7:53here at Harvard and at the Kennedy
7:55School on the President's Council of
7:57Advisors on science and technology which
8:00is the sole advisory group group to the
8:03President on all matters of Science and
8:06Technology that cut across agencies and
8:08cut across the federal government and
8:10that was an amazing opportunity we
8:13worked on 39 separate reports on a huge
8:15variety of topics and I gotta say what
8:18an education that was and a fair amount
8:20of work well you have a day job and
8:22everybody on this group had day jobs but
8:24everybody viewed it is a tremendous
8:26honor to be able to to work on these
8:28things and as is the case with
8:30scientists and technologists in
8:31producing 39 reports and these are like
8:33real reports the group actually wrote
8:36every word this was none of this was
8:38given out to staff because scientists
8:40and technologists like asking you guys
8:42you know deeply believe that the words
8:44have to be exactly what we mean
8:45be so that's that's I suppose vaguely
8:50why Martha thought it would be fun to
8:52invite me to come do this and what I've
8:55chosen to do with this reading group is
8:59essentially to talk about whatever I
9:03happen to be interested in which as the
9:06in Manning acknowledged is the best way
9:08to teach a course because then you're
9:09passionate about whatever it is you're
9:11talking about and so the subjects that
9:13we have or will cover were partisan
9:17gerrymandering this case Gil that is
9:21before the Supreme Court right now
9:23and Charles freed came and joined me for
9:25that first class and we talked about the
9:27case both Charles and I had written
9:30amicus briefs in that case and we have
9:33just twelve amazing students I think
9:36based on the representation in the class
9:37almost everybody at the Harvard Law
9:39School has deep backgrounds in science
9:41technology medicine you know the the
9:44average is about twenty five percent of
9:46the people have or are getting an MD you
9:49know for people who have experiences
9:51patent agents I just salute you for this
9:54average across the the whole law school
9:56as judged from that sample the second
9:59class was on forensic science and
10:02criminal courts and as I've already said
10:03that's a topic that's been of interest
10:05to me since the 1980s and continued with
10:09a report that we wrote on PCAST on
10:11forensic science in the criminal courts
10:13which I'll talk about the third meeting
10:17and for that one judge Gertner nancy
10:20gertner joined us for that course and
10:22that was really fantastic the third one
10:25was on gene patenting this case before
10:27the Supreme Court in 2013 on the
10:31patentability of the breast cancer gene
10:34which was a fascinating case in which I
10:38also had written an amicus and we were
10:40joined that day by Tanya Simoncelli the
10:43woman who actually thought up the idea
10:45of challenging gene patents when she was
10:47a scientific advisory to the ACLU just
10:50last sorry in the beginning of March we
10:53had a case on the regulation of hearing
10:55aids a topic I won't
10:57talk about today but it was P castes
11:00most effective report per page that you
11:04can imagine maybe at all it was an 11
11:08page report that called for the creation
11:10of an over-the-counter class of basic
11:12hearing aids laid out what the issues
11:15were of how well-meaning ideas in the
11:171970s had turned into a small oligopoly
11:21that had made a pair of hearing aids
11:24cost 10 times more than an iPad and it's
11:27tough that's what they caused and it's
11:29tough to imagine that the technology
11:32underlying a pair of hearing aids
11:33justifies that premium and it's a very
11:36slow innovation cycle etc we sort of
11:39pulled the thread on how regulation had
11:41affected all of those things and
11:42suggested that it would be medically
11:44safe to declare a certain class as
11:47over-the-counter and we did and
11:49remarkably through the efforts of
11:52Senator Warren our own senator here and
11:55Senator Grassley from from Iowa it made
12:00it into law and so this will in fact
12:01become the case that we will have
12:04over-the-counter hearing aids the FDA is
12:06instructed to be developing that class I
12:08guess they now have another two and a
12:10half years to go but it will make a huge
12:12difference and should bring prices down
12:14from at $6,000 to I'm guessing a couple
12:17hundred bucks or much less so in any
12:20case it was it was interesting
12:21adventures and we talked about the ups
12:23and downs of trying to get such a thing
12:25through last week on Tuesday we talked
12:29about artificial intelligence including
12:31sentencing recommendations and we'll
12:33talk about that a little bit today and
12:34our last class next Tuesday which will
12:37hold over the Broad Institute will be on
12:39genome editing and synthetic biology and
12:41so we've covered a wide range of cases
12:44looking at Supreme Court cases on the
12:46one hand and looking at underlying
12:49scientific issues on the other looking
12:52at issues of fact and looking at issues
12:54of advocacy and looking at how science
12:56and law may come together that's what we
12:59been up to and I figured I had to give
13:00an accounting of that publicly somewhere
13:02so that's now been done
13:04but of course in incoming and saying I'm
13:06I'm willing to and excited about
13:10teaching it has to be admitted for every
13:13teacher that the reason you teach is to
13:15learn as well and I saw this as a great
13:18opportunity to learn by engaging with
13:21twelve super smart students here and to
13:25engage with six really interesting
13:27topics and think afresh about them in a
13:30way that I wouldn't otherwise if I
13:32didn't have the responsibility of trying
13:34to teach a reading group so we're not
13:38done yet
13:39and my thoughts haven't settled but I do
13:41want to talk a little bit both at the
13:44beginning and then I'd like to return in
13:46a question and answer discussion
13:47hopefully at the end so the issue of the
13:51conversations we're not having between
13:53law and science when scientists because
13:57this is much on my mind you'll forgive
13:59me that the thoughts are still only
14:01partially formed but I hope I'm gonna
14:03provoke further conversation and maybe
14:05some some venues where we really can
14:08talk about these things further it's
14:10always said by laws so different than
14:12science because you know look the
14:15processes the law uses the adversarial
14:18process is so different than what goes
14:20on in science you know you get two
14:24people taking polar opposite opinions
14:27and they they try to advocate for these
14:29two alternatives and somehow truth is
14:31supposed to emerge from that and
14:33scientists look at that and say that's
14:35kind of crazy in addition to time frames
14:38on which the law operates well they need
14:43a resolution you got a resolve stuff you
14:46got to just keep churning out
14:47resolutions to disputes and it's not
14:50critical that everyone be perfect but it
14:52is critical that they get decided and
14:55scientists are very unhappy about that
14:57because as a matter of science
14:59nothing is ever resolved indeed the
15:02fundamental notion about science is that
15:04things remain open to challenge at all
15:06times and we don't want to say this
15:09issue is completely settled and
15:11certainly we're not going to get it
15:13settled within the length of a
15:14particular trial
15:15but these sorts of things apply to any
15:18given case they don't really apply to
15:21the broader question of the law in many
15:25ways when you go up to that level the
15:28law is kind of similar to many ways we
15:31think about things in science because we
15:33truly try to grapple in science with
15:36fundamentals with principles with the
15:38meaning of things and the law over long
15:40periods of time does try to do that and
15:43while you can think about engaging
15:45between science and law at the level of
15:48an individual case and a lot of that has
15:50to happen and many people do it in
15:51patents and other things it isn't the
15:53most interesting thing to me to me the
15:55most interesting thing is about those
15:57conversations that are extended
15:59conversations over time and what I
16:01admire so much about American law is
16:04that we make these sweeping statements
16:07all men are created equal and then we
16:11take two centuries to figure out what
16:12did we mean by that and we grow into the
16:15meanings of those words I had occasion
16:17to talk to a German colleague whose mind
16:19was blown when he realized that we do
16:21that of course you don't do that in
16:22continental law you specify you know
16:24what it is you mean we actually go out
16:26there and we don't know what we mean and
16:27we come to understand it all men are
16:30created equal by men we don't mean men
16:32we mean everybody it turns out
16:34eventually by equal what do we mean by
16:37equal and all those sort of things equal
16:38protection due process I think this is
16:41one of most majestic things that we
16:43actually put out aspirations and we grow
16:45into them and in science we do it too we
16:48want things to be right and reliable and
16:51we know we're imperfect and we're
16:52grappling for it and I think that this
16:54level is the most interesting
16:56conversations that can happen between
16:58science and law where we recognize that
17:00we're both feeling around in the dark
17:02things aren't going to be resolved
17:04quickly and although in the courtroom it
17:07will be adversaries butting heads in the
17:09real world it isn't adversaries butting
17:11heads it is well-meaning people who
17:13bring different points of view to try to
17:15figure out some meaningful resolution so
17:18I think the most productive
17:19conversations are not about complex the
17:22tails is in a patent case although God
17:24knows very important but about these
17:25simple ideas but I do worry that we do
17:28have effective forms for those
17:30conversations that scientists don't
17:33speak all lawyers don't speak science we
17:35don't have places to meet and sadly the
17:37place we mostly have to meet is in a
17:39court room which just isn't such a great
17:41place to have these conversations and so
17:44I want to return later to can we create
17:47these and I think since we have a Dean
17:50here I'm gonna say that the law school
17:52could turn out to be a marvelous place
17:54to experiment with forums for such
17:56conversations that might produce some
17:58useful products that might have some big
18:00impacts so anyway I'm gonna grapple for
18:04just a little bit before we go to
18:06question answers with some words
18:09reliable what do we mean by reliable
18:13that's something of interest to the law
18:16and to science extreme what do we mean
18:20by something being extreme a product of
18:25nature the law has gotten itself into
18:29caring about that in the patent law
18:30about products of nature and about
18:33unbiased predictions because of course
18:36when a judge sentences someone or when a
18:39judge grants bail they're making a
18:41prediction we want these to be accurate
18:44unbiased predictions so these are places
18:47of fruitful conversation between law and
18:50science and I'm gonna touch on them
18:51because we've touched on these topics
18:53within the course I'll go lightly
18:57through them not compared to the two
18:58hours that were devoted to each of these
19:00topics and we didn't frame them in these
19:02ways we framed them around particular
19:04cases and decisions and situations but
19:07for now I just want to deal with that so
19:09this topic I've told you I care about
19:11forensic science in the criminal courts
19:13going back to DNA fingerprinting in the
19:151980s this People v Castro case what
19:24does it mean have a reliable method for
19:28telling whether two DNA samples match or
19:31two fingerprints match or two bullets
19:33came from the same gunner
19:35well this reliable word is not my
19:41it's your word it's the word of rule 702
19:44of the Federal Rules of Evidence so rule
19:47702 C says that for an expert to come to
19:51court and present testimony as an expert
19:58one of the criteria is the testimony
20:01must be the product of reliable
20:03principles and methods this comes from
20:07the Federal Rules of Evidence first
20:08introduced in 1975 but the Dalbert case
20:11ended up reshaping the meaning and
20:14eventually these rules were restated a
20:16couple of times after Dalbert this is
20:17the current form of 702 C and so they
20:22came out of initial Rules the Supreme
20:24Court made various comments about these
20:26things they got reshaped in but what we
20:28have right now is an instruction the
20:30court the judge is to be a gatekeeper
20:32and let in only expert testimony that is
20:36based on reliable principles and methods
20:38what in the world does that mean I guess
20:42a it's a mess
20:43it's an extraordinary mass of what that
20:45means because it refers to all possible
20:48expert testimony and you can't talk
20:50about the reliability of everything you
20:53can though talk about the reliability of
20:55particular broad areas in any case the
21:01court said in famous footnote 9 and
21:03Dalbert and in many other places in that
21:05decision essentially in a case involving
21:08scientific evidence evidentiary
21:10reliability will be based on scientific
21:13validity I can't think of a clearer
21:16instruction that says there must be a
21:18conversation between law and science
21:20if evidentiary reliability is to be
21:22based on scientific validity it can't
21:25just be a lawyer's opinion about
21:27scientific validity a scientific
21:29validity and this compels the
21:31conversation so we in the form of the
21:37scientific community the National
21:38Academy of Sciences and PCAST in in two
21:41reports that I've referenced here the
21:43National Academy in 2009 and PCAST in
21:462016 have been grappling with
21:48of reliability for just one broad class
21:52of evidence but an important one in
21:55forensic science
21:56forensic feature comparison methods this
21:59is the stuff you see in CSI DNA samples
22:03would they say it's a match or latent
22:06fingerprints it's a match or the
22:09firearms or bite marks when they say
22:14despite having no evidence to support it
22:16it's a match that person bit that person
22:19Footwear analysis this print must have
22:23come from that shoe or hair analysis
22:26this hair has to have come from the
22:28defendant the methods all are
22:31essentially similar in their structure
22:33there's a laboratory component you get
22:35two samples you have a set of features
22:38you're supposed to look at the width of
22:40the hair the shape of the hair those the
22:41roughness of the hair the color their
22:43same for DNA forever you look at these
22:46features and you ask do they agree
22:49within a certain tolerance with respect
22:51to those features if so is that
22:56surprising would you expect things that
22:58came from different sources to agree
23:01notice you can't answer the question
23:02till you do both it's fine to say they
23:05agree on height but I mean big deal
23:09there are a lot of people who agree with
23:10you on height that's not enough we need
23:12a database to understand how frequently
23:15there are people of your height I so
23:18those are the two key things you've got
23:20to have so what does it mean for a
23:22method like this to be reliable how do
23:25we know that it gets you a reliable
23:27answer when you declare the two things
23:29are likely to have come from the same
23:30source well before I get there why does
23:34it matter who cares if they're really
23:37reliable well I mean obviously rule 702
23:40told us tells us we're supposed to care
23:42if they're reliable and of course you
23:43know we respect rule 702 but outside of
23:46the fact that the Federal Rules of
23:47Evidence compel us to pay attention to
23:49that there are a lot of good reasons to
23:51care particularly about forensic feature
23:53comparison methods being reliable first
23:56of all they were not
23:58in any scientific laboratory with the
24:00exception of DNA they were developed as
24:03very rough heuristics to guide an
24:05investigation they are not grounded in
24:08the validation practices of science and
24:11of course for the purpose of
24:12investigation you can use a Ouija board
24:14the Federal Rules of Evidence do not
24:16speak to how you investigate you can use
24:18any intuition you want to do that
24:21perhaps this refers to though when we
24:23bring it to court is it valid well so
24:27they were not developed for any normal
24:30procedures of scientific validation
24:31they're developed by well-meaning
24:33Constabulary around the world often in
24:36the 1800's 1900s in addition they are
24:41particularly concerning because they
24:43claim to be and are seen to be
24:46tremendously probative that they claim
24:49often to be able to discriminate to a
24:51level of one-in-a-million and not only
24:53that when you ask mock jurors they say
24:55oh yeah that's probably one in a million
24:57when you say it matches if you don't
24:58give them a number they'll fill in that
25:00number that seems to be the notion out
25:01there is that these things are highly
25:03highly discriminated just able to
25:06discriminate they were accepted by
25:09courts rule II and middle of the
25:13twentieth century before there's any
25:14well-developed notion and before by the
25:16way there was a rule 702 that said this
25:18and so we we have a long history of
25:20acceptance of methods without underlying
25:23evidence they are particularly
25:26concerning as compared to many other
25:28things an expert might talk about
25:29because al a juror has no chance of
25:32understanding what most of these things
25:34are about so this is entirely going to
25:36be people coming in garbed in in the
25:39robes of science and then special
25:42compared to most other things we know
25:44they get it wrong because we now know
25:46there have been 350 wrongful convictions
25:49when it's not a question of somebody was
25:51later released or a conviction was
25:55overturned it was agreed the person was
25:58actually innocent that they had the
26:00wrong person and that's of course what
26:01DNA fingerprinting did for us was it
26:03allowed us to say wrong person so we can
26:07actually find false positives in the
26:09legal system
26:10and so when something
26:11seen as extremely probative and gets it
26:15wrong that hasn't been validated there's
26:18a special reason to attend to it and
26:20that's why rather than taking on the
26:22meaning of reliability for all of
26:23science which i think is a is it is a
26:26mistake to try to go so broadly which on
26:28this one area so the National Academy
26:32wrote a report in 2009 judge Edwards
26:34from the from the DC Circuit co-chaired
26:38that report and it focused on all the
26:42ways in which this could be improved
26:44improvements for forensic science
26:47it had no effect no effect there's so
26:52curious judge Edwards a distinguished
26:54jurist basically approach this and his
26:57committee approached it scientists and
26:58they said look here are the defects
27:00here's how it can be improved etc but
27:03what ended up happening was there was an
27:04immune reaction by the Department of
27:07Justice and by the aspects of the legal
27:10system that said that's great
27:12improvement we all want improvement
27:13improvement as incremental we should
27:15keep improving but this has nothing to
27:17do with the miscibility this really
27:19doesn't bear on what comes into court
27:20and therefore please feel free to keep
27:23improving we support that but there were
27:26strong statements that this really had
27:29no impact on admissibility and yet of
27:31course admissibility is the thing people
27:33pay attention to because if it's not
27:35admissible then you got to start paying
27:37attention if you can get it in you know
27:39you're down to your particular courtroom
27:41in your particular jury
27:42so when PCAST took up this question at
27:45the behest of the president you know he
27:47said we've started a National Commission
27:50on forensic science we've done some
27:51other things etc what else could we be
27:54doing the president asked we decided to
27:56focus very squarely on the question of
28:00admissibility because we figured I thing
28:04else really mattered
28:05and so we deliberately wrote a report
28:09about 177 pages 300 footnotes it's quite
28:13a report but it focuses on one basic
28:15question what does it mean for something
28:17to be reliable if it's forensic feature
28:21comparison method
28:23and then we actually took our definition
28:25and we tried it out on a whole bunch of
28:28fields and asked how are they doing
28:29those fields so here is the radical
28:33conclusion and I say that not in jest
28:35because this is a conclusion that most
28:37people if the Department of Justice will
28:39disagree with our radical conclusion a
28:42forensic feature comparison method can
28:44only be considered reliable if its
28:48accuracy has been empirically tested
28:51pretty much it it's a radical statement
28:55that empiricism is necessary if you
28:58don't have empirical evidence that that
29:01measures the accuracy under reasonable
29:04conditions and they accurately actually
29:06is reasonably usable we'd have to fuss
29:07too much about it you don't have a
29:10method you don't have no business in
29:11court this turns out to be a flashpoint
29:13it is stunning that this is a flashpoint
29:16but it is I'll explain why but first let
29:19me tell you about the history of these
29:21methods history of feature comparison
29:25methods they go through certain stages I
29:28won't say stages of denial but in any
29:31case they go through stages here the
29:33first might be called data free theories
29:35the second spurious estimates then you
29:40get to empirical tests often tiny
29:43empirical tests and then finally
29:45meaningful empirical tests so I'll just
29:47take fingerprints in 1984 the Justice
29:50Department issued a formal document that
29:52declared it was like people in its in
29:56its nature it declared fingerprints to
29:58be the only method that that were both
30:02feasible and infallible so it's a
30:06remarkable statement in 2016 the
30:09Department of Justice did disavow this
30:11statement as as overreaching but as
30:14recently as 2009 the former head of the
30:17Crime Lab FBI crime lab did testify in a
30:20case that here's a spurious estimate
30:23that fingerprints were accurate to at
30:25least one part in 11 million how did
30:28that estimate arise because during the
30:32time he had been there they had done 11
30:34million fingerprint comparison
30:36and he was only aware of one error
30:38that's hope but this was testimony in a
30:42case it was it was remarkable and then
30:45God bless him the FBI FBI did the right
30:47thing the FBI Crime Laboratory is filled
30:49with some very good scientists and they
30:52did what's called the black box study
30:53you don't know what's in the examiners
30:55head you have to treat it as a black box
30:56give them problems see how often they
30:59get the right answer they found they
31:01make a mistake about one time every 600
31:03you know with error borås maybe one in
31:06300 it's not infallibility it's not one
31:08in eleven million bits not bad you could
31:10go to court and say they get it wrong
31:11about one time in 600 you suddenly can
31:13weigh that against other evidence and we
31:15don't have to worry about it precisely
31:17what that is I believe now that
31:19fingerprints can be considered to a
31:22first-order having some statement of
31:24reliability and it didn't kill him to
31:26measure the accuracy firearms well there
31:30there were funny spurious estimates
31:33where they gave people bags of bullets
31:35and they they knew they were related in
31:37various ways and they got them right
31:38about with an error rate of only one in
31:405,000 but there are a lot of internal
31:43hints when you thought about it as soon
31:45as more recently started by the
31:46Department of Defense people began
31:47comparing things where you didn't have
31:49those internal hints they make mistakes
31:51about one time in 50 that's a number
31:54that's reliability
31:56let's take bite marks the famous paper
32:00in 1984 and bite marks looked at all the
32:02teeth you have and that they could be at
32:03different angles and different things
32:05and calculated no malice action no bites
32:08actually made calculated that there
32:11could be six trillion different
32:12hypothetical patterns that bite that
32:15that the dentition z' could make and
32:17therefore dentition was unique and there
32:19were one in six trillion patterns and
32:21therefore bite marks you know were an
32:23identifying feature but nobody actually
32:26tested even if it were true that that
32:28room that that mouths were also
32:30different whether mouths biting into
32:32flesh and then blood oozing and and and
32:35swelling everybody you can tell anything
32:37until relatively recently when studies
32:40were done that found that they can't
32:41they really do basically even when there
32:45are only four biters and you have to
32:46associate with one of the four
32:48you get it wrong one time in six that is
32:52an empirical test but it's a test that
32:53says it's useless and onward Footwear is
32:57really great there's one that calculates
32:59the chance that three random dings would
33:01be in particular places and estimates at
33:03one in 683 billion and as of today there
33:06is yet to be a single test whatsoever of
33:09empirical evidence of whether they can
33:11do it or not so anyway this all seems
33:14radical it's all sorry this all seems
33:16obvious and so we said nothing can
33:18substitute for empirical evidence this
33:21is sort of stuff from the 1600s the idea
33:23of empirical evidence or so but it is
33:26broadly contended including by the
33:28Department of Justice that things can
33:29substitute it's okay it's it's okay if
33:33the method is based on science that's
33:36like the screenplays based on a true
33:38story examiner's have good professional
33:41practices training certification
33:43accreditation professional organizations
33:45best practices manuals this is stuff the
33:47Supreme Court recognizes and Albert is
33:49good indicia the problem is psychics
33:52have all of these psychics have
33:56accreditation they have organizations
33:58their proficiency tests they have
33:59peer-reviewed journals
34:01what is the difference between psychics
34:03and science empirical tests that's the
34:08only difference and so all of these
34:09things are nice to have but irrelevant
34:11with regard to reliability should count
34:14for nothing because we know that
34:16psychics do this you can put all these
34:19things in without being right so in any
34:21case we've argued this and we go back
34:23and forth all the time we made a set of
34:25recommendations to the Justice
34:29Department to the FBI to NIST OSTP the
34:32Justice Department rejected all the
34:35recommendations NIST was much more
34:37accommodating others were coming the
34:39Trump administration went further and
34:40abolished the National Commission on
34:42printed science and appointed one guy
34:45and a prosecutor at the Department of
34:47Justice to serve as the advisor on
34:50forensic science because he had been on
34:51the aforesaid abolished Commission and
34:54so we made one recommendation though to
34:57the judiciary which was the judges
35:00should take into account these
35:01these criteria the scientific criteria
35:03and the judges should ensure that
35:06testimony is consistent with the
35:08empirical evidence and that the Judicial
35:11Conference in the United States through
35:12its standing committee the standing
35:14Advisory Committee on their Federal
35:15Rules of Evidence should prepare a best
35:17practices manual and perhaps an advisory
35:19a new Advisory Committee note about 702
35:22and god bless him they listened last
35:27October the Advisory Committee on
35:30evidence held a one-day symposium here
35:33in Boston where they brought together 22
35:35people and they brought together the
35:37committee to hear talks on how rule 702
35:40might be revised in principle and
35:43promised to revise it but they wanted to
35:44understand what were the problems what
35:45might me to do and that this is the
35:51transcript of that together with
35:52articles written by many of the
35:54participants myself included on how one
35:57might fix rule 702 and so the legal
36:01system is actually responding very
36:05slowly I'm not holding my breath I don't
36:06expect to change but people are
36:08beginning to pay attention and it's an
36:10interesting thing that they're grappling
36:13with the fact that rule similar to it
36:14doesn't really work in this way what are
36:16some of the problems we don't have a
36:17good channel for conversation courts
36:20don't and can't take in some people's
36:24views judicial notice of a number of
36:26scientific authorities various courts
36:28have said loudly we can't admit this
36:30National Research Council National
36:31Academy report at least we got to go
36:34witness every single time you want to
36:35get it in the PCAST report you know the
36:38Department of Justice complained
36:40bitterly about the report cassette of
36:42the seal the president on the cover of
36:43the report and they said people might
36:46think this is a report from the White
36:47House and we said this is a report from
36:48the White House and so so there you go
36:53and so but whether it comes in or not it
36:55depends on the judge and all that and I
36:57get phone calls all the time would I
36:58come to Montana to testify on the PCAST
37:00report or and I can't do that and
37:02they're like not that many people in the
37:03committee we couldn't possibly cover
37:05this so we don't have ways for courts
37:07naturally to take this in we don't have
37:10a great way for the scientific
37:14authorities in this country
37:15like the leading advisory groups of the
37:18legislature the National Academy of
37:19Sciences in the leading advisory group
37:20to the White House to even have a
37:22channel of conversation say this should
37:25be taken you should take judicial notice
37:27of these things but in addition it's a
37:29little worse than that because
37:31evidentiary decisions including these
37:33abouts about expert testimony are
37:35reviewed under the abuse of discretion
37:37standard so it means the two utterly
37:40contradictory decisions about a
37:42technology are both fine as long as a
37:45reasonable judge could have reached
37:47either and since we have no overarching
37:49conversation about it it's not hard to
37:51read to that conclusion so the usual
37:53forces that would that would engender
37:55convergence do not occur but I note that
37:59this week also is a great thing about
38:01giving a talk as all sorts of things
38:03happen the week you're giving a talk I
38:04learned that the Royal Society in the
38:07United Kingdom has begun writing little
38:11manuals 35-page friendly just friendly
38:16whitespace manuals directed to the
38:19courts primers for the courts on topics
38:23like forensic gait analysis can you tell
38:26from a video of somebody walking whether
38:28that was you or you and they do a
38:32careful analysis and they say there are
38:34no black box studies Oh be still my
38:36heart they followed the whole PCAST
38:37framework it was wonderful there no
38:39black box studies yes gate might be
38:42completely unique if you could measure
38:43it perfectly but there's no evidence
38:45anybody measures it perfectly enough the
38:47only empirical study that's been done
38:49got it wrong like one time in four with
38:52a limited set of people and it laid it
38:54out in such simple clear terms they're
38:56doing for DNA makes me think and I'll
38:59return to the question how can we do
39:01such things here to lay things out in a
39:03language that communicates this so
39:06anyway I'm gonna touch on a couple other
39:08topics before I want to throw it open to
39:10everybody and I'll just touch but here
39:13I'll touch more lightly electoral maps
39:16same question what is the word excessive
39:20mean when we talk about excessive
39:22partisan gerrymandering in our course I
39:25handed this picture I've handed that
39:27picture out on our first
39:28that is the famous Pennsylvania vii
39:30affectionately known as goofy kicking
39:32Donald Duck and and it was the district
39:37when we started the course and again
39:39through superb planning we had the
39:41Pennsylvania Supreme Court declare
39:43Pennsylvania's map to be an
39:45unconstitutional gerrymander based only
39:46on the state constitution not reviewable
39:49by the Supreme Court and a new map has
39:52been issued and poofy kicking Donald
39:53Duck is no longer in force but in any
39:56case the timing could not have been
39:58better for the course but the key thing
40:00and I'll just go quickly through this is
40:02the Supreme Court is grappled with this
40:03following question excessive partisan
40:06gerrymandering is a problem it's
40:08incompatible with democratic principles
40:09and everybody agrees it is
40:10unconstitutional but it may not be just
40:13disabled because there may not be a
40:15judicially manageable standard for
40:17recognizing one and that has been the
40:19big issue before the court not that it's
40:21okay not even that it's constitutional
40:23but that there may not be a judicially
40:25manageable standard but in the most
40:29recent case in 2004 Justice Kennedy the
40:32swing vote in this whole business wrote
40:34technology is both a threat and a
40:36promise saying that the same
40:38technologies that are allowing us to
40:40make more and more excessive
40:41gerrymanders might be a microscope to
40:44let us recognize excessive gerrymanders
40:47and so we debated this a lot in the
40:51class and here who knows what will
40:53happen in guilty Whitford I don't know
40:55but what I wrote in an amicus brief was
40:58that excessiveness is a inherently
41:00quantitative concept that requires at
41:03least some quantitative underpinning you
41:05can't hope to talk about whether
41:07something is an excessive gerrymander
41:09without some quantitative grounding so
41:13how do you read you know how do you
41:14focus on this well the legal literature
41:16some cases including some articles
41:19including one about something called the
41:20efficiency gap have focused on defining
41:24various metrics of partisanship but
41:27defining a measure doesn't end the story
41:29is that measure excessive the only way
41:33you can tell if a measure is excessive
41:34if you know the answer excessive
41:36compared to what but we do this all the
41:39time in laboratory so we measure
41:40something we say does it
41:42fit our model does it have a p-value is
41:44it far out on the tail of the
41:45distribution or is it within what we
41:47might expect and do we reject some
41:50hypothesis the natural question to be
41:52asked how does the state's chosen map
41:56compare in its partisanship to all other
41:59maps it could have drawn consistent with
42:01its declared principles that's it
42:03excessiveness should mean far out on
42:06that distribution of all possible maps
42:08you could have drawn and so I argued in
42:10this particular amicus that that was a
42:13perfectly easy thing to compute or at
42:15least doable thing to compute it was
42:17sort of like the way we compute whether
42:18a a nuclear bomb will blow up we look at
42:22a distribution of outcomes or a nuclear
42:24power plant won't blow up or a hurricane
42:27will hit Miami the federal government
42:29does this all the time of look at the
42:30distribution of outcomes and see which
42:32ones have a particular property how far
42:34out is things on a distribution if
42:35something's an extreme outlier the
42:37argument that this was plausibly
42:39explained by the state's goals don't
42:42don't hold water so that's a way to do
42:45it I believe that we need to have
42:47whatever the court does I hope they keep
42:49the door open to this question of
42:50excessive partisan gerrymandering and we
42:52get to further flesh out what this can
42:54mean but there's got to be a
42:56conversation between folks and science
42:58who think naturally about how do we know
43:00whether something is extreme and know
43:02that you can actually run large-scale
43:04computer calculations to know the answer
43:06there's no guessing there's no machine
43:08you draw all possible Maps or at least
43:10sample from the distribution of all
43:12possible maps there's a fertile
43:13conversation for us to have gene
43:16patenting similar sort of questions
43:18arose in this case I won't go into much
43:20detail other than to say after a
43:23surprising victory by the ACLU at the
43:26federal district level the Court of
43:28Appeals for the Federal Circuit that
43:30here's patent Appeals ruled that genes
43:33were patentable because of all your
43:35chromosomes were products of nature
43:38fragments of the DNA of a chromosome did
43:42not occur in nature and therefore were
43:44the work of man this is stated in the
43:51opinion of the court because fragments
43:54do not occur in nature no citation so it
43:58turns out this 40 years of literature
44:00showing that fragments of DNA occur and
44:02not only that prenatal testing for Down
44:06syndrome in moms is often now done by
44:09taking blood for maternal circulation to
44:12find the fragments of fetal DNA that are
44:14circulating there so this is not
44:16actually like a hypothetical question
44:18it's the basis of Down's syndrome
44:21testing that there are fragments
44:22circulating a maternal circulation and
44:24yet it made it all the way through the
44:26Federal Circuit without anybody saying
44:29whoa wait a second that actually isn't
44:30actually true remarkably I wrote an
44:34amicus in this since some other folks
44:36did remarkably the Supreme Court ruled
44:38nine nothing in favor in favor of the
44:42plaintiffs in this case arguing that
44:44genes war indeed patents the products of
44:47nature and could not be patented and
44:50it's a fascinating situation it's we
44:53spent a long time in our class talking
44:56about that decision which is a confused
44:58decision that we reached in my opinion
44:59the right result but very confusingly
45:02and good luck trying to understand
45:04there's only one critical paragraph in
45:07the opinion by Justice Thomas and that
45:09one paragraph will not help you a great
45:11deal so where it goes who knows but
45:13there's still going to be more such
45:15conversation finally last topic
45:17artificial intelligence unbiased
45:21prediction well you may be aware and
45:24when I walked into Dean Manning's office
45:26at 11:45 today was the first question he
45:28asked me about the use of machine
45:31systems to try to improve the decisions
45:35we make in court about who should get
45:38bail or what the sentence should be
45:39because of chance of recidivism on the
45:41part of someone who's been convicted and
45:44there's this cool system called compass
45:46that is used in a number of states we're
45:48based on 21 variables that somebody
45:51fills out the the individual the
45:53defendant now convicted fills out these
45:5521 variables the computer turns and it
45:58says you get a risk score of reoffending
46:02between one and ten and this offers
46:06great possible as we know judges are
46:07import
46:08we know there are biases in the heads of
46:10judges we know that they behave
46:11differently after luncheon before lunch
46:13we know judges you know why shouldn't
46:15just be done by a machine and get it
46:17right and get all of this subjectivity
46:20out of it well one reason Pro Publica
46:24found was that the algorithms are only
46:28as good as the stuff you pour into them
46:30and it turned out that if you have
46:33african-american individuals who it
46:35turns out don't commit a crime in the
46:38next two years and white individuals who
46:40it turns out don't commit a crime in the
46:42next two years and you compare the
46:44scores they were assigned by the
46:45algorithm the african-american
46:48individuals get much higher risk scores
46:50despite the fact that those are between
46:53two populations that do not reoffending
46:57is about right but the errors are
47:00distributed highly unequally across
47:02groups included here including your
47:04protected groups well that's a little
47:07worrisome how did it do that well we
47:10have no idea because the code is
47:11proprietary the organ is you can't see
47:15the code because this state bought the
47:16code and the state signed the contract
47:19that said it's the intellectual property
47:21the company and that might just might
47:24cause issues with equal protection due
47:25process so who knows all sorts of things
47:27there I gotta say I admire tremendously
47:30the work of a bunch of faculty here at
47:32the law school at MIT who wrote an open
47:34letter to the legislature that is
47:36considering adopting risk assessment
47:38from Massachusetts saying if you're
47:40gonna do it all sorts of measures have
47:42to be in place I think it's a great
47:44model for the sort of things we could be
47:46doing and turning it more into just an
47:48open letter on medium to some series of
47:51reports about if you're gonna do it here
47:53other method measures to know you got to
47:55keep checking every couple years because
47:56it could be drifting you got to do this
47:58you got to do that and we could set
48:00standards for these things early and
48:02then that was just 21 variables 21
48:06variables is easy what we now are facing
48:10is a world of deep learning where
48:12millions of variables go in about
48:14millions of things as I put to my law
48:16school class when you get hired at log
48:19whant when they're considering you for
48:20hiring at a law firm they scrape
48:22all of your social media records they
48:24scrape everything toss it into the big
48:26neural net and find out whether they
48:28should hire you or not who knows what
48:32the neural net is doing nobody knows
48:33it's a million coefficients in there
48:35in what sense can we speak of intent now
48:38they will agree we will make sure not to
48:40put in your race or your gender or other
48:42things but of course with all those data
48:44it's not very hard for those things to
48:46be represented anyway no matter how you
48:49try to take them out there there we can
48:51learn all those things we can predict
48:53your gender we can predict your race so
48:55in no sense can we take them out and we
48:56can't tell what the machine is doing how
48:59are we going to decide if these are
49:00unbiased predictors we actually need
49:03guidance here and happily we haven't
49:05gone too far that we can't still try to
49:08develop such guidance so I'm gonna stop
49:11because I want to have at least ten
49:12minutes of discussion here we are not
49:14having the conversations we could have
49:16but there is no better place than Boston
49:18to have them Harvard particularly is a
49:20place that is rich in law richly
49:23humanities rich in the sciences and
49:25technologies and MIT just down the road
49:27has tremendous assets to bring to bear
49:30as well the thing that UK is doing
49:32fantastic we could be doing stuff like
49:35that and we could draw draw the National
49:37Academies into it we could be doing
49:39things like that like Chris Babbitt says
49:41letter to the to the legislature if we
49:44choose things about principle where each
49:47side can inform about principles I think
49:51we could provide a lot of guidance that
49:53isn't quite so hard it isn't in the
49:56weeds and it could be tremendously
49:58influential so how do we create more
50:00effective forms for the conversation
50:02between law and science there's no
50:03better time than just right now starting
50:06our third century here to think about
50:08how to do it because I suspect this
50:10third century will be filled with many
50:12many such questions thank you very much
50:15for the invitation to speak here and to
50:17teach
50:28until we shine up you use the
50:32empirically tested results where 2006 is
50:36an era where one in six is an arrow or
50:39one in six hundred throw line so Mary
50:43said 106 a lot of visible one in sixteen
50:46or 600 is admissible or you're allowed
50:50to tell them how accurate it is only six
50:54the same shoe Rosa journey when we talk
50:56about how parameter distributions a
50:58widely popular classes the team doesn't
51:01have to be here's the shrub on its
51:03another bigger before evolution say
51:07districts could be like that that's it
51:10one person and over to work this way to
51:20the fact one percent in some
51:33we pay attention so I don't want hang on
51:37I want and then the courts will digest
51:44and metabolize where did you go - in
51:48space this is been a lively discussion
51:49for our course yes
52:02[Applause]
52:03I'm in tension between the optimism of
52:07your perception is fine and the fruits
52:10of many other examples so the first set
52:12of slides are the long sides are really
52:19doing the same thing or at least wise
52:22it's not do you say yeah see if you
52:30agree
52:31yep none of the fruits of many of the
52:34particular examples reinforce our common
52:37sense in the law there are distressingly
52:39few participants in the process who are
52:43primarily focused on either truth or
52:46justice yep so fortunately their
52:49obligation is to represent legislators
52:57rarely are easier moved by various
52:59incentives seeking their support judges
53:03self-consciously found their zones of
53:07discretion
53:10so where is the contrast of immunity
53:13scientists largely are pursuing truth
53:19evidence-based enquiries the dominant
53:21orientation sadly outside of optimistic
53:26interpretation of the law schools are
53:27relatively few players from the legal
53:29system that cut that in mind in the
53:32small gets supply reckons small I
53:34totally limit even important this does
53:37not apply but we manage still to extend
53:43the certain meanings rolling to certain
53:45meaning and it's not because anyone
53:47legislature it's not the case as a as a
53:57society we somehow manage to have these
54:01conversations but they take decades well
54:06you know it is do you believe the ARCIC
54:09justice the arc of the universe is long
54:12depends to justice or justice you know
54:15at least you read American history woman
54:17enough I'd say we've been more than you
54:19might have thought beginning it's not
54:21it's not fantastic it gets made and when
54:25I talk about these issues I'm tilting it
54:28that the forensic scientist uses 1989
54:32DNA is now pretty three rigorous foods
54:35practically story we've now have forced
54:38the FBI to do the black box study on
54:40fingerprints they're now joining up
54:42black box on
54:44firearms that this is all these and then
54:53any of this stuff is gonna happen
54:55quickly which is why is a conversation
54:57we've had outside because and if I'm a
55:01little over optimistic Sophia is going
55:05in but look if you could we don't we
55:27don't overturn past convictions they
55:29don't want to serve cases that aren't
55:30going
55:31couldn't we just do this just to say
55:37that these statements of science so this
55:47is terrific the best seminar I ever
55:50taught was maybe 20 years ago when I had
55:52law students and medical students in the
55:55same room trying to understand what's
55:57going on in this broad way and I guess
56:01the thing that came through to me the
56:03most was building a little bit on Terry
56:05Fisher's last point was the cultural
56:08difference in the way in which people
56:10thought about these issues and so I
56:15guess it's a kind of a two-part question
56:16one is it's hard to separate the small
56:20from the big because the culture of the
56:22small dominates the way at which people
56:24think about the big I guess the the the
56:27part I really love you to say about what
56:29you didn't say much about in your talk
56:31is how can the culture of law how should
56:35the culture of law affect the culture of
56:36science that as you said a lot about how
56:39the culture of science should affect the
56:40culture of law most of which I agree
56:42with somebody who does empirical
56:44research but you haven't said much about
56:46how the culture of law might affect the
56:49culture of science in this discussion
56:51and you would be great to talk about
56:53that even though Schwartz fine
56:56very short answer you know culture
57:02I liked the questions we do within wall
57:06should it expands
57:08Walter Agnew in the end of the 1800
57:14it was the arduous distance dorita
57:16statistics book today and it's been
57:19dehydrated the application just math
57:23stuff but no one came from the desire to
57:26do aggregate many things will provoke
57:30these questions about what is it fair
57:33prediction for itself these questions
57:36about what's an excessive parity manager
57:38I swept them the rug those are nice
57:48choose the right shape the focus some
57:52people in science I know that the rule
57:54is we stop at one I'm going to turn back
58:05[Applause]
58:20you

About this transcript

This page contains the full transcript of Klinsky Lecture: Eric Lander by Harvard Law School, generated from the public captions YouTube serves with the video. The transcript has 9,220 words across 1,356 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to translate the transcript into 100+ languages.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, translation to 100+ languages, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.