YT YouTube2Text

YouTube2Text → Transcripts → Responsible Research and Scientific Integrity

Responsible Research and Scientific Integrity — Transcript

by YUFE Alliance · 9,313 words · 1,546 segments · language en · Watch on YouTube

Open in transcriber Summarize with AI Download SRT/VTT

Full transcript

0:08Hello, this is a great pleasure of uh
0:11having this presentation on responsible
0:14research and scientific integrity for
0:16the uh EU
0:20for the EU 2030 uh meeting.
0:24And uh let me begin by introducing
0:29the uh uh our own group the synopsis
0:33meta research group
0:35which uh is unique in the sense that it
0:40applies research research methodologies
0:43to the study of uh uh practices of
0:47university practices themselves. In some
0:49sense, it incorporates
0:52scientific knowledge to study research
0:54policy and uh optimize the the practices
0:58of uh uh
1:02universities themselves. Um
1:24Apologies for the apologies for the
1:27stop. I have a little bit of a sound
1:28issue. Can you please confirm that you
1:30can hear my presentation clearly and
1:32then I will uh I will
1:36>> Yes, we can hear you clearly and we can
1:37see the presentation as well.
1:40>> Great. So the synopsis group is uh uh
1:46consisting of several uh tasks. So it
1:51serves as a meta research center at the
1:53University of Cyprus. Metal research is
1:55an emerging uh research field that
1:58involves uh different disciplines and
2:01different uh research centers and
2:04interdisciplinary
2:05uh working groups in different countries
2:08again with a uh purpose of studying the
2:11faculty of science itself.
2:13Uh one of our main uh interest is to
2:17bring together other scholars at the
2:19University of Cyprus interested in
2:22research policy interested in
2:23understanding the practices of academia
2:26uh and uh basically create a forum and a
2:31group that allows discussions
2:34discussions to take place and research
2:36interdisiplinary research to be
2:38conducted but also the university policy
2:41itself to be informed.
2:44And uh we are also trying to uh generate
2:49meaningful policy advice. So this kind
2:52of uh uh coalitions and this kind of
2:56initiatives like uh EU they have this uh
3:00uh special features of uh outline the
3:04importance of uh young research
3:07universities in Europe and how they can
3:10be special. they can be pioneers in uh
3:13uh new approaches to the research
3:16policy.
3:19So uh once more welcome to our uh uh
3:25to our guests and especially the the the
3:28junior people interested in uh research
3:30practices and integrity. So this is part
3:33of course of the uh youth training
3:36series on responsible uh research
3:38practices and this is the uh domain that
3:42we're going to be uh touching upon.
3:45Uh importantly we will speak about what
3:48might go wrong in uh science and which
3:53are the approaches that have been
3:54discussed in order to improve things.
3:57Basically when we all start as
4:00researchers, we start a little bit in
4:03the idealistic view that research is the
4:07search uh that science is the search of
4:09truth that it is an impassionate process
4:13of discovery. And then the more we uh
4:17work and we experience uh actual
4:20research, we realize that science is of
4:22course as we should have known in the in
4:25the first place a a human institution
4:28uh with its uh advantages and
4:30disadvantages. And therefore uh we
4:34should be careful and treated in the
4:36same way as as other institutions with
4:39uh weaknesses and uh uh potential
4:43policies to uh amilarate this.
4:48So today we will focus on the
4:51problematic uh of uh
4:56credibility crisis in research. the
4:58antidote to this which are good
5:00practices involving transparency and uh
5:04leading to better reproducibility and
5:07also we will also focus on an
5:09interactive approach. I hope my uh sound
5:13does not look very problematic there
5:17where we will illustrate uh the ideas
5:20the particular ideas about uh
5:23problematic versus good practices with a
5:26real example
5:29and again the idea is to be a sort of
5:32self-contained
5:33uh material and uh tools that can be
5:37applied uh practically
5:40So in particular in terms of the
5:43learning objectives
5:45after the end of this uh meeting you are
5:50expected to be able to understand the
5:53replication crisis in science and its
5:55implications to learn about
5:59specific practical open science tool.
6:03For instance, the new developments in
6:05pre-registration and the open code uh
6:08sharing often in real time uh should be
6:12able to apply uh responsible metrics.
6:17responsible metrics are closely related
6:19to the problem of uh the quality of
6:21science, reproducibility, prices and so
6:24on and so forth because essentially
6:27uh the incentive of scientists how they
6:29are rewarded affects their behavior
6:32especially from an economics
6:33perspective. Incentives are the most
6:36important uh element in uh an economic
6:39behavior. So how metrics are used to
6:43assess uh the quality of research is of
6:45fundamental importance for good science
6:48and that's that is where the concept of
6:50responsible metrics goes in uh for
6:54example as as applied by uh the Kara
6:58initiative
7:00and related to that going a little bit
7:04uh deeper into the concept of
7:07responsible research evaluation we will
7:10also mention mention the uh what these
7:13are and how they can be used to showcase
7:17diverse contributions. These measure
7:19such narrative are important especially
7:22today that modern tools allow us to
7:25reduce the cost of producing a high
7:28quality narratives of our uh
7:32contributions and essentially explaining
7:34the research journey of the scientists
7:36rather than bullet points with
7:38individual obligations.
7:41And finally,
7:44you should be able to recognize and
7:45address integrity uh challenges with
7:48research.
7:50And of course, uh the issue of integrity
7:54uh is closely related to almost all of
7:56the aspects that we have uh discussed so
7:59far and the ultimate quality of uh
8:01research uh being produced by science.
8:06So when I was uh involved in this topic
8:08of the uh of meta science, my first uh
8:14contact if you wish uh was uh an article
8:18at the New Yorker which had this uh this
8:21particular image which talks about the
8:25truth wearing off. So about 15 years ago
8:29to 20 years ago there was a
8:33initiation an initiation of a discussion
8:36about the potential replication crisis
8:39in the sense that uh it seems that
8:42subsequent studies that started from
8:44some often famous fightings did not seem
8:48to find similar results. In other words,
8:51uh the initial results did not seem to
8:54be uh replicable.
8:58And uh this pattern in the following
9:00years since the 2010s has uh exhibited
9:05itself across different disciplines. And
9:08uh the most systematic way to show the
9:12problem was large replication
9:14initiatives. initiatives that brought
9:16together
9:18uh collaborators in different
9:20universities, often a very large number,
9:22that conducted a great number of
9:24replications
9:25uh in a coordinated way. That was the
9:27best way to actually uh bring things
9:30together and show clearly the the
9:33picture which was that in uh important
9:36social science disciplines such as
9:38psychology
9:40uh about one/ird of studies are
9:42replicated.
9:43So which means that for uh starting
9:46researchers uh from Eupet let's say in
9:49the domain of psychology or social
9:51science you should be um contemplating
9:55on the underlying reasons why uh
9:58although as scientists we're supposed to
10:01to produce uh robust and uh uh
10:05generalizable knowledge we can often
10:08find ourselves in this uh difficulties.
10:10So we're talking about the work of
10:12established
10:13uh very distinguished uh sometimes
10:16researchers
10:18uh who were conducted the initial
10:19studies but nevertheless uh the large
10:23initiatives in replication showed that
10:25these results were not uh replicable
10:30and uh along more or less along these
10:34lines and at the same time similar
10:36initiatives happened in in economics.
10:39one led by Colin Came and his team which
10:43shows that economics fared best. There
10:45are many reasons why one would consider
10:49one discipline to be more ethical than
10:51than the other. We didn't have time to
10:53go in into that depth but in
10:56experimental economics the uh
10:58reproducibility
11:00rate was uh better.
11:03And then of course uh the poster child
11:07of the discussion about the
11:09reproducibility crisis was biome
11:11medicine. It started all this. Um so
11:15already since 2012 there were some uh uh
11:20famous uh reproducibility initiatives to
11:22replicate very important findings in
11:25cancer research with very low
11:27reproducibility rates. And uh often
11:31times it was pharmaceutical companies
11:33that conducted this uh replication that
11:36showed the uh often disappointing
11:39reproducibility rates for uh published
11:42preclinical studies.
11:46So now that we set the stage and showed
11:50that uh
11:53things do not go always in a rosy and
11:56expected way in science but there can be
11:59problems. We can go a little bit uh more
12:02deeper and think about the underlying
12:04reasons for these problems. So uh for
12:08starting researchers for junior
12:10researchers it is important to think and
12:12contemplate about the driving factors of
12:15nonreproducible research. Uh the general
12:18discussion has taken place across
12:20several uh general categories including
12:23theological flexibility.
12:26uh this roughly means that you're
12:28allowed to make conclusions on the basis
12:31of analysis often statistical analysis
12:34that takes place a lot of different
12:36forms. So the more experienced you one
12:39becomes with statistical analysis the
12:40more we realize that there are very many
12:42different ways to analyze a given data
12:45set and often times this can lead to
12:48different conclusions. This is the
12:50concept of methodological flexibility
12:52and it is problematic because often
12:54times as researchers we know if you wish
12:57what is the desirable outcome in terms
13:01of propagation prospects which of course
13:03is is problematic.
13:05Uh similarly the lack of transparency of
13:09uh the methods is if you wish
13:11interactive methodological flexibility
13:14uh exacerbating things and then uh this
13:19all these play a role um together with
13:24statistical noise which is if you wish a
13:26natural reason why results would fail to
13:29be uh replicable.
13:32And uh ultimately
13:35in some uh rare cases there might be
13:38even a research misbehavior in terms of
13:41uh uh conscious problematic reporting of
13:45the evidence because of uh incentives to
13:48find the results in a certain uh in a
13:51particular uh direction.
13:53And uh also milder forms of research
13:57misbehavior relate to the lack of
13:59transparency.
14:00uh those are called are referred to as a
14:04question of research practices whereas
14:06whereas research behavior is more
14:08related to conscious uh fraud.
14:12Okay. At the at the same time at the
14:16institutional level the macro level
14:18there are factors such as uh publication
14:20bias. These are not directly related to
14:23the practices of individual researchers
14:26but uh about systemic incentives in uh
14:31the publication process. uh it's not
14:34just individual researchers who are
14:35interested in getting interesting
14:36results but of course uh it is the
14:39publishers themselves and the editors
14:41themselves what are interested to uh
14:46declare interesting discoveries in terms
14:48of their readability their scientific
14:50impact and as a result publication bias
14:53refers to phenomenon whereby what
14:56actually ends up being published is only
14:58a biased sample of the actual uh
15:02research
15:03biased in the way that significant
15:06results are over represented. In other
15:07words, findings that declare a new
15:09discovery, some association between
15:12variables important for scientific
15:15knowledge. These kind of discoveries are
15:17often uh heavily uh over represented in
15:22the literature. But the failure to uh
15:26confirm an association or a discovery
15:29that is much uh less interesting uh if
15:33you wish but it's not less important for
15:35science and this bias creates important
15:38problems in seeing the totality of what
15:41we should be knowing about uh uh
15:44different uh variables and relationships
15:47in nature.
15:50Now be hacking is a sort of more
15:52concrete notion related to the notion of
15:55methodological flexibility.
15:58Uh the more we gain experience with
16:00statistics, the more we realize that in
16:03practically every scientific uh
16:06empirical article there is a large
16:08number of statistical tests. There's a
16:11large number of interesting variables.
16:13So usually there is like a general uh
16:17interesting uh relationship that we want
16:20to examine but how you operationalize
16:23showing that this relationship holds
16:25often takes place in a complex way using
16:28different variables different dependent
16:30variables different independent
16:31variables even. So there are very many
16:34ways that you can show evidence
16:36corroborating the uh fundamental
16:40relationship. So often times what can
16:44take place is that there is a choice of
16:49those statistical tests that seem to
16:51work that seems to show uh a a
16:54relationship which is interesting and uh
16:59uh most likely publishable and uh hiding
17:03the potentially great number of
17:06statistical tests that was not
17:07supporting this uh this relationship.
17:10This is the concept of B hacking and of
17:12course it's very closely related to the
17:14notion of uh transparency
17:17and the lack of transparency and
17:18methological flexibility because
17:20obviously if there is enough
17:22transparency then you cannot selectively
17:25choose which uh statistical tests to uh
17:29to present in the final
17:32and then hacking is is related in the
17:35sense of uh
17:38uh the initial study being uh
17:42not sufficiently driven by theory. So it
17:44is important as scientists that we uh
17:48advance theory in our discoveries.
17:50Essentially theory is the tool by which
17:52we learn about uh truths in nature.
17:57Science essentially is theory mechanisms
18:01uh of uh our understanding. So if we
18:04don't start with strong enough theory,
18:07we might result describe the result is
18:10harding or hypothesize after the results
18:12are known. Then you don't have two
18:14hypothesis after observing a large
18:17number of statistical tests. You might
18:19decide that
18:22it seems that there is some pattern in
18:23the data. So you uh post talk theorize
18:28what might be causing what and you might
18:31even present this as a hypothesis that
18:33we had in the first place which is of
18:35course is disingenuous.
18:39So again uh researched degrees of
18:42freedom is related to most of these
18:44issues at the individual level
18:46and lack of transparency.
18:50Of course, this is a fundamental problem
18:53and this is a why meta research, meta
18:56science as a new domain focuses so much
18:59on this
19:02and uh those important consequences are
19:06of course the wasted resources for
19:09society of investing and building on
19:12false findings for instance the
19:14development of non epicious drugs.
19:19uh it has been estimated that this this
19:21amount is uh extremely high at the rate
19:25of uh 28 billion per year according to
19:28estimation.
19:30Uh there's also more fundamental long
19:33run uh erosion of the public trust in
19:36science. Uh, of course we're living in
19:39an era where unfortunately we see a lot
19:42of uh uh aspects and a lot of uh
19:47appearances of this problematic uh
19:49phenomenon. The erosion of public trust
19:51in science.
19:53Uh often populist movements uh utilize
19:56uh this for their own interests.
20:00And of course these things are
20:02translated of
20:04uh inevitably in delayed uh process in
20:09addressing societal problems. So issues
20:12that could have been resolved faster if
20:15knowledge were was able to accumulate uh
20:18better without these biases now uh are
20:22not addressed as they as they should.
20:26And then you have the implication for
20:29career impacts. Often times we observe
20:33uh scandals and research uh non
20:36reproducible findings scandals about
20:39behavior and these reverberate on the
20:43scientific careers of juniors and senior
20:45researchers. Often times there are as we
20:48shall go into this later
20:51there are cradles and uh there are uh uh
20:55difficult decisions that need to be made
20:57within a scientific team and uh
21:02this is very important for career
21:04developments and uh
21:07in terms of ethical choices. Uh actually
21:11within synopsis we have uh economic
21:14theory models that try to examine these
21:17different incentives of junior and
21:20senior researchers and how uh the
21:23responsibility for research behavior can
21:25be optimally allocated in order to uh
21:30optimally solve this problem of uh
21:33genuine revelation.
21:35uh in cases of uh recent misbehavior by
21:39uh other parts of of a given group.
21:44So now in this part we will speak about
21:48uh more in depth about meta research
21:51study research itself. So the founding
21:55uh documents the hub of the domain of
21:58meta research was the very famous uh uh
22:02publication by John Eidis
22:05in 2005 with a very provocative title
22:09why most published recent findings are
22:12all.
22:13So this came rather suddenly into a
22:17scientific uh reality and society that
22:22was not alert to such an important
22:24problem of uh lack of reproducibility
22:27and John wanted to make this strong
22:30statement. Actually uh Ian and his team
22:34along with other researchers for several
22:37years before 2005 were conducting and uh
22:41advancing the domain of meta analysis to
22:44show how knowledge accumulates and they
22:47were they had gathered quite substantial
22:50evidence about the problems there but uh
22:54society and science at large not
22:56acknowledged the issue. So therefore
22:59this was sort of a manifesto rather than
23:01a pure scientific article in 2005 and it
23:04initiated
23:06uh the domain of meta research. But what
23:09exactly uh is meta research? So this is
23:13John and uh he along with others has
23:17defined meta research is the stud study
23:20of science itself.
23:22To be fair, the study of science has
23:25been uh studied for more than seven
23:28eight decades in mainstream research in
23:32the domains of philosophy of science, uh
23:34sociology of science and uh so on and so
23:37forth.
23:38Uh however, what is sort of new in this
23:41domain of meta research is the highly
23:43interdisciplinary aspect. So now
23:46researchers from different disciplines
23:47come together to understand the workings
23:49of science and it is also very
23:52quantitative is closely related to the
23:55statistical area of meta analysis
23:58uh which which are very strong ways in
24:01which it diverges from traditional
24:03philosophy and
24:06sociology of science but also
24:10uh it is uh determined as a solution
24:15to a important topical phenomenon which
24:18is the credibility crisis in science. So
24:20these are characteristics that uh define
24:23it uh and differentiated from
24:25traditional approaches.
24:29So what does meta researcher study? It
24:33studies the different uh science
24:35methodologies whether they are
24:36appropriate whether they are robust
24:39uh whether they can be improved. Okay.
24:43Uh similarly, it's not just the method
24:45that can cause biases and problems in
24:47science. It's also the reporting of the
24:49results.
24:51Uh we we referred to this previously. Uh
24:54selective reporting, dehacking,
24:57uh empirical
24:59uh
25:01examination of the extent of this
25:03phenomena. Uh potential remedies, all of
25:07those are within the domain of uh meta
25:09research.
25:11uh measures of reproducibility as we saw
25:14before uh
25:18solutions to the problem of
25:19reproducibility
25:21uh interventions
25:22policy interventions to uh improve them
25:26and of course research evaluation and
25:29assessment. How can we optimally assign
25:33uh rewards in uh in research such that
25:37they uh
25:40they improve on the problems of bias,
25:44honorable disability and so on and so
25:45forth. Align them better with the
25:48underlying objective of uh scientific uh
25:51discovery of the truth in in nature.
25:56Okay. And again as I hinted to before in
26:00those uh all those domains empirical
26:02methods play a very crucial role. So
26:05these are not just uh uh generic uh
26:11verbal uh discussions about how uh
26:15science works but a much more focused
26:18empirical approach.
26:22Okay. And as we we mentioned before,
26:25we're talking about
26:27uh measuring the degree that problems
26:29exist and the uh possible solutions and
26:33interventions.
26:35And uh importantly for junior
26:37researchers, it would be important to
26:39know that meta research is a growing
26:41field that attracts increasingly
26:44uh the support of funders and
26:46international organizations obviously
26:49because of the importance of the waste
26:52in uh uh public funds that come from the
26:55problem of reproducibility. So as a
26:58result uh there's a great state for
27:00society to improve things and getting it
27:03right. That's why meta research is uh
27:06strongly supported in
27:09Europe and beyond and that's why we see
27:12and we we hope and synopsis that we have
27:15held there have held there that
27:17initiatives that incorporate open
27:19science aspects that is UK itself is
27:23starting to incorporate research
27:25research within its uh domains the same
27:28with uh with kara
27:35So there's a large literature in meta
27:38research that provides specific examples
27:41of what we said before uh problems in
27:44science uh problematic methods
27:48problematic evaluation
27:50uh criteria and so on and so forth.
27:54So there's a large number of studies
27:57actually extremely large uh within the
28:00meta analysis literature that shows the
28:03problematic effect of uh small sample
28:06sizes especially combined with
28:09publication bias. You reach a situation
28:11where you have a proliferation of
28:13studies with small samples declaring
28:15large treatment effects. large
28:19uh associations between variables in
28:22science that are often times very weakly
28:26reproducible.
28:29Um then the the recent solutions that
28:33have been suggested for uh these
28:35problems include pre-registration.
28:38Pre-registration is important. We will
28:40get into more uh depth but uh roughly
28:44speaking
28:47If the problem is the researcher degrees
28:50of freedom, how a researcher can conduct
28:52a large number of statistical tests and
28:55then report only uh a fraction of these
28:58tests that seem to be favoring uh their
29:02own uh objectives in terms of what is
29:04interesting and publishable. Then what
29:07we should be doing is tying our hands
29:09essentially saying beforehand what kind
29:12of u test we will be reporting exactly
29:16which kind of hypothesis we have even in
29:20case of experimental research how we're
29:22going to be generating new data uh and
29:26how exactly our uh studies will be
29:28conducted with what sample size and so
29:30on and so forth. So registration is uh
29:34one of the most important and popular
29:37suggested remedies to the problem of
29:40research reproducibility
29:42and it has been found uh relatively
29:45recently by uh strong research teams
29:49among them the team of Abel Bier who has
29:52an institute of for replication at the
29:55University of Otawa who plays a large
29:57role in social sciences that indeed
30:00registration plays an important role
30:02role in reducing uh false positives.
30:06False positives are findings that are
30:08declared as true in the literature.
30:12This for instance this uh uh drug tends
30:15to uh
30:18reduce certain forms of cancer. This is
30:20sort of a uh scientific declaration.
30:25This is a false finding. false finding
30:27false positive excuse me to the degree
30:30to which it has been published in the
30:32literature that there is this
30:33association this uh efficacy of the drug
30:37which is actually false okay so
30:40registration tends to reduce the rate of
30:43false positives misleading findings in
30:46the literature
30:48uh in addition there is increasing
30:51evidence about how the
30:54additional suggestions about open data
30:57uh improve the the degree to which u
31:03scientific studies uh adhere to high
31:07standards of uh analysis, transparency
31:10and methodology.
31:13So the the new uh policies of open data
31:19where each uh researcher should be
31:23particularly revealing their data to the
31:26degree possible to the uh society and to
31:29the scientific community to be verified
31:32plays a positive role and has there have
31:34been some recent studies there.
31:38Okay. And in addition, this is an even
31:40older literature
31:42that shows how problematic is the degree
31:46to which publication bias exists, how
31:48widespread it is across disciplines and
31:51how important are the limitations of uh
31:54peer review.
31:57Uh this is important because peer review
31:59is the fundamental cornerstone of uh
32:02modern science. Uh what is considered
32:06significant enough? what is considered
32:09of sufficient quality in order to be
32:12added to the corpus of scientific
32:14knowledge.
32:17This essentially has been
32:21that decided using the institution of
32:24peer review. Uh similar researchers
32:27decide on the
32:30uh on what should be uh revealed
32:35uh
32:37on what should be published, what should
32:39be uh added to the corpus of knowledge.
32:42And uh this is the way this is the
32:45safeguard of uh quality and uh uh for
32:49science.
32:54And in addition, this is also a current
32:56work we have conducted in synopsis and
32:58with in other research groups
33:03uh examining how incentive structures uh
33:06matter for science rewarding quantity
33:09over quality. For instance, this is an
33:11important uh underlying
33:15uh problem behind the initiative and uh
33:20how to improve this uh current
33:22incentives is a is a key challenge. But
33:25first meta research comes forward and
33:29uh shows how this uh incentive
33:32structures reward
33:34quantity over quality.
33:47So at this stage,
33:50let me
33:52let me ask you a little bit uh whether
33:57you have been able to uh to follow the
33:59presentation so far and whether there
34:01are any uh general questions to be
34:04addressed.
34:12Are there any questions so far?
34:21>> No.
34:23>> All right.
34:25>> Very good.
34:26>> Can you please just now that we we have
34:29a small interruption, can you please
34:30press this hide hide in your screen
34:34because we can't see
34:37the full now we have the full
34:43>> thank you. So uh
34:49as I hinted uh before a lot uh after the
34:54discipline of meta research has matured
34:58one of the most important uh uh aspects
35:01was to uh find solutions to these
35:04reproducibility problems. Make sure that
35:07researchers who start their career now
35:11start with the right foot on how to
35:14follow uh best practices, how to conduct
35:18their obesity research, how they don't
35:20fall into the trap of uh
35:24excessive uh flexibility, lack of
35:26transparency
35:28uh no and the lack of open research
35:32which uh to a large degree we have been
35:34at hold for the problems that were uh
35:37substantiated before.
35:40So uh for instance here you can see two
35:44key platforms
35:46where uh in a systematic way and with
35:50systematic guidance modern researchers
35:53can uh upload not only their data but
35:57also their research protocols,
35:59methodologies,
36:01analysis and code uh publicly available.
36:04So this is a standard practice that uh
36:08many researchers who are inspired by uh
36:13the meta research movement are
36:14following. So in other words, whenever
36:16there is a new article
36:20uh either in the form of final
36:22publication or a working paper, it is
36:24always accommodated by the link at the
36:27open science foundation or
36:30at GitHub where all this information is
36:33uh is available
36:36and I have to emphasize that with the
36:39development of AI these uh sort of
36:42documentations which are traditionally
36:45considered burdensome and uh taking time
36:48and reducing the uh the potential
36:52productivity of researchers those will
36:53be streamlined and much faster to do so.
36:56So that will be something that
36:57technology will do for open science and
37:01uh I'm very optimistic about that uh
37:03making all this uh a standard practice
37:06and relatively low cost of time for
37:10scientists.
37:12So let's go into maybe uh the most
37:16popular approach in experimental science
37:20uh about how to deal with the problem of
37:24uh uh non-reproducible research bias
37:28uh flexibility methodology as I hinted
37:32before uh this uh
37:36this method is pre-registration and uh
37:38accompanied with a pre-analysis
37:41Pre-registration
37:43essentially says that before conducting
37:46an experiment you publicly reveal that
37:49you will do so you publicly reveal your
37:52hypothesis what you want to examine uh
37:55by this by this experiment the sampling
37:57process who are going to be the
38:00participants how they will be chosen
38:03uh the sample sizes so all the details
38:07about the uh data generation process
38:10This part is pre-registration and the
38:13pre-analysis plan concerns the
38:16specification of the future statistical
38:19analysis data analysis that will be uh
38:22conducted.
38:24So currently as we as we know so far
38:28from meta research uh uh evidence
38:33registration together with a
38:34pre-analysis plan is feasible in uh
38:38experimental research. Again there is an
38:40active discussion and we are also
38:42participating in that to what degree
38:46uh reanalysis plans tying our hands on
38:49the uh statistical analysis we conduct
38:53whether it can be used to non uh
38:57experimental empirical analysis. But
39:01generally this is this is difficult
39:02because as long as the data have been
39:05out there it is relatively hard to
39:08verify almost impossible to verify that
39:10the analysis have not been conducted
39:12already and then selectively
39:16uh revealed in a pre-analysis plan. It's
39:19not truly a pre-analysis plan. So there
39:21are issues of trust there. As long as
39:23the data have not been generated yet,
39:25this can be quite uh a substantial
39:28improvement.
39:30Okay.
39:33So one closely related concept that uh
39:38many of uh our junior researchers here
39:40should be very conscious about is the
39:43concept of confirmatory from exploratory
39:45research. So in other words when we
39:49conduct a new empirical study we should
39:51always start with some theory some
39:53strong theoretical hypothesis. This is
39:56called confirmatory research. As long as
39:59we do that, we have strong hypothesis,
40:01then we can use the usual interpretation
40:05of statistical tests and statistical
40:06evidence as testing a hypothesis
40:09providing us with a strong support as
40:12long as our significance level is
40:14particularly high. In other words,
40:16unless this association that we have
40:19hypothesized existed, it would be very
40:22unlikely
40:26that we would have been able to observe
40:28this kind of uh
40:31data patterns.
40:34Okay. But in order for this whole
40:36statistical process to work out should
40:38be based on clear hypothesis.
40:42Okay. If you don't have a hypothesis but
40:44you have a large number of regressions
40:46with all sorts of uh
40:49statistical tests being conducted on uh
40:52uh
40:54on different uh on different domains and
40:58uh you you cherrypick what kind of tests
41:02you you basically report. This is a pure
41:04violation of the the principles of of
41:07statistics because beforehand you should
41:10have hypothesized clearly on the
41:13direction of effects.
41:15So in summary
41:18confirmatory research starts from
41:20rigorous theory rigorous uh hypothesis
41:25supported by by pre-registration
41:28especially.
41:30So when you have this kind of findings
41:32in the literature where you have uh
41:35previously clearly made your hypothesis
41:39uh salient and transparent and the
41:42results uh supported this this is
41:44confirmatory research. It's much
41:47stronger than exploratory research which
41:50is the second category where you don't
41:52have uh clear evidence. I'm sorry where
41:56you don't have strong hypothesis.
41:59So as I told you before you conduct very
42:02many uh statistical tests in in a given
42:06analysis some of them show you
42:08interesting patterns but these were not
42:10hypothesized. We don't really know why
42:12they are happening but nevertheless they
42:14might be interesting in order to reveal
42:16in a scientific article maybe for future
42:19exploration. The important thing is as
42:21young researchers uh what should be
42:24conscious is to be clear, transparent
42:26and uh honest regarding which of the
42:29results were confirmatory and which were
42:32exploratory. This is this is the key and
42:34current in research practices this plays
42:37an important role
42:40and of course I think it should be clear
42:42by now that these strategies of
42:43prehacking and targeting that we said
42:45before cherry picking which results to
42:48reveal uh making a
42:52theorization after the results are known
42:55those are going to be ruled out as long
42:57as there is pre-registration in
43:06in practice for pre-registration
43:08including the open science uh uh
43:10framework as predicted uh the an
43:14American economic association randomized
43:16control trials registry and so on and so
43:18forth. So there are several options and
43:21importantly for starting researchers
43:24there's a guided process in these
43:26platforms about how to conduct uh
43:29pre-registration and pre-analysis plans.
43:31So to make sure that you can proceed and
43:34you have a complete uh information about
43:39your studies
43:44as I mentioned before
43:46uh in terms of pre-registration what is
43:48clear is the initial uh research
43:52question how many subjects are going to
43:55be chosen and how uh inclusion exclusion
43:59criteria and going uh
44:04and going further into a pre-analysis
44:07plan. Uh
44:11we we will be adding information about
44:13the actual analysis will be conducted,
44:15which are the variables that will be
44:17measured, how they will be measured.
44:20uh and of course the the specifications
44:22of the uh statistical analysis which
44:25exact tests are going to be used what
44:28will happen in the cases of uh outliers
44:31and so on and so forth. So this is the
44:34new tendency very strong and for several
44:37domains of research uh it's actually
44:41increasingly enforcable which means that
44:44uh for important uh uh aspects of the
44:49reputation of the researcher the ability
44:51to publish and so on and so forth
44:53registration plays an important role. So
44:55it is important for junior researchers
44:57to start utilizing as soon as possible
45:00preferably for the totality of uh of
45:03their careers and of course under the
45:05guidance of more uh senior researchers.
45:10Another uh popular approach is the uh
45:14the use of open methods and code. Uh so
45:17this is part of the more general
45:21uh movement for uh for open science.
45:27So it has been used for a uh for a long
45:30time for uh
45:32um
45:37uh for a long time. It has been
45:39discussed in the uh in the open science
45:43uh movement from the perspective of uh
45:47h
45:49from the ethical perspective if you wish
45:51that openness is good. So it is
45:54important for all the evidence to be out
45:55there. There's no reason to have uh
45:58science uh results methodologies hidden
46:02not open to the public. Meta research
46:06after it has uh come to the to the front
46:10have shown also the consequences of not
46:13having open methods in terms of the
46:15credibility crisis. So now the two
46:17things have come together. The general
46:20philosophical principles, ethical
46:22principles of open science for having
46:24more openness and the more practical
46:27advantages of open uh science in terms
46:30of uh increasing their reproducibility
46:32of results.
46:39So these are sort of more technical uh
46:44details and guidance on on open methods
46:48uh using modern tools such as uh Jupyter
46:52notebooks. Uh it is possible to uh
46:57document this the processes of the uh of
47:02the coding uh methods much more detail
47:05and often times even in in real time to
47:08make everything transparent, make
47:10everything reproducible to make sure
47:13that we know exactly how to produce the
47:15same results. And it is important to
47:18emphasize that uh
47:21oftent times uh this seems like an easy
47:25process to reproduce using the same code
47:27but it can be very difficult when you do
47:30it in practice. from per personal
47:33experience. Uh
47:36these uh reproducibility
47:39checkins
47:40uh checks, excuse me, when you have uh
47:45been uh successfully
47:47uh successful in publishing your work in
47:50uh in top journals uh are increasingly
47:55adopted. In other words, the journals
47:57come back and say, "We are interested in
48:00your study. This is good. We will uh
48:02publish it. But we kind of need to uh
48:09uh conduct a reproducibility check
48:11beforehand.
48:12Okay. And then when often times this
48:15process takes months and when this
48:18happens often differences from the
48:20published results to the uh to the uh
48:26reproducibility efforts uh to the
48:29replication efforts excuse me come to
48:31the front. In other words, we reproduce
48:35we we produce some initial results. We
48:38publish them in a paper. We describe
48:40them and sometimes when you want to
48:43reproduce them just prior to publication
48:45there are differences sometimes small
48:47sometimes large.
48:50All those issues would be avoided if in
48:53the beginning of a research progress uh
48:56process of uh methods open methods and
49:00code are implemented.
49:04Okay. If from the beginning the
49:06protocols are shared, the analysis is
49:08publicly available the workflows are
49:10reproducible.
49:12These things that are actually
49:15now imposed uh exposed
49:18would be available could be resolved uh
49:22exactly. Okay. So again once more for
49:25junior researchers it's important to
49:28start now by being very conscious about
49:32the importance of sharing methods and
49:34codes publicly in order to get more
49:36feedback in order to get more
49:39recognition about the open practices in
49:41order to be prepared for this internal
49:43application process that I just
49:45described and of course because once
49:48more uh the cost of these adjustments uh
49:52strongly decrease ries with the
49:54artificial intelligence methods and I'm
49:58optimistic that these things will be uh
50:00very still soon mainstream with a
50:03fraction of the cost that they use uh to
50:05have
50:08okay and of course often times small
50:11differences in the replication process
50:14come from uh different software versions
50:17and dependencies and all of those things
50:19will be strongly smooth smoothed out by
50:22having the methods are probably publicly
50:24available.
50:26And of course, this will be very
50:28important in the uh process of uh
50:31catching errors both for the feedback
50:33but most importantly for the process of
50:36cleaning our code to make it uh
50:39presentable, understandable to uh to
50:43somebody else that will allow us to
50:45catch most of the uh of the problems.
50:50I will have to apologize at this moment.
50:52I would really need to make a a small
50:55water break. Uh I will be back in in one
50:58minute. Once more, please kindly accept
51:00my apologies.
53:47Thank you so much for your patience.
53:49Apologies for the uh for the break. Uh
53:54can I ask you if there are any issues or
53:55any questions that might memorize?
54:03Everything good?
54:04>> All good, Professor Man. Thank you.
54:07Okay.
54:10So uh
54:12we'll get into the particular uh problem
54:16of open data which means uh um
54:21statistical uh uh quantitative
54:24information about uh uh phenomena in
54:28nature codified into
54:31uh into data when can they be publicly
54:35revealed? uh what would be the
54:37advantages and disadvantages of doing
54:39so. From the perspective of open science
54:42uh it is clear that uh this would be a
54:46beneficial process to have uh data as
54:49publicly available as possible. At the
54:52same time it should be acknowledged that
54:54uh uh
54:58there are some limitations often times
55:00legal implications in the uh revelation
55:04of data depending on how Of course they
55:06were uh produced because obviously data
55:09can be uh used for uh
55:13having financial and otherwise
55:16advantages
55:18uh by exclusive exploitation of the data
55:20and therefore the data is a sort of
55:23unique aspect of the uh of open science
55:27that needs to be having special
55:29consideration.
55:32So some of the ethical considerations
55:34that should be uh minded when you think
55:37about open data is the issue of
55:39identification. Sometimes you have
55:42participants in experiment for instance
55:44and uh they should not of course be
55:46identified. There should not be uh
55:49revelation of uh private information
55:53regarding their behavior uh in economic
55:56experiment for instance. Uh
56:00so
56:01it's important to share this data but in
56:04a deidentified way.
56:07Of course an important uh advantage an
56:11important benefit of open data is sort
56:13of uh clear by our discussion so far
56:16allow for verification the reanalysis
56:20and the summary of the overall evidence
56:22by means of meta analysis by pulling uh
56:25data from different studies together and
56:28from the individual perspective of the
56:30researcher if we make our data publicly
56:33available it allows for further analysis
56:37increasing also our citation rates and
56:39and impact.
56:42But there this is a domain where the new
56:45generations of researchers should be
56:47more conscious and know more about the
56:50uh legal issues of it. Okay. Issues of
56:54privacy, consent, sensitive information.
56:57uh how from the technical point of view
57:00to use the appropriate repositories how
57:02to make the appropriate rights uh for uh
57:07uh for the use of the data. So these are
57:09aspects that uh it is clear that the new
57:14generations of researchers should be
57:16more uh prepared to take into account
57:19and I think that the more time passes
57:21the education the scientific education
57:24of junior researchers should incorporate
57:26these things making them more mature to
57:29the legal issues uh behind it. So for
57:33instance uh when we uh have a uh grant
57:37application when we receive funding for
57:40our uh work we are often asked to have a
57:44data management plan which addresses all
57:47these considerations
57:48making things as open as possible but at
57:51the same time respecting the uh legal
57:55and ethical aspects and uh showing uh
58:00from a technical point of view how it is
58:02physical that you achieve those
58:03objectives. So it is important that in
58:07uh the modern uh uh development of
58:11junior researchers these things are not
58:14learned as our generation uh empirically
58:18when we had a only when we had a uh
58:22funding application process but a modern
58:26scientific education makes uh people
58:29more conscious of this of these concerns
58:31earlier.
58:36So to all the issues that we mentioned
58:38before uh an important role is played
58:42from individual incentives.
58:46Uh so why do uh researchers
58:51conduct these problematic practices? Why
58:55are they not always?
58:58Why don't they al always uh make data
59:01openly available when necessary? Why
59:05don't they always make code available?
59:08Why do they engage in practices such as
59:10P hack? All of those things can be
59:12traced back to the incentive structures
59:15of uh the researchers. What are
59:18researchers rewarded for? So going into
59:20the question of what they are rewarded
59:22for uh leads us to a deeper
59:26understanding of why sometimes science
59:29can uh move astray can can go astray
59:34into problematic uh domains.
59:39For a long time it has been assumed that
59:43traditional uh metrics are uh rewarding
59:48quantity rather than quality. So for
59:50people who are not very familiar with
59:52research metrics and how research is
59:54assessed.
59:56Uh important times researchers produce a
59:59large number of articles.
1:00:01The difficulty is to assess the quality
1:00:04of the science included in these
1:00:07articles. So what is the quality of
1:00:10research that I have produced in my
1:00:12career in order to reward me for that to
1:00:15give incentives to do even better
1:00:18research and so on and so forth.
1:00:20It is a very difficult problem to
1:00:22consider especially if somebody has
1:00:25produced a large number of articles
1:00:28because uh of the time necessary to read
1:00:32in depth all those articles and to make
1:00:35strong assessments about them. In most
1:00:39situations, it's very unrealistic to
1:00:41believe that in every assessment uh
1:00:44scenario, for instance, when I apply for
1:00:46a research position, when I apply for
1:00:49promotion, where I apply for some uh
1:00:53reward
1:00:55that the uh committees will be able to
1:00:58read all of my research. That's where
1:01:01the so-called research metrics play a
1:01:03role. Those are automatic tools that use
1:01:07uh online uh material and make a summary
1:01:12of the quality of research that I
1:01:15produce. For instance, uh age index
1:01:18impact factor citations. So it's
1:01:21publicly available. How many other
1:01:23works, how many other uh scientific
1:01:27articles have referred to my own work?
1:01:30This is called citation. So with metrics
1:01:33one can see immediately how many
1:01:34articles that a researcher has, how many
1:01:38citations they have and other more
1:01:40concrete and more specialized uh uh
1:01:44metrics such as the age index and the
1:01:46impact factor. So the age index is a is
1:01:50an effort to make sure that uh somebody
1:01:53has a good a critical mass of important
1:01:57work. at the same time many uh studies
1:02:01and at the same time impactful too
1:02:04rather than having a very large number
1:02:06of uh works that nobody has ever uh
1:02:09cited uh before or that they have a
1:02:13unique lack if you wish uh
1:02:18run that has received thousands of
1:02:20citations but they haven't really
1:02:22followed up with with additional
1:02:24research anyway. So these are specific
1:02:27technical uh uh
1:02:31uh metrics
1:02:33but the problem with them is that uh
1:02:36they can be gained. In other words, they
1:02:40are used as proxies to the quality of
1:02:42research. But after they become public,
1:02:45we know
1:02:46people spo stop responding
1:02:50uh in the
1:02:52impassionate manner of just doing my
1:02:54best to produce my research and then
1:02:57these metrics will be just captured that
1:02:59I will be conscious of the metrics
1:03:01beforehand before I conduct my research
1:03:03and I will adjust my practices. All of
1:03:06us uh basically uh will adjust our
1:03:09practices in such a way to uh
1:03:13to be rewarded to be positively assessed
1:03:18using these methods. So this is famously
1:03:21captured by the concept that when a
1:03:23measure becomes a target, it ceases to
1:03:25be good. Instead of trying to produce
1:03:27the best possible research, we try to
1:03:29achieve the best possible metrics.
1:03:34So the uh movement for the responsible
1:03:37use of metrics criticizes specific types
1:03:40of uh uh metrics that they are very
1:03:43shallow that they don't show the a
1:03:46well-rounded uh picture of the of a
1:03:50researchers's careers and therefore
1:03:52essentially that they distort incentive
1:03:54for researchers often times resulting to
1:03:57the problems uh their reproducibility
1:03:59problems that we mentioned before.
1:04:03So responsible metrics it has been
1:04:05argued must be adjusted for context uh
1:04:10specific discipline, career stage,
1:04:13research type and the way that they
1:04:16should be used is in in a not in a naive
1:04:19way but in a more sophisticated way such
1:04:22as using multiple indicators
1:04:25uh combined with qualitative assessments
1:04:28and so on and so forth. So from the
1:04:31junior researchers point of view, these
1:04:34comments have been a little bit uh above
1:04:38your head if you wish because they they
1:04:40speak a little bit from the perspective
1:04:42of the the research assessor of
1:04:45established researchers. But when you
1:04:47don't have your h your research yet, uh
1:04:51how should you why should you be
1:04:53interested into that as as a startup
1:04:55researcher?
1:05:00For instance, I will clarify this now
1:05:02and I will return. You should be ready
1:05:05to present yourselves to present your
1:05:08work in the form of a
1:05:11a new approach if you wish. It is called
1:05:14the narrative CV. So as you as you grow
1:05:17in your scientific journey, you will be
1:05:20having a CV that has a
1:05:23a fixed form. Most notably, it will have
1:05:27a list of publications. Okay? And then
1:05:29you will have most likely a scholar
1:05:31Google profile which will show also your
1:05:34uh your citations.
1:05:36The important thing and how this should
1:05:38concern uh junior researchers is to keep
1:05:41in mind that you should also have at the
1:05:45same time an Arabic C.
1:05:48Generally speaking, how I view an
1:05:50article a bit is that it it describes
1:05:53your research journey. Okay? Every time
1:05:57that you are attracted to a research
1:05:59problem and an agenda, there is a little
1:06:01bit of a story behind that. Why why have
1:06:04you been interested in that research
1:06:06question and then why were you drawn in
1:06:08the other projects and so on and so
1:06:10forth? How do you make all of your books
1:06:14consistent? So essentially
1:06:18uh what were what were your
1:06:19contributions in the
1:06:23you know in any given uh research
1:06:25project? What were you why were you
1:06:27attracted to that? What do you think you
1:06:29have learned from this exercise in your
1:06:32own personal tone rather than just uh
1:06:35presenting a shallow bullet point set of
1:06:39bullet points of individual uh
1:06:41publications?
1:06:45So the narrative simply contextualizes
1:06:48the descriptions, emphasizes quality,
1:06:50emphasizes the personal aspect, the
1:06:53personal contribution within a research
1:06:55pro uh a project, the exact role in the
1:06:58team, why you were attracted into that,
1:07:02why did you expect this to achieve terms
1:07:04of social societal impact? Okay. And uh
1:07:09other aspects such as mentoring and
1:07:11public engagement. Uh so once more with
1:07:15the new tools from AI uh it is
1:07:18increasingly uh low cost to produce high
1:07:21quality uh uh material and at the same
1:07:27time of course adjusted to your exact
1:07:29personal uh research journey and
1:07:32prospect and uh and perspective excuse
1:07:36me of uh you know how you made your
1:07:39choices, what were your contributions,
1:07:41how do you think are the uh the
1:07:44important uh implications of your of
1:07:47your research and so on and so forth. So
1:07:50again for junior researchers should be
1:07:53already focusing on uh more qualitative
1:07:56presentations of your work at the same
1:07:59times those should be combined
1:08:02before
1:08:03uh traditional quantitative metrics. For
1:08:07instance, we have a recent paper where
1:08:11you have three possible candidates that
1:08:13are caused by example one, example two
1:08:15and example three. Instead of just
1:08:17checking the uh if you wish the the
1:08:20number of their citations or the number
1:08:22of their publications, you see a much
1:08:24more advanced picture using a large
1:08:27number of uh indices to account for
1:08:30different uh uh parts of their
1:08:34productivity, their career, their impact
1:08:37to their discipline and maybe of
1:08:40possible possibilities to capture
1:08:42problematic uh practices. Okay, we don't
1:08:45have enough time to do that but we can
1:08:46see that there are not just one metrics
1:08:48but one metric but very many different
1:08:51metrics that can be carefully used
1:08:54combined uh with uh a narrative story if
1:08:59necessary in order to really be able to
1:09:02properly assess uh researchers.
1:09:06So now I will sum up this uh this part
1:09:10by speaking about the dor and the
1:09:13principles.
1:09:15Those were the key initiatives that
1:09:18argue favor of uh
1:09:22responsible research metrics.
1:09:24Okay. So they are closely related to the
1:09:29uh movements in within Europe. For
1:09:32instance, uh UK is also one part of
1:09:35them. the the effort to be uh consistent
1:09:39with the principles of these initiatives
1:09:41and declarations.
1:09:44So basically, roughly speaking, they
1:09:46just uh uh both of them argue in favor
1:09:50of a more holistic and well-rounded
1:09:53approach to research evaluation, not
1:09:55only shallowly based on uh a
1:09:58quantitative indicators taking into
1:10:01account of diverse contributions,
1:10:03uh social impact uh and so on and so
1:10:07forth.
1:10:16Okay. So we have the uh
1:10:20the narrative situation. What's it
1:10:22included more or less is what we
1:10:24discussed uh discussed
1:10:27discussed before. uh the specific
1:10:30contributions within a uh within a team,
1:10:33the implications of the
1:10:35of the project, potentially
1:10:39uh potential methodological innovations,
1:10:41impact and so on and so forth, as well
1:10:44as uh team building and mentoring and
1:10:48your role in this uh uh in this
1:10:51engagement.
1:10:55I think we might be running a little bit
1:10:58uh low in time. Uh
1:11:02how much time do we have left?
1:11:06>> Five more minutes.
1:11:10>> Okay, great.
1:11:13So I will uh summarize the discussion
1:11:16with the recent identity integrity
1:11:19aspects. Apologies and uh I will leave
1:11:22the last part with the interact
1:11:25interactive question for uh sorry
1:11:29interactive uh session for uh
1:11:32considering when we share the uh this
1:11:35lecture notes. Uh so another important
1:11:40aspect that the junior researchers uh uh
1:11:44should know are all the aspects of
1:11:47research integrity. For instance, when I
1:11:50was conducting my first uh experiments
1:11:54uh with human participants, I uh I had
1:11:58to take an online course on the
1:12:00implications of uh uh
1:12:04uh
1:12:06implications of running experiments with
1:12:08with human subjects. the ethical
1:12:09principles that should be respected all
1:12:12the components that should be carefully
1:12:14followed in order to make this uh
1:12:17consistent with ethical principles and
1:12:19foundations.
1:12:22So you should all be familiar with the
1:12:25core principles of research integrity
1:12:27such as honesty, objectivity,
1:12:30transparency, accountability, fairness
1:12:33and respect.
1:12:36Okay. And in particular when this goes
1:12:40into personal interaction with
1:12:42participants
1:12:44who will generate uh data for scientific
1:12:48purposes. To this one should argue uh
1:12:52informed consent a respect for the
1:12:56personality and respect for the
1:12:58principle of non harm. So whenever there
1:13:02are interventions with human
1:13:03participants, we should make sure that
1:13:05they the benefits that they receive are
1:13:07always uh uh greater than the costs.
1:13:13But what should be important I mean this
1:13:16should shall not be an exclusive list
1:13:18for you. What should it be important for
1:13:20senior researchers is to make sure that
1:13:23they address uh aspects of integrity and
1:13:27uh uh within the scientific education at
1:13:31an early stage which I think still needs
1:13:35to be uh improved upon and for junior
1:13:39researchers try to get as much
1:13:42information as possible about these
1:13:44aspects to avoid the fact that later on
1:13:47in your uh research career you might end
1:13:49up conducting empirical studies or
1:13:52experiments and you find yourself a
1:13:55little bit immature regarding how to
1:13:58protect uh ethical principles and you
1:14:01know this might complicate things. So
1:14:03it's much much more important in the
1:14:05first place from the beginning to be
1:14:08clear about how you design your research
1:14:10in order to respect these principles.
1:14:13So finally what we should be uh very
1:14:17careful is to avoid question of research
1:14:19practices.
1:14:21The first category is pure fraud uh
1:14:24fabrication of plagiarism. There are
1:14:27very few scientists practically almost
1:14:30nobody goes into science with the uh
1:14:33objective of uh
1:14:36presenting false data. However this
1:14:40happens but very frequently. But what is
1:14:42catch for frequent is what we have been
1:14:44discussing before hacking uh hacking
1:14:49selective reporting and so on and so
1:14:51forth. These fall into the category of
1:14:53question of research practices
1:14:55uh sometimes one can be also held
1:14:59accountable for problematic practices
1:15:01there. Sometimes it is an issue of gray
1:15:04zone but uh as especially as young
1:15:08researchers just make sure that you
1:15:11don't do any of those practices. You use
1:15:13pre-registration, you use open science
1:15:16to exclude that you use this kind of
1:15:19problematic practices that previous
1:15:21generations were subject to meta
1:15:25research. Uh this might be the most
1:15:28important uh uh contribution of meta
1:15:31research. it goes to the front and shows
1:15:33how you can avoid uh doing all those
1:15:36problematic practices.
1:15:42So this is just an example. This is just
1:15:45an example and I will sum up with this
1:15:48of uh gray areas and difficult
1:15:50decisions. When is it a decision that
1:15:54must be
1:15:55followed versus
1:15:59one
1:16:01that should be avoided on the basis of
1:16:03ethical principles.
1:16:05How should you deal with outliers
1:16:08from the one perspective? One number
1:16:10might be very extreme in your data
1:16:14uh which make things unrealistic and
1:16:17biased. So it should be removed. On the
1:16:20other hand, if you're dealing with
1:16:22outliers who have not been to specified,
1:16:25for instance, in a British registration
1:16:27document, there concerns
1:16:30to what degree will the removal of
1:16:32outliers be uh
1:16:37self-erving.
1:16:38So for instance when you observe the
1:16:40data with and without the outlier what
1:16:43kind of temptations will be to uh have
1:16:47flexibility what's keep
1:16:53registration
1:16:54uh what kind of how much mile what kind
1:16:57of exploratory analysis should we have
1:16:59done before the registration this is
1:17:01still an open question and a gray area
1:17:05uh despite the progress that have been
1:17:10Authorship still remains uh an issue.
1:17:13Who has contributed enough? In some
1:17:15domains, uh lab managers are uh part of
1:17:20a publication
1:17:22uh for everything that is produced with
1:17:26within their lab. Should this be the
1:17:29case or not? Again, this is a little bit
1:17:31of a of a difficult uh decision.
1:17:34uh
1:17:38how how do you deal with the
1:17:40relationship with your supervisors and
1:17:43partners? This is extremely difficult.
1:17:45What do you do in cases where you uh you
1:17:49see that the practices followed by uh
1:17:52supervisors might might be problematic.
1:17:54This is extremely difficult especially
1:17:57given the importance of a supervisor in
1:18:01uh uh in your career and in your
1:18:05development. Then that emphasizes
1:18:07particularly for junior people the
1:18:09importance of uh choosing uh supervisors
1:18:13in a uh you know in the mo in the most
1:18:16mature possible way and having access to
1:18:20uh the people who will be able to help
1:18:24in in terms of this uh these great areas
1:18:27in terms of research. So I think I think
1:18:30this is it. We're out of time. There
1:18:34there are some uh interesting particular
1:18:37scenarios that we are that we are
1:18:39considering. These can be left as food
1:18:41for thought for uh for the participants.
1:18:45Uh
1:18:47and these are the key uh takeaways from
1:18:50uh today's presentation. It's been a
1:18:53real pleasure. Apologies for not being
1:18:55there and being able to deliver it uh
1:18:58live. Uh so I don't know if we might
1:19:02have a little bit of uh time for
1:19:03questions. Thank you.
1:19:06>> Thank you very much Professor Madavis
1:19:08for accepting to do it even though you
1:19:11could not be physically here. Do you
1:19:13have any questions for Professor
1:19:15Madavis? There's nothing from the online
1:19:17attendees.
1:19:20We will make sure that you we share all
1:19:23your contact details together with your
1:19:25presentation. So uh later in future if
1:19:28someone wants to contact you for uh
1:19:31anything further uh they will. Thank you
1:19:34very much for your insight. Thank you
1:19:36very much uh for being with us and have
1:19:39a nice day ahead. Thank you.
1:19:42>> Thank you. Thank you so much.

About this transcript

This page contains the full transcript of Responsible Research and Scientific Integrity by YUFE Alliance, generated from the public captions YouTube serves with the video. The transcript has 9,313 words across 1,546 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to translate the transcript into 100+ languages.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, translation to 100+ languages, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.