YT YouTube2Text

YouTube2Text → Transcripts → Complex Systems - Jean-Philippe Bouchaud - Lecture 3: Multiplicative Growth (II). Redistribution

Complex Systems - Jean-Philippe Bouchaud - Lecture 3: Multiplicative Growth (II). Redistribution — Transcript

by EconophysiX Lab · 23,292 words · 3,978 segments · language en · Watch on YouTube

Open in transcriber Summarize with AI Download SRT/VTT

Full transcript

0:01this conference will now be recorded
0:04okay so it's 902 so let's slowly get
0:07started
0:09um so
0:11I talked about random multiplicative
0:13growth and I'm going to continue talking
0:16about this today
0:18so I told you about independent growth
0:20uh objects growing independently from
0:22one another
0:24nothing much happens except that if
0:26you're looking at a system of a large
0:29number of in of non-interact interacting
0:33uh growth
0:36objects growing then I told you that
0:39there's this interesting concentration
0:41transition after a while
0:45um the halfindl index of for example if
0:48you think about cities in a country the
0:51half indoor index of these cities will
0:53become non-zero even in the large
0:56population size in the largest country
0:59size
1:00and in order to illustrate this I
1:03promise to show you something last time
1:05but I couldn't so I am going to do it
1:07today which is an illustration of
1:12of this concentration phenomenon
1:15okay so this is the the graph I want to
1:17show you the the upper graph here
1:20uh so usually I talk about this graph as
1:24the story of the grocerer versus the
1:28jeweler
1:29so what am I plotting here
1:32um so on the x-axis is the number of
1:36items that's been sold either by the
1:39grocerer that's going to be the black
1:41line or the Jeweler and that's going to
1:44be the the red line
1:46and what I'm doing here is I'm imagining
1:49that each item that is sold is has a
1:54price
1:55that is distributed according to a
1:58parallel distribution with some index mu
2:02and so what I'm plotting on the y-axis
2:06is the total sales after having sold end
2:10product okay
2:12so on the y-axis here you have the total
2:15sales
2:16after having sold end products on the
2:18x-axis
2:19and so the black line corresponds to Mu
2:23equal 2.5 so this is numerical
2:25simulations
2:27and U equal to 0.5 you remember means
2:29that the underlying distribution has
2:32uh well-defined mean and a well-defined
2:35variance
2:37and in this case the central limit
2:39theorem holds and actually the law of
2:41large number holes that's what you're
2:43seeing in action on this graph
2:45and with the law of large number means
2:47is that the total sales is growing is
2:51going to grow linearly with the number
2:52of items sold and the slope of this
2:55black line is just the average price of
2:59the of of items in the in the grocery
3:02store
3:04and what's in a sense uh
3:08interesting to realize although it's
3:10trivial is that this is not exactly a
3:12straight line there are fluctuations but
3:14you don't see them by the eye
3:17um here there are a total 10 000 objects
3:21that are sold and on this scale it looks
3:23like a a straight line although we know
3:27that there are fluctuations there are
3:29fluctuations of all the square roots of
3:31n but you you don't even see them
3:36and so the grocery store is a place
3:39where you can find
3:42objects that are a little more expensive
3:44than than others like I don't know a
3:46bottle of champagne or caviar or things
3:49like that but it never goes very far
3:51right
3:52and the Jeweler on the other hand I'm
3:55assuming he has a distribution with
3:57power law with index me equal 0.5 so
4:00less than a half and in this case uh the
4:04graph looks completely different
4:06so this is the concentration effect and
4:09the concentration effect is that
4:12the whole total sales after 10 000 items
4:17sold is dominated by the largest one
4:20more than a half of the total sales is
4:22due to this single sale here which might
4:26be I don't know you know energy at a
4:28jeweler you can find unique gems that
4:32are incredibly expensive actually 10 100
4:37000 times more expensive than the things
4:39that you sell every day but what I want
4:43you to realize is that this is a scale
4:46invariant curve this is sometimes called
4:49a devil star case because there are
4:52steps of all sizes and if you zoom for
4:55example on this region here you have the
4:58same dominance effect the fact that even
5:02on on this little scale here you see the
5:04big jump here is going to dominate the
5:06picture and even if you zoom here where
5:09you you think that there's nothing but
5:11actually the Jeweler has been active
5:14even in this region and if you did zoom
5:16in this region you would see exactly the
5:18same statistical picture that is uh
5:20domination of extreme events
5:23and so this is you know visually what it
5:26means to have
5:27a non-hero handle index it means that
5:31some big chunks of the sum
5:34are realized by
5:37single events okay
5:40so that's what I wanted to share to
5:42share with you
5:44and now I'm going to go back
5:48to the screen
5:52sharing
5:56okay
6:00okay
6:01any question about what I just said or
6:05about you know something that I said
6:08last time and that's some
6:10someone would like to
6:13go back to
6:16okay
6:18okay so as I said
6:21what I talked about is independent
6:23growth and concentration now I'm going
6:24to introduce some interaction between
6:27these growing objects
6:29and I'm going to call this interaction
6:31redistribution so this is going to be a
6:34general uh word to say that
6:38growth is going to happen you know
6:41within different objects but sometimes
6:43the result of that growth is going to
6:46jump so to say from one object to the
6:49other so let me write down the equation
6:51I have in mind and then I'm going to
6:55to comment but I'm writing
7:00so remember z i is the thing that's
7:04growing
7:05and so think of it as the population in
7:08city I said I
7:10and so what I wrote last time was
7:13something that was m i Plus
7:17beta I
7:20of t
7:21that I
7:23so this is what I've called growth
7:28this is multiplicative growth because
7:31it's proportional everything here is
7:33proportional to z i
7:35so this is the mean growth rate of CTI
7:37this is the random part taking into
7:40account health conditions or whatever
7:44and now what I'm going to add to this
7:46equation is some coupling between cities
7:49so you know it's very natural in the
7:51context of cities that people you know
7:54of course they they they have children
7:57and the population growth but they can
7:59move from one city to another right and
8:03so what I'm going to assume is that
8:06uh the more people you have in a given
8:09City the more probable it is that that
8:12some of these people will leave and go
8:15elsewhere
8:16so what I'm going to write is uh a
8:19transfer term
8:20so sum over J
8:22of J
8:25Little J to I
8:27w z J
8:31minus
8:33sum over J of j i to J
8:38the said I
8:40okay
8:42so this is transfer
8:47people moving from one city to the next
8:50so
8:51j j i is the probability per unit time
8:56that someone living in Con in City J
8:59moves from J to I okay and the same year
9:02this is a probability per unit time that
9:04someone in the city we're looking at I
9:08moves from I to J okay so this is a very
9:12general equation because I haven't
9:13specified what this Matrix j i j is and
9:18I'm going to say more in a second
9:20but you know imagine another uh
9:24framework for this equation which would
9:26be
9:27the growth of viruses with a sudden type
9:30so I would be the type of a virus you
9:34know very appropriate these days
9:36and so viruses can grow or disappear
9:40hopefully as an exponential rate
9:44but they can also mutate and so these
9:46would be mutation rates okay
9:49and you can mutate from J to I you can
9:52possibly mutate back from I to J and so
9:55this is the A model for uh mutation if
9:59you want
10:01foreign
10:04in terms of wealth
10:06I is an individual that is his wealth so
10:10there's a growth term and then by
10:13trading part of its wealth or high
10:16wealth can be uh you know handed over to
10:20other individuals or vice versa he can
10:23get richer by getting the wealth of of
10:27others through Trading
10:29okay
10:30so of course this is a very naive model
10:32but let's assume that uh this is
10:36something like that happens and the
10:38basic question I'm going to try to ask
10:40today is how
10:46does redistribution
10:55affect condensation
11:04okay I'm writing this question is full
11:06because this is going to be the main
11:08topic of today
11:12so let me frame this again for you you
11:15remember that without these transfer
11:18terms okay I've shown you that this
11:22independent growth processes
11:25at some point there is this
11:27concentration transition
11:30okay I said that Beyond some critical
11:33time scale uh the hassanal index
11:37is going to go from a zero value to a
11:40non-zero value and so if there's no
11:44transfer
11:45the longer you wait the more probable it
11:48is that you get you will get
11:50concentration or condensation
11:53so now intuitively
11:57it seems that if you allow from for
12:00redistribution mixing if you want if you
12:03allow for Rich guys to spend their money
12:06and enrich poorer guys if you allow
12:10large cities to spill over and have
12:15people moving to smaller cities then
12:17maybe you can avoid this condensation
12:19effect okay
12:21and so what I'm going to show to you is
12:23that it's actually quite interesting
12:25because depending on the conditions you
12:28can avoid conversation or you still get
12:31condensation if the transfer rate
12:35is small enough okay
12:38so that's what I'm going to talk about
12:40today
12:41so of course this model is completely
12:44General and there's a little you can say
12:48about it
12:49except one thing which is that these
12:52transfer terms they conserve math
12:56okay so let me write it here
12:59Mass conserving Mass
13:03and serving
13:07so what I mean by this is that in the
13:10absence of growth okay if I didn't have
13:12any growth then you can show it takes
13:16the really one minute that
13:19in the absence of growth if I sum Over
13:22All Eyes this equation so sum over I of
13:25VZ I DT then I find zero and the reason
13:30is that you know the total number of
13:32people going from uh J to i's are going
13:36to compensate the number of people going
13:38from I to J's uh on our average overall
13:42City I mean in words it's pretty obvious
13:44that moving people from One City to the
13:48other doesn't change the number of uh
13:50the the size of the population but you
13:52can show it formally so these terms are
13:56mass conserving whereas this one
13:58obviously is not and contributes to
14:01growth of the population okay
14:04so as I said apart from this General uh
14:09observation that these terms are mass
14:12conserving there's little you can say
14:14about this equation in full generality
14:17so one has to make uh the model a little
14:21more specific in order to say anything
14:25so I'm going to give a few examples
14:29and tell you more about these specific
14:33examples
14:34so one example is what people like to
14:38call in particular in physics the fully
14:40connected case
14:47and this amounts to say that j i to J
14:53is equal to some
14:55j0 over n
14:58sorry just be careful you're going out
15:00of telescope
15:02okay
15:05so let me check where I am
15:08okay now you've seen
15:11yeah line I'm going to
15:14you'll tell it's my
15:18line
15:20okay thank you
15:25so what does this mean it means that
15:28this there's sometimes this transfer
15:31rate is also called a Hopping rate
15:33so this transfer rate or hopping rate
15:36is the same independent of the starting
15:39point and of the end point and it's
15:42equal to uh
15:45a constant j0 divided by n the total
15:49number of sides so I'm going to
15:52specify the fact that there's an an end
15:54here that I've uh
15:56also talked about last time
16:00the same notations as last time and the
16:03one over n is going to be needed
16:06um in order to have a well-defined model
16:10in the limit when n goes to infinity and
16:13the reason is because you can go
16:14anywhere if you're in city I you can go
16:16anywhere to any other City J
16:19uh the total hopping rate if you want is
16:22n times j0 over n so it's j0 so it means
16:27that the fine that the total moving rate
16:30from one city to the next remains
16:33constant in the limit of large country
16:36so this is a very simple framework
16:39corresponds to uh what in statistical
16:43physics model is called the mean field
16:45limit
16:47and
16:49of course in this case when we are going
16:53to be able to solve the model
16:55and that's what I'm going to show you in
16:57a second
17:00another possibility is to have a random
17:03graph
17:10with the same J for all links but apart
17:13from that something that's that's random
17:16so let me draw something uh
17:18that looks random of course
17:21uh
17:22can be more complicated than this so
17:24every node is a city and every link
17:28is
17:30a a link and input in principle one
17:35should also specify a direction because
17:38maybe there are jumps that are possible
17:42from I to J and not from J to I okay
17:46so that's another case
17:49and the last case
17:52on which I'm going to say a lot is the
17:55regular graph
18:03so for example
18:04you know
18:06you could have
18:08a two-dimensional lattice like this so
18:11each node would be a city again and each
18:14link responds to a possible hopping rate
18:17so this means that there's only nearest
18:20neighbor hopping
18:22and it could be a model for uh
18:24population spreading
18:26so you know imagine that we're in uh uh
18:31ancient ages and people can only move
18:34locally uh from one uh site to a nearby
18:40site and in this case we would describe
18:44with this equation random growth of uh
18:48tribes and the fact that tribes can move
18:51uh walking by walking distance from one
18:56uh site to to a near near a nearby site
19:01so okay this is again a very simplified
19:03view but uh you can keep that in mind
19:08and of course you know you can imagine
19:11many more complicated things because
19:14here what I'm saying by uh by random
19:17graphs or regular graph is is the
19:19topology topology of the jij but I'm
19:22still going to assume that
19:25there's the same value
19:28of j0 on all the links okay
19:32so I'm neglecting the fact that the JS
19:35can also be link dependent
19:38so there's a whole variety of models
19:41that you can encapsulate in such a
19:44framework
19:45okay
19:47so
19:49so what I'm going to consider so here is
19:53the mean field limit
19:56because you're going to see that it's
19:58interesting in itself
20:00and uh we'll find a fast conclusion
20:04about the role of redistribution on
20:07condensation
20:09so I'm going to
20:11um assume for now
20:13that all the Mis are equal to m
20:18so
20:20sites are growing at the same average
20:23speed
20:24average rate M of course there are
20:26fluctuations that are independent but
20:29the average rate the average growth rate
20:31is the same
20:34and I'm going to um
20:37introduce
20:41something that is
20:43quite natural which is the average value
20:45of bed
20:47at time t
20:49which is defined
20:53by 1 over n
20:56sum over I of
20:59z i
21:02and of course the mean field will
21:04correspond to n going to Infinity
21:09okay
21:13so let's look at the general equation
21:17for the choice I'm making here so mean
21:20field is the response to
21:24jij equals j0 Over N for all I and J
21:28so let's see for example this time here
21:31you're summing n terms that are all
21:34equal to j0 Over N so this last term is
21:38just minus j0 times z i
21:41and this time here
21:43is j0 over n
21:46sum over J of ZJ so this is exactly j0
21:50times z bar okay
21:53so in this limit
21:57or in this case
22:00the equation on z i decouples or
22:04actually it's only coupled through the
22:06mean of Z but it's not explicitly
22:09coupled to individual JS anymore
22:11it's uh it's the same z bar that couples
22:15everybody so it's equal to M plus ETA I
22:19of t
22:22that I
22:24Plus
22:25j0
22:27set bar
22:29of t
22:30minus z-i
22:34okay
22:45so again this term here comes from that
22:47one which
22:48exactly reproduces this object and this
22:52term is that one
22:57okay so that's the equation I'm going to
23:00study now
23:03remember that I'm going to choose uh for
23:09now the stratonovich
23:15convention
23:16[Music]
23:22so the stratanovic convention you
23:25remember from last time it is to assume
23:28that SRI f t h i of T Prime
23:32or it's a j of T Prime even
23:35is Delta i j so growth is independent
23:39from side to side times uh Sigma squared
23:44over 2 Tau C
23:46exponential of minus t minus t Prime
23:49over Tau C
23:51and so I'm assuming that there is a
23:54finite but very small correlation time
23:57that allows us to deal with these
24:00equations as if ETA I was a regular
24:04function
24:06later on today I'm going to speak about
24:09another problem where I'm going to use
24:11ETO convention because it will be more
24:13natural to think in that case
24:16of of the process that's with zero
24:20correlation time as completely
24:22independent from one time step to the
24:24next but for now as I said last time I'm
24:27going to keep this simple way of
24:32of dealing with the
24:34with this process
24:36so
24:38for example this equation here you see
24:41it's a linear equation
24:43for all z i given that bar of t
24:46and so the explicit solution
24:50is z i of t
24:52equal
24:54uh say 0
24:57integral from 0
24:59to T
25:01or two minus infinity to T
25:04uh
25:05DT Prime
25:09exponential of M
25:11P minus P Prime
25:15minus j0 E minus t Prime
25:20Plus
25:21integral from P Prime to T
25:25DT double Prime
25:27ETA I
25:28of t double Prime
25:31everything acting on z bar of C Prime
25:36okay so all this is in the exponential
25:40and this is the solution this is a
25:42general solution of
25:45linear differential equation
25:48with a non-zero uh right hand side okay
25:53so don't worry too much you can redo it
25:56quietly but what the only reason I'm
25:59writing it writing this explicitly is
26:03that from this thing you can actually
26:06derive
26:07by summing it over all eyes
26:10so if I sum here
26:13this thing and divide by n
26:16you see I'm going to sum
26:20here
26:21and divide by n
26:24and I can get a closed equation in the
26:27limit when n goes to Infinity
26:29I can get a closed equation on set bar
26:31because essentially this term will
26:34average average out so I'm going to
26:36average some over I of exponential of
26:39these terms and it's going to be okay in
26:42that in that case and so all this to say
26:45that I can find
26:47with a little work that set bar of t
26:51is the z bar of zero
26:55the initial condition
26:56exponential of M
26:59plus Sigma squared over 2
27:02times t
27:05and that's when T is much greater than
27:08Tau C
27:10okay
27:14foreign
27:18so what I'm finding in this model is
27:20that the total population size
27:23or the average population size up to a
27:26factor n is growing exponentially at the
27:29rate
27:30that is not exactly the average rate m
27:33is given by M plus a sigma squared over
27:362. so there's a little bit of the
27:38fluctuations that help
27:40the total population to grow a little
27:43faster
27:44okay
27:46so
27:47you know when m is positive it means
27:50that
27:50the population is growing forever
27:53and so superficially it means that
27:56there's no stationary State because it
27:58means that you know the whole thing
28:00diverges to Infinity
28:02but so in order to get a stationary
28:05State for the
28:06population sizes z i I'm going to do
28:10something very natural which is to
28:13rescale
28:15the population of a city by the average
28:18population size
28:21so what I'm going to introduce is this
28:23one is small small caps that I
28:26equal
28:27capital z i divided by z bar
28:32okay
28:35and so intuitively again the logic is
28:39that the population size is you know
28:41diverging with time that's what I've
28:43just shown you here
28:46but if I rescale the population in each
28:50City by this average population size
28:53then maybe and that's what we're going
28:55to show now maybe these rescale
28:58quantities have a well-defined
29:00distribution in the large time limit
29:02okay and that's going to be the
29:05stationary distribution I'm looking for
29:09so
29:10if I
29:12if I um
29:14look for the evolution of zi so dzi DT
29:19this is going to be equal to 1 over Z
29:22Bar D capital Z IDT minus
29:28z i over z bar squared e z bar
29:34e t okay
29:37so here I'm using
29:40the the equation for z i here I'm using
29:45this equation here
29:47and so finally what you get
29:50is an equation for z i d z i DT which is
29:56ETA I
29:58of t
30:01minus Sigma squared over two
30:05set I
30:08Plus j0
30:121 minus that I
30:18and you can you know guess where all
30:20these terms are coming from a to I small
30:23z i it just comes from here
30:25uh 1 minus zi comes from here
30:29and then because of this term here
30:32the the small empty the small M
30:35contribution has disappeared and there's
30:37an extra Sigma squared over two coming
30:39from from here okay
30:44Okay so
30:46now I get something that has a chance of
30:49having a well-defined
30:52stationary distribution for large times
30:54and in order to show that more clearly
30:57what I'm going to do is to introduce a
31:01new variable
31:02from said I I'm going to introduce
31:06something
31:07that you will see helps a lot which is a
31:11log of Zi
31:14UI equal log of z i
31:16and I'm going to derive an equation for
31:19DUI DT and now again take just one
31:25minute to remember that what I told you
31:27is when I have a stratanovic convention
31:29it means that change of variables
31:34are allowed
31:42allowed in the sense that
31:45they are trivial they're the usual uh
31:48rules apply the chain rule applies for
31:51uh differential equations
31:53and in the Ito convention
31:56the change of variables are allowed of
31:59course but you have to be careful in the
32:01sense that there's an extra contribution
32:03that would change the equation
32:06Okay so
32:09just as an illustration d u i DT
32:14this is you know as usual now we can do
32:18as usual is D log zidt so it's 1 over z
32:22i
32:23e z i
32:24e t
32:26and there's nothing else because it's
32:29it's a Stratton of Edge Convention and
32:32so you just have to um
32:36divide by z i this equation
32:39and it gives
32:43so let's not go too far in that
32:45direction
32:48let's use
32:50the room I have here
32:52so dydt
32:57using the fact that it's just one of
32:59that ID said idg
33:01is equal to
33:04ETA I of T minus Sigma squared over 2.
33:10Plus j0
33:13so 1 will become 1 over z i
33:16okay and one over z i is
33:20uh exponential of minus UI so
33:24exponential
33:27of minus UI minus 1.
33:31okay
33:34and now I'm happy because this is the
33:37standard launch line equation
33:45and we know everything about logical
33:47equations
33:48uh so this describes the motion of a
33:52fictitious particle
33:54driven by a certain Force the
33:56deterministic part of this equation is
33:58usually called
34:00the fourth term and then there's a
34:02temperature term of a fluctuating random
34:05term ETA which plays the role of
34:08temperature
34:09so we know that this is describing a
34:12particle in a given potential which is
34:16driven by thermal noise and equilibrium
34:19solution will be the boltzmann wave
34:22so let me be more explicit
34:52so the equation for you I can write as d
34:55u d t
34:57equals minus DV
35:00U
35:02I
35:04plus h i of t
35:08okay
35:10where V of U
35:13with
35:17via View
35:19so it's just the term I need to
35:22introduce to see this as the derivative
35:25of a potential
35:26and it's given by uh
35:30j0 exponential of minus U
35:35Plus
35:38j0 plus Sigma squared over two
35:42times U
35:45okay
35:46clearly if I take the derivative with
35:48respect to U I'll have a j zero plus
35:51Sigma square root of the two with a
35:52minus sign so these are these two terms
35:54and then I have the J zero exponential
35:57of minus U from here
36:00so as I said what does this describe it
36:02describes the thermal motion of a
36:06fictitious particle
36:08of course here we're not speaking about
36:10particles we're speaking about
36:13population sizes rescaled by the average
36:16population size or wealth we scale by
36:18the average wealth but let me think of
36:20this equation as a larger equation for
36:24a particle in a potential
36:27so what this potential looks like
36:30well for a lot for very large U positive
36:34this goes to zero so it's going to grow
36:36linearly with u we don't see what's on
36:40the left
36:43ah okay so you know
36:48let me see what you're not seeing
36:52just before the person
36:55I'm just going to uh
36:59cheat for
37:00then you have to remind me
37:07it's okay
37:09thank you
37:11so d y d t minus DV Dy plus a to I this
37:15is the launch my equation of a particle
37:16in a given potential and what I'm
37:18drawing here is V of U
37:22so asymptotically for a very large
37:25positive view is growing linearly and
37:29for very large negative view then
37:31exponential of minus U becomes
37:34exponentially large for you negative and
37:36so it's going to grow like this so if
37:38I'm drawing
37:39the actual potential
37:42looks like this
37:45okay
37:47and so you know intuitively what's going
37:49to happen is that this thermal particle
37:52here it it just hovers around the
37:56minimum of the potential but sometimes
37:58it's going to be able to go
38:01uphill here and reach reasonably High
38:04values of U but because the potential is
38:07growing extremely fast there is going to
38:10have a hard time going for you negative
38:12okay
38:14but very uh
38:17more and more precisely what we know is
38:20that the stationary distribution the
38:23equilibrium distribution of U
38:26is nothing but the boltzmann wait
38:30so it's a sudden normalization and
38:34exponential of minus V of U
38:38over temperature
38:40and here temperature is Sigma squared
38:44over two so it's exponential of minus V
38:46of U over 2 Sigma squared
38:53so this is the stationary distribution
38:55if you're um
38:57if you're more greedy and if you want to
38:59know
39:00what happens at finite time how fast do
39:03you reach this space redistribution
39:05you can do it
39:07you can do it thanks to
39:10the so-called focal Planck equation so
39:13to each larger equation you can
39:15associate
39:17an equation describing the evolution of
39:21you the probability to find the particle
39:23at
39:25U at time t
39:27and I'm not going to derive this focal
39:31Planck equation but you'll see that also
39:33in the today's and if you look at
39:38um the lecture notes vehicle Polytechnic
39:40lecture notes you'll see a derivation of
39:42this uh like a Planck equation but the
39:45the general shape of the focal Planck
39:47equation is for the for the larger
39:50equation I just wrote
39:51is d by d u
39:56DV by d u
39:58p
40:01this is called the drift term this comes
40:03from
40:05this uh
40:07deterministic Force term plus the
40:11so-called diffusion term
40:12the term that comes from data the launch
40:15web term
40:16Sigma squared over 2 d2p
40:19over d u squared
40:23so let me
40:25move this in
40:32so this is called the focal flank
40:34equation
40:42and you see in principle it allows you
40:45to
40:46uh
40:49solve for the full dynamics of peer view
40:52so if you start by a peer View at time T
40:55equals zero
40:56then you can evolve the probability
40:58through this focal blank equation and
41:02what you'll find is that for very long
41:05time
41:06the probability distribution Settles to
41:10this boltzmann weight
41:12and if you want to do it uh quickly you
41:16can check that
41:18if you inject the shape of the
41:20equilibrium here you find that this term
41:23is zero
41:24so that DP DT is indeed zero at
41:27equilibrium
41:29so the focal flank equation is a very
41:31useful
41:33uh formalism and I'm going to use it
41:36several times later on in these lectures
41:41okay so how far can I go to the right or
41:45maybe I can uh
41:51camera back
42:06okay so I have my peer View
42:10now of course I'm interested in Notting
42:12you but in z
42:20but essentially I'm done
42:25because once I get peer View
42:28I get that P equilibrium of Z
42:32the stationary distribution I'm looking
42:34for the rescaled population sizes of
42:37wealth this is just given by T
42:40equilibrium of u d u d z
42:46with uh
42:47U equal log V
42:52okay
42:54so if I you remember I told you that
42:56what I'm calling P of Z means the
42:59distribution of Z and it's not the same
43:01function as P of uh although as I said
43:05last time this is a little bit an
43:07abusive notation but a very uh
43:11convenient one
43:12okay so if I put everything together
43:16and inject
43:18here via View and make the change of
43:21variables
43:22then what we find is that
43:25the stage redistribution of Z
43:29is the sudden normalization coefficient
43:31that I'm not going to care
43:34writing down of course one can compute
43:37it
43:38exponential of minus
43:412 j0 over
43:44Sigma squared
43:46Z
43:47divided by Z to the 1 plus mu
43:53with mu
43:55equal one plus two J zero over Sigma
44:00squared
44:07okay so what do I get here I get
44:11exponential of minus one over Z
44:14on the top and a parallel
44:17on the bottom
44:19so if I if I plot here
44:22P of Z
44:26the equilibrium of Z
44:30what does this look like well for Z very
44:32small very close to zero
44:35this is exponentially small this is a an
44:37essential Singularity as Z going to zero
44:40so it's very very flat here
44:43then it goes like like this
44:46and then for very large D this becomes
44:49one okay because one over Z goes to zero
44:52and you get a parallel
45:02okay
45:04so that's nice
45:07because this calculation can be done to
45:09the end as we just did
45:12and the conclusion are quite into the
45:14conclusions are quite interesting
45:16uh first of all what you what we get
45:19is that we've generated a parallel
45:22distribution okay
45:25there is a stationary State once I work
45:28with uh we scale variables
45:31no don't forget that I've I've rescaled
45:34Zi
45:35capital z i by the average wealth
45:38so I'm on the correct scale
45:40comparing Apples to Apples if you want
45:43and on that scale I get a stationary
45:46distribution where the probability to
45:48have a very small City sizes or very
45:51poor people is very small then there's a
45:54hump and then there's a very long tail
45:57to the right saying that
46:00there is a appreciable probability to
46:03get very rich people or very large
46:06cities
46:08distributed with a parallel and we have
46:11an explicit
46:12value for the parallel index which is
46:14one plus two j0 over Sigma squared
46:19so this is a mechanism leading to
46:21parallel distribution
46:24note that
46:29a
46:30mu is not Universal
46:40okay so we have a parallel that's
46:42continuously dependent on
46:45uh on the value of the parameters
46:49and okay this is a model that gives you
46:52something that leads to non-universal
46:54parallels
46:57there's nothing critical in this model
46:59there's no critical point
47:01and so we have a power law a skill free
47:04phenomenon without
47:06any
47:08phase transition or critical point
47:10usually when there's a critical point
47:12the value of news Universal and we'll
47:15see examples of that
47:17uh later on but in this case actually mu
47:20is is not Universal so if you remember
47:22for example the title distributions for
47:26wealth is the remarkably stable over
47:30time and of over countries
47:32and so if we believe this model we would
47:35have to understand why the ratio j0 over
47:38Sigma squared is so
47:40uh you know Common to different types of
47:43economies
47:44so okay maybe there's an argument for
47:47that maybe we're not on the right path
47:49actually it's it's still very much an
47:52open problem as we speak
47:54I'll comment that a little more in a
47:56second
47:59the second
48:01remark
48:02is that mu is strictly greater than one
48:06for j0
48:10when j0 is greater than zero
48:15so that's interesting too because
48:18you remember remember that mu greater
48:20than one means that this distribution is
48:22a finite mean
48:25and so in this case
48:27as soon as there's some a little bit of
48:30redistribution as soon as this j0
48:33is non-zero is present it can be as
48:37small as you want prob provided it's
48:39it's positive
48:41then provided there's some kind of
48:44redistribution
48:45the distribution the final distribution
48:47of wealth or or population has a finite
48:51mean which also means that the halfindl
48:54index goes to zero for large n
48:58okay
49:00so no condensation
49:12are you still seeing this
49:16yes
49:21so this is strange right because in the
49:23absence of j0
49:26I showed you last time that this
49:28independent growing cities
49:31are going to condense at one point
49:32there's a Time TC that I call TC Beyond
49:36which the population is condensed in a
49:38few cities or the wealthiest contents in
49:41a few individuals as soon as you add a j
49:43zero term
49:45in the mean field limit
49:47again I'm in the mean field limit then
49:50you you break this condensation
49:52and you get back to a kind of more
49:55democratic type of distribution
49:59and the last remark
50:02I mean two more remarks Maybe
50:07um
50:08Okay C
50:11when j0
50:18I hope you can still see this when j0 is
50:21much less than Sigma squared
50:24so when
50:27the noise the noise in the growth rate
50:30is much bigger
50:32than the exchange rate
50:35by the way dimensionally Sigma squared
50:37is the frequency like j0 so this is a
50:41meaningful comparison
50:44um then
50:46one gets
50:48a zip flow
50:53which corresponds to Mu equal one
50:55and you remember that there's a lot of
50:58phenomena that are described by the zip
51:01floor and this is a mechanism that
51:04naturally leads to a zip flow provided
51:06during the limit of small
51:09redistribution
51:11when this redistribution is is weak then
51:14generically you're going to generate
51:16that model
51:18a zip floor
51:20and so finally if I take you know
51:23reasonable all this magnitude
51:26for um
51:29if I if I really think of this model as
51:32a model for wealth distributions
51:36then
51:38it is not completely ridiculous to think
51:42of this model as a kind of wealth tax
51:44model
51:45because you see here the the
51:49proportionally to your wealth you're
51:51going to give away a part of your wealth
51:54and it's going to be redistributed
51:56uniformly in the in the population so in
52:00spirit this is this is a kind of wealth
52:03tag
52:04and uh so if I take
52:07j0 equals one percent a year
52:15so
52:16you have a web tax that that is one
52:20percent of your wealth every year that
52:22you give away uh to this to to the tax
52:25man to the state
52:27and sigma Sigma is the
52:29the variance of your Investments so you
52:33know sometimes you make more sometimes
52:35you make less
52:36and so if you take stigma on the order
52:38of twenty percent per year
52:41in fact the square root of year
52:44so some years you do
52:47say five percent other years you do 25
52:50and so on
52:52then you get that mu is
52:56around three half
52:58which is the title value
53:01so there's a lot of problems with this
53:04model if you try to compare it to a real
53:06wealth data so it's clearly not the the
53:10final model to describe what's going on
53:13to understand web distribution but if
53:16you're adamant to use it as a model for
53:18web distribution you see the kind of
53:20numbers that you get if you um
53:23if you if you make it go at trying to be
53:27realistic and so these are not
53:28completely crazy numbers maybe Sigma 20
53:30per year is is a little too high
53:34um but anyway that's that in order to
53:36give you that's just to give you a
53:38feeling of the type of numbers that you
53:40would would get from this model
53:44okay
53:50so that that's the the story of the
53:52fully connected model
53:54and again the story of the fully
53:56connected model is that
53:59there is no condensation anymore
54:28so no redistribution
54:32concentration transition at the finite
54:34time
54:37any small redistribution in the mean
54:40field model
54:41leads to
54:48no condensation
54:57so what I want to tell you a little bit
54:59about is what happens in other cases
55:05before I do that
55:09okay Valentina
55:11great so indeed the up long
55:14will be
55:16derived in the next day
55:20so
55:21um
55:26let me give you a few words on the other
55:30cases
55:32So Random graph
55:42so for the general case of a random
55:46graph you know must be specified because
55:48random what does that mean and we'll see
55:51examples of random graphs later on in
55:54the lecture so for example the simplest
55:57random graph is called the other srini
55:59graph where you draw at random the
56:02presence of a link or the absence of a
56:04link
56:05there's also another type of graph which
56:09are tree light graphs
56:11so for example where each node
56:15is connected
56:17randomly
56:19to three other nodes
56:22okay
56:24so this is a little zoom on what's
56:28called regular random graph
56:33foreign
56:43but you know you might have loops in
56:45such a graph so for example this guy
56:47here might be connected uh
56:50to these two ones and then
56:55okay
56:57so
57:00so this is the
57:04a random regular graph in the sense that
57:06sites are connected randomly to other
57:09sites but each side has a fixed
57:12connectivity
57:14okay
57:17and so the connectivity is is sometimes
57:19called C
57:21so here in my example C equals three
57:26and the fact that it's a locally
57:28tree line graph allows you to make exact
57:32calculations
57:33and what you find in this case
57:36is that
57:38when
57:40Sigma squared over J is 0
57:44is less than a sudden critical value
57:46that I'm going to call a critical
57:51then
57:52there is no condensation
57:55a single is zero
57:58and if
58:00Sigma squared over j0
58:03is greater than a sudden a
58:06that depends on C
58:09then h
58:11is positive
58:15so you see now we have a new type of uh
58:19result because what I got is
58:22up to now either always condensation now
58:27it's the case of independent growth or
58:29never a condensation as the case of
58:31reconnected graph
58:33I can have an intermediate situation
58:35where you have sometimes
58:39no condensation
58:40and sometimes condensation
58:43and what matters is this ratio Sigma
58:46squared over j0 the same ratio as this
58:48one
58:50and so what you intuitively see is that
58:54if j0 is large enough
58:58you avoid condensation if j0 is small
59:02enough
59:03you are condensed
59:06so this is the case where there's a true
59:07phase transition in this model
59:10foreign so if I plot
59:14again just to keep pedantic because this
59:17is just
59:18reformulating what I just said so if you
59:21plot for example this function as j0
59:24the half anal index
59:27what you're going to get is that for
59:29large enough j0s if there's enough
59:31redistribution you have signal index
59:34goes to zero
59:36this is going to look like this
59:40and then
59:41when j0 becomes too small
59:46it's going to increase and tends to 1
59:50when j0 goes to 0.
59:57and this is for a fixed value of Sigma
1:00:00squared
1:00:02so for sixth amplitude of the noise
1:00:06you need to redistribute
1:00:09forcefully if you want to avoid extreme
1:00:12condensation okay
1:00:16but it's possible
1:00:18so if you want in order to understand
1:00:22the previous limit
1:00:24in the case where
1:00:29j0 is zero that is if there's no
1:00:33redistribution this is the independent
1:00:36growing
1:00:37um population model this is the model
1:00:40that I talked about last time and we
1:00:43know that at a long time the health
1:00:44center will index tends to one so we
1:00:46recover the previous result
1:00:48and if
1:00:51if you're in the Midfield model that is
1:00:53if
1:01:01sorry Sigma squared over AC
1:01:08so in the mean field model AC goes to
1:01:10Infinity if you want and so this rings
1:01:13to zero and so there's no
1:01:16content space anymore
1:01:19so that's how you recover the mean seal
1:01:22limit from this model is that in the
1:01:25limit C goes to Infinity
1:01:29AC also goes to Infinity
1:01:34so the condensation disappears
1:01:37in this limit okay
1:01:53so this is an example of a random graph
1:01:56but this is a generic conclusion
1:01:59yes
1:02:03um so this is a generic conclusion for
1:02:06these random graphs more complicated
1:02:08there's
1:02:10in general a phase transition between a
1:02:12condensed and a non-condense phase yes
1:02:14please
1:02:16so sure we're looking at the limits and
1:02:18tends to Infinity what happens in the
1:02:21case of
1:02:22donate n
1:02:24ah well you know the the fact that
1:02:27there's a phase transition only exists
1:02:29when n goes to Infinity if
1:02:31um if n is finite
1:02:34all the statements I made are only
1:02:37approximate because for example you know
1:02:40that
1:02:41if you're in a non-continent space of
1:02:44finite n
1:02:50the non-continent phase means that the
1:02:52halfindl index is of order one over n
1:02:55so it's not zero
1:02:57so there's there's nothing that you can
1:03:00stay you know mathematically rigorously
1:03:02you have a crossover between a regime
1:03:06where age is of all the one over n and
1:03:08the regime where it's of all the one so
1:03:11if you want if I plot again
1:03:13my phase diagram
1:03:15for finite n so if I do have signal
1:03:19index as a function of j0 for finite n
1:03:21then instead of having the nice
1:03:26Place true phase transition regime where
1:03:29I have this line okay with a a true
1:03:33point
1:03:34separating a region where it's zero from
1:03:37a regime it's not zero then for finite N
1:03:40I will have something like this right
1:03:46so it's going to be a crossover
1:03:54so you can ask interesting questions you
1:03:56can ask okay so how does this point here
1:03:59how does it go to zero as a function of
1:04:02n if I'm right at the transition with
1:04:05that finite n
1:04:06then this will go to zero with a strange
1:04:10power of n
1:04:12whereas these points here will go to
1:04:14zero as one over n
1:04:16and these points won't go to zero so
1:04:18there are interesting questions to ask
1:04:20but you know in terms of this the
1:04:22existence of a true phase transition
1:04:24it's only a well-defined for n going
1:04:26into the infinity
1:04:29of course it's enough for many purposes
1:04:31because you know what the system is
1:04:33going to behave like
1:04:36okay
1:04:41foreign
1:04:47[Music]
1:04:50regular
1:04:55euclidean
1:04:58lattices
1:05:05so D equal one
1:05:08this is just a chain
1:05:15okay
1:05:17equal to
1:05:19it would be what I've drawn before
1:05:21something like this
1:05:25you know maybe more complicated cell
1:05:27structures but essentially something
1:05:29like this and then equal three you can
1:05:32think of as a as a cubic lapis
1:05:41and so on
1:05:43okay
1:05:45and you can you can also if you want
1:05:48formulate this model in higher
1:05:50Dimensions although it may not have such
1:05:52a clear physical motivation but it can
1:05:55be useful from from a theoretical point
1:05:59of view to think of higher dimensions
1:06:01and it's also useful in non-physical
1:06:04context to uh
1:06:07think of these extended problems
1:06:11so now what happens
1:06:14so it's uh that's where you know if you
1:06:17if you were in the class it would be
1:06:19easier but um
1:06:21uh you know what's your guess what's
1:06:24going to happen
1:06:26so if does anyone have a guess for the
1:06:28equal one
1:06:32I agree that this is a complicated
1:06:34exercise
1:06:36um
1:06:39being remote but uh
1:06:41sorry
1:06:45can you hear me
1:06:46for example one is like having a tree
1:06:50with a very low connectivity
1:06:52yes
1:06:54exactly so this will obtain the same as
1:06:57before
1:06:59right well in this case
1:07:02it is the case that the AC goes to zero
1:07:06when you only have two neighbors like
1:07:09this and you're always in a condensed
1:07:11space
1:07:18so you write this is
1:07:21a random regular graph if you want to
1:07:23see it that way but the value of AC
1:07:25goes to zero so if you remember the
1:07:28graph that I uh removed that I erased it
1:07:31means that the transition goes all the
1:07:33way to J Infinity
1:07:35and so for any finite J
1:07:38you're always going to con to be
1:07:39condemned so that's quite interesting it
1:07:41means that it's not only the strength of
1:07:44J that matters
1:07:46there's also something about the
1:07:48topology of the graph that matters and
1:07:51in the case where you have a
1:07:54one-dimensional chain you'll never get
1:07:56away from condensation okay at long
1:07:59times
1:08:01you are actually always going to be in
1:08:04in fact it's extreme condensation in
1:08:07that case
1:08:08so I should call this extreme
1:08:13X condensed extreme condensed content in
1:08:17the sense that h
1:08:19is equal to one
1:08:22okay so at long time whatever the value
1:08:24of j0
1:08:26you will never get away from uh a few
1:08:30people or a few cities
1:08:32condensing the whole population
1:08:35it's not completely intuitive that uh
1:08:39you know there's not enough mixing if
1:08:41you want you you think of that as mixing
1:08:43it's stirring uh the brew and if j0 is
1:08:48very high but you have a one-dimensional
1:08:50chain well if you wait long enough
1:08:53doesn't matter you're still going to get
1:08:55everybody at the same place okay
1:08:58so so there's a little bit of a surprise
1:09:00here but it's an interesting surprise
1:09:02and actually as I'm going to mention the
1:09:05one-dimensional case has become one one
1:09:08of the most important models in
1:09:10theoretical physics in the last few
1:09:13decades so I'm going to go back to that
1:09:15in a second
1:09:17so D equal to
1:09:19well again you get extreme condensation
1:09:29so it's a little more subtle in the
1:09:31sense that
1:09:33at finite time you need to wait very
1:09:35very long time in order to get
1:09:38you know H equal one but asymptotically
1:09:41as a mathematical statement
1:09:43G equal to is again
1:09:45a case where you get extreme
1:09:47condensation so there again
1:09:49mixing is not strong enough if you want
1:09:54and the reason if you you know
1:09:55intuitively is that you you always go
1:09:58back to the same place and equal to if
1:10:01you make random walks on on a
1:10:03two-dimensional lattice
1:10:04or on a one-dimensional lattice you
1:10:07always go back to your initial starting
1:10:09site
1:10:10and the fact that you always revisit the
1:10:13same sites
1:10:15presents the model from uh you know
1:10:19avoiding condensation
1:10:23but for the equal three things change
1:10:25and equal three
1:10:27there exists a critical value of J JC
1:10:33for a given signal squared
1:10:35such that well J greater than JC
1:10:40j0
1:10:42then you're not condensed
1:10:47and if j0 is less than JC
1:10:51H is positive
1:10:55okay
1:10:56so
1:10:58you see that the phenomenology is quite
1:11:00interesting and depending on not only
1:11:03the strength of
1:11:05redistribution effects but also the
1:11:07topology of the lapis or the graph you
1:11:10can get various situations
1:11:13okay
1:11:16so this is a a description you know
1:11:19a very uh rough description of what's
1:11:22going on but let me show you
1:11:24why this model
1:11:27in euclidean space is actually related
1:11:31to
1:11:33um a whole family of
1:11:36uh of models that have attracted amazing
1:11:39attention in the last uh 20 to 30 years
1:11:42in the
1:11:43in the mathematical and theoretical
1:11:47physics community
1:12:11so that's what I'm going to discuss here
1:12:13now
1:12:14uh link
1:12:18with other problems
1:12:29and that's going to be a segue into my
1:12:33part 4 and of course you don't see part
1:12:364 anymore
1:12:45which is optimization and
1:12:48um Hamilton Jacoby Bellman methods okay
1:12:51so where does that come from so I talked
1:12:53about
1:12:55redistribution and growth
1:12:58and I'm going to end up
1:13:01speaking about optimization
1:13:04and interestingly is the same formatism
1:13:07is the same equations that can be
1:13:09interpreted in different ways
1:13:11that I'm going to
1:13:13uh that's about now
1:13:18so let's let's look again at
1:13:21the equation I wrote
1:13:24is that the T the zidt there was a
1:13:29random term M plus a to I
1:13:33said I
1:13:34Plus
1:13:36sum over J of j i j h a i
1:13:40a j
1:13:42minus sum over J of j i j
1:13:47z i okay
1:13:50now let me put this model on
1:13:55irregular lattice
1:13:58the ones the types of lattices I've just
1:14:01talked about with each link here
1:14:06so this is ing
1:14:08inj must be nearest Neighbors and if
1:14:11they aren't nearest neighbors then j i
1:14:14to J
1:14:16is equal to j0
1:14:20okay
1:14:21if
1:14:24I and J are nearest Neighbors
1:14:29okay
1:14:35so let's look at what it looks like this
1:14:38time in this case well
1:14:41um
1:14:43you have
1:14:45two indices
1:14:47to uh to Define Where You Are
1:14:51in in a two-dimensional axis
1:14:54so I'm going to introduce a notation
1:14:56which
1:14:58is that
1:15:00the position
1:15:02on the lattice is a vector each
1:15:05component of this Vector taking only
1:15:07discrete values okay
1:15:11so this is a 2d Vector with only
1:15:13discrete values telling you where you
1:15:15are on the lap is
1:15:16and what you get if you expand this sum
1:15:22here
1:15:23is for a given
1:15:27X it is a given I I now becomes a
1:15:30two-dimensional Vector if you want
1:15:33who I becomes X
1:15:37then I have that the the second term
1:15:41is minus 4 Z of x
1:15:47okay
1:15:49because there are four neighbors so
1:15:51minus four J zero
1:15:54set of X this is this term here
1:15:58and this sum
1:16:01is going to give me Z of x
1:16:05plus one in the uh
1:16:10in the
1:16:12yeah okay
1:16:17so this is the X Direction and the Y
1:16:19Direction
1:16:21but I'm okay maybe I should call this R
1:16:24and not X
1:16:27right r
1:16:36okay sorry for this change of notation
1:16:38so
1:16:43I need some space
1:16:46so this time here is going to be j0
1:16:49Z of
1:16:52X Plus 1X
1:16:55plus Z of x
1:16:58R of R plus 1X instead of R minus 1X
1:17:06plus Z of r
1:17:09plus 1 y
1:17:12plus Z of r
1:17:14minus
1:17:161 y
1:17:17okay
1:17:24and of course one in the direction of X
1:17:27is this vector
1:17:301X and one in the direction of Y is this
1:17:33vector
1:17:37so now imagine that Z is is a smooth
1:17:42function is varying smoothly
1:17:46over space over R then I can Fourier
1:17:50expand as sorry not three uh Taylor
1:17:53expand
1:17:54this expression here and I guess that
1:17:58you won't be surprised to think to see
1:18:00that the first term is four times Z of R
1:18:04so it goes away with that one the second
1:18:06term vanishes
1:18:09and what you get in the end is a
1:18:13laplacian term so this equation in the
1:18:16Continuum limit
1:18:20okay so I was just guiding you to see
1:18:22how it works
1:18:25but
1:18:27I'm sure you've done that somewhere in
1:18:29your curriculum before
1:18:31but if you do this Taylor expansion
1:18:40then this discrete
1:18:42equation for redistribution becomes a
1:18:45continuous equation which is DZ of r
1:18:49and he GT
1:18:52equals
1:18:54M plus ETA of R and T
1:18:59so now you have a noise term that
1:19:01depends on where you are in space
1:19:04times Z of r
1:19:08and T
1:19:10and then the term that I labored on
1:19:13trying to convince you that you had a
1:19:15laplacian coming from the Taylor
1:19:17expansion is going to be plus j0
1:19:23um if you want to be
1:19:26okay with uh physical dimensions I
1:19:29should also introduce
1:19:34the lattice spacing a so a is the actual
1:19:37physical distance between two sides and
1:19:40what I get is j0
1:19:42a squared
1:19:44times the laplacian
1:19:47of said
1:20:03because
1:20:05of R and T
1:20:09okay
1:20:17so let's reflect on this we have a term
1:20:21that is just a diffusion term this is if
1:20:25I didn't have growth I would have pure a
1:20:29pure diffusion equation
1:20:33um
1:20:33you know heat also called heat equation
1:20:37and now on top of the heat equation I'm
1:20:42throwing in a term that describes random
1:20:45growth
1:20:46so it's very physical what we get we get
1:20:50you know population on around site R and
1:20:52T that grows randomly and diffuses in
1:20:56space and this is exactly what we try to
1:20:59capture with this equation so it's not a
1:21:02surprise it's just a continuous limit of
1:21:05this equation
1:21:06of the first equation is the second
1:21:09equation which I'm going to give a name
1:21:10to and call this equation a because
1:21:13we're going to see it again
1:21:15in another context
1:21:18so this equation is is extremely
1:21:20important
1:21:21and it has various names in the
1:21:23literature
1:21:25uh so for example it's called the
1:21:27stochastic heat equation
1:21:38foreign
1:21:57equation for the reason I just said
1:22:00there's a heat part and the stochastic
1:22:01part
1:22:03is also called the parabolic on the
1:22:06Anderson equation
1:22:09foreign
1:22:15is that this looks like a Schrodinger
1:22:19equation a little bit with a random
1:22:21potential
1:22:23so of course in the case of quantum
1:22:25mechanics you would have a an I here
1:22:28okay
1:22:30and Z would be
1:22:33a complex object the
1:22:36the wave function and then this would be
1:22:40the Schrodinger equations for a particle
1:22:43in a random potential
1:22:46and perhaps some of you know about this
1:22:49problem this is called the localization
1:22:51problem that was invented by Phil
1:22:54Anderson in 1958 and is a hugely
1:22:58important part of uh solid state physics
1:23:01and uh describes what's called Anderson
1:23:04insulators
1:23:06so in surprisingly the the story the
1:23:09physical story is that the particle in
1:23:12the random potential can be completely
1:23:15localized
1:23:16and never go away to Infinity so it's it
1:23:21cannot carry electricity for example
1:23:23such a such an object so you can have
1:23:26localization just because of of
1:23:28randomness and in a sense although
1:23:31technically it's different in a sense
1:23:33this is similar to the condensation
1:23:36problems that I've talked to you about
1:23:38instead of finding the particle anywhere
1:23:41in space
1:23:42the hassanal index of the wave function
1:23:45which in physics is called the
1:23:46participation Ratio or the inverse
1:23:48oxidation ratio
1:23:49is non-zero in in this in certain cases
1:23:54because the particle cannot escape to
1:23:57infinity and so uh it's uh it's very
1:24:00related so if you want this equation is
1:24:03the Schrodinger equation corresponding
1:24:05to Anderson's problem without an i and
1:24:09of course this changes a lot but uh
1:24:13it explains the the name parabolic
1:24:16Anderson equation
1:24:18and as I'm going to uh allude to uh
1:24:22later on it's also related to What's
1:24:25called the kpz equation
1:24:32after
1:24:33three theoretical physicists called our
1:24:36parity
1:24:39and sang
1:24:44that as I said has become uh you know
1:24:47one of the
1:24:48totem equations in in theoretical
1:24:51physics and in particular if you're in
1:24:54one dimension
1:24:56so if instead of two Dimensions like my
1:24:59drawing here you're on a chain
1:25:01then the ppz equation is exactly solved
1:25:05and I'll code some of the results later
1:25:09on so you can have an exact solution for
1:25:12this
1:25:13stochastic heat equation
1:25:16exactly in a sense that has to be
1:25:18defined but everything is known about
1:25:19this problem in uh one dimension
1:25:24okay
1:25:26so now I'm going to move to
1:25:30uh part 4 optimization and uh Hamilton
1:25:34Jacob email man and talk about
1:25:38something that superficially seems
1:25:40completely unrelated to what I've talked
1:25:42to what I've just talked about
1:25:45and this is a little bit the magic of
1:25:48of mathematics if you want or
1:25:50theoretical physics as you wish to be
1:25:53able to map
1:25:55problems that superficially have nothing
1:25:57to do with one another
1:25:58and get inspired from the solutions of
1:26:01one to say things about the other
1:26:06and in passing I hope that you will
1:26:09learn something that
1:26:12is rarely discussed in in physics
1:26:15curricula
1:26:17whereas it's uh hugely important in
1:26:19other fields in particular in economics
1:26:22or in certain parts of engineering
1:26:28so what I'm going to talk about now
1:26:31is
1:26:33um
1:26:35part four let me switch back to
1:26:43right
1:26:46so of course I won't be able to finish
1:26:48everything today but uh I'm going to
1:26:51speak about optimization
1:26:58and
1:26:59Hamilton
1:27:03Jacoby
1:27:06Bellman
1:27:15Okay so
1:27:17let me introduce you to
1:27:22a kind of fun game a treasury hunt
1:27:41okay so what I'm drawing here is time in
1:27:45this direction
1:27:46and X in that direction
1:27:49okay
1:27:50and uh so you're on a bike think of
1:27:53yourself on a bike
1:27:55and you're on the bike but along the
1:27:57road a one-dimensional road so there's a
1:28:00one-dimensional coordinate that
1:28:02describes your position
1:28:05and what you have to do
1:28:08is to try to get as many bounties as
1:28:11possible
1:28:13so I'm going to assume that
1:28:15in this
1:28:17FaceTime representation
1:28:20they are
1:28:23what I'm going to call a bounty so a
1:28:25reward if you want
1:28:27that are randomly scattered
1:28:30so they appear
1:28:32at random or maybe they're deterministic
1:28:34it doesn't matter
1:28:36but there are objects that you should
1:28:38collect in order to improve your score
1:28:43and and you have to do that while riding
1:28:46your bike
1:28:47so your trajectory is going to be a path
1:28:52so this is your trajectory
1:29:02okay
1:29:05and so you're you're pedaling in order
1:29:07to get as many bounties as possible so
1:29:10we have as as high a score as possible
1:29:13okay
1:29:14the problem is that you also pay some
1:29:18efforts in order to bike fast
1:29:21so
1:29:23you know collecting bounties will come
1:29:25at a cost and the cost is how
1:29:28fast you have to buy so there will be an
1:29:31optimization problem here which is to
1:29:35get as many bounties as possible
1:29:37while not paying too much in uh kinetic
1:29:41energy if you want
1:29:42so that's that's what the game is going
1:29:44to be about
1:29:45and now let me frame it in a more
1:29:48mathematical
1:29:49way okay but so if you
1:29:52uh or at least get the general idea at
1:29:56this point it's good it's a game you
1:29:58have to collect bounties but you won't
1:30:01be able to you know move fast enough to
1:30:05get all the possible bounties on the
1:30:07space-time trajectory okay
1:30:09so how does it Translate
1:30:13so first let me describe the trajectory
1:30:20so the trajectory is
1:30:23given by your speed GX DT
1:30:27okay
1:30:29and this DX DT is going to be given by
1:30:32the sum of two terms one that you
1:30:35control and the other one that
1:30:37unfortunately
1:30:40nature or the person who has invented
1:30:43that game imposes on you so what I'm
1:30:46saying here is that your speed is given
1:30:49by
1:30:51the sudden velocity that you choose
1:30:56so this is called sometimes the policy
1:31:00or a control
1:31:04so that's what you control
1:31:06okay
1:31:08but unfortunately as I said there's
1:31:12a problem which is in the sense of a uh
1:31:16a random noise
1:31:25so as I said it's not as easy as my
1:31:27initial description seems to be because
1:31:29on top of this optimization there's
1:31:33uh someone wins for example the wind
1:31:36that blows and that speeds you up or
1:31:40slows you down in a way that you don't
1:31:42control
1:31:45not
1:31:47controlled
1:31:52so this is wind these are obstacles that
1:31:57are randomly scattered and so you don't
1:32:00see them and at the last minute you have
1:32:03an obstacle that speeds you up or slows
1:32:06you down okay
1:32:08so that's the trajectory
1:32:10as a description of the trajectory
1:32:16is that you know the
1:32:19the line X of T that I've drawn here is
1:32:22just integrating this equation of motion
1:32:25but the problem is that I mean the first
1:32:29problem is that there's a noise term
1:32:31that prevents you from fully controlling
1:32:33your actual velocity okay
1:32:38so what we're going to assume here for
1:32:40PSI
1:32:41is an etonoid
1:32:49and I'm going to assume that PSI of t
1:32:54up
1:33:10I'm going to assume that 5T
1:33:14PSI of T Prime
1:33:18is equal to 2 j0 Delta of T minus D
1:33:23Prime
1:33:25are you still seeing this
1:33:32hello
1:33:33yes and yes okay you still seeing this
1:33:36okay
1:33:37fine
1:33:39so I I've introduced JZ over here for a
1:33:43reason of course because it's going to
1:33:45end up being the same j0 as that one but
1:33:47for the moment I'm introducing j0 as the
1:33:51strength of this wind that bothers you
1:33:55and here I'm assuming to start from the
1:33:58start that this is a Delta correlated
1:34:00noise
1:34:01and I'm thinking of this noise in a
1:34:03Neato sense
1:34:04and if you remember what I uh told you
1:34:07last time the Ito convention comes when
1:34:11you think of time as discrete
1:34:13and so you're at T here and the noise 8
1:34:17x i
1:34:19materializes after T and so you cannot
1:34:23anticipate you cannot predict anything
1:34:25about PSI you're always surprised by the
1:34:28next sign
1:34:30so if you want to think of this that way
1:34:32Ito
1:34:34is
1:34:36surprise
1:34:38full surprise
1:34:43full surprise
1:34:47there's nothing from the past that can
1:34:49help you anticipating next time step
1:34:52so that's in these cases you should use
1:34:55digital convention and that's what I
1:34:57want to model here I want to model you
1:35:00on a bike and things completely
1:35:02unexpected that happen
1:35:05okay
1:35:06this also simplifies uh the discussion
1:35:10quite a bit
1:35:13good
1:35:14so now what's your objective function
1:35:38foreign
1:35:42you want to accumulate as many bounties
1:35:45as possible as many Rewards
1:35:49but you want also to spare your
1:35:53your energy so you you don't want to
1:35:55overspend in kinetic energy
1:35:58so I'm going to introduce
1:36:00the game
1:36:01Carly G
1:36:03so this is again
1:36:08and this will have two contributions
1:36:11one is
1:36:14the amount of bounties that you've been
1:36:16able to connect collect
1:36:23so here I'm assuming that the the game
1:36:26lasts for the Sun
1:36:27time capital T
1:36:30capital t is the end of the of the game
1:36:38end game
1:36:42and you see what this tells you is that
1:36:45you're going to gather bounties along
1:36:48your trajectory along X of t
1:36:51so the Bounty that happens to be on your
1:36:54path at time t
1:36:56you're going to collect it okay so this
1:36:59is the total amount of bounties that
1:37:01you've been able to collect and of
1:37:04course you won't like you'd like this to
1:37:06be as large as possible
1:37:08but as I said the second problem on top
1:37:11of the noise is that your uh
1:37:14you pay something if you want to buy
1:37:16carb and what you pay we're going to
1:37:20assume that it's proportional to kinetic
1:37:22energy
1:37:23so integral VT
1:37:26V squared
1:37:28of X of T and T
1:37:32okay
1:37:35note that it's the part that you control
1:37:38that makes you pay something at least
1:37:41the noise is free for you I mean you
1:37:43don't have to pay
1:37:44GX DT squared you have to pay V Square
1:37:47that's what you decide to do
1:37:49and then if the wind helps you or if the
1:37:52wind is against you it doesn't count in
1:37:55your gain function okay
1:37:57so this is a typical optimization
1:38:00problem
1:38:01a problem that in physics is called uh
1:38:05frustrated because there are two
1:38:07opposing terms one is that you would
1:38:10like to collect bounties that maybe are
1:38:14very very far
1:38:16and so you won't have time to collect
1:38:18them because you would have to bike
1:38:19extremely quickly
1:38:22so there's this gain term and this lost
1:38:26term that kind of conflict that are
1:38:29conflicting and oppose each other okay
1:38:35so
1:38:37the aim of the game
1:38:39is to determine the optimal policy
1:38:44so the solution of this problem if you
1:38:47want
1:38:52the solution to my treasury hunt problem
1:39:02the solution that we are looking for is
1:39:04to find
1:39:06foreign
1:39:08policy
1:39:19the optimal control so what I can call V
1:39:22Star of X and P
1:39:25such that
1:39:28G is maximized
1:39:34okay
1:39:42and of course
1:39:44because there is noise in the trajectory
1:39:48because of
1:39:49of PSI here
1:39:51G is also kind of random so what we're
1:39:56going to look for is not the
1:39:58maximization of G because G contains
1:40:00Randomness but the maximization of the
1:40:04average G
1:40:05average of x i
1:40:08so you don't know what PSI is so you
1:40:10can't maximize something that you don't
1:40:12know so the only thing that you can
1:40:14maximize is something that is averaged
1:40:17over the realization of the noise that
1:40:19you don't know so that's what we're
1:40:20going to try to maximize is the average
1:40:23gain average over the realization of the
1:40:26noise sign okay
1:40:28so that's what we're going to do next
1:40:30time
1:40:31because I won't have time to
1:40:34stop this now
1:40:36but before letting you go for 15 minutes
1:40:42I want to tell you what is this treasury
1:40:46Hunt game in different contexts
1:40:49so for example there's a very important
1:40:52problem in physics which is called uh
1:40:56pinned
1:40:57pinning
1:41:00of
1:41:01one-dimensional objects
1:41:08so in physics as you know there are
1:41:10points like objects like atoms but there
1:41:14are also uh extended objects like
1:41:18polymers or dislocations
1:41:21or Vortex tubes in superconductors
1:41:25there are also two-dimensional objects
1:41:27like membranes and so on
1:41:29and often these one-dimensional objects
1:41:33so think of it as the polymer for
1:41:36example
1:41:37it interacts with uh
1:41:40the environment
1:41:42and often that there are what's called
1:41:46pinning sites
1:41:49and the object the polymer tries to
1:41:52minimize its energy and the energy of a
1:41:56polymer is made of the interaction
1:41:59energy with the impurities
1:42:01so this is pinning by impurities
1:42:12and this V squared in the language of
1:42:15polymer V squared measure how costly it
1:42:20is to make bends like this okay
1:42:23so you're stretching the polymer
1:42:26and by stretching the polymer you pay
1:42:28some elastic energy
1:42:37so this problem falls in the general
1:42:41framework of randomly pinned elastic
1:42:45object
1:42:48which has led to very interesting
1:42:50results that as you're going to see are
1:42:52related to things that I talked about
1:42:56um in particular you remember
1:42:58maybe what I told you in the very first
1:43:01lecture I told you about Buck has a
1:43:03noise
1:43:04well what is back has a noise
1:43:06it is the fact that in magnets you have
1:43:10domain walls you have domains where
1:43:13spins point up and the Maze where spins
1:43:15Point down and the separation between
1:43:17these domains are called domain walls
1:43:19and you can think of these domain walls
1:43:21as kind of membranes
1:43:23and these membranes are pinned by the
1:43:26impurities in the material
1:43:28and so if the object is pinned when you
1:43:31try to make it move
1:43:32you have to you know exceed some
1:43:34threshold and once you exceed the
1:43:37threshold the thing moves and in the
1:43:39case of magnets it creates a noise which
1:43:41is called The Buck has annoy
1:43:44so the question is whether impurities
1:43:47will be strong enough to pin the object
1:43:50or whether the object is going to freely
1:43:53move
1:43:54not even if there are pinning size that
1:43:57it's free from from pinning and the
1:44:00object is is is actually free to move
1:44:04and what you'll see is that
1:44:07pinning but I'm going to explain next
1:44:10week is that pinning
1:44:12so the fact that an elastic object is
1:44:15not free to move
1:44:17is the analog
1:44:19of
1:44:21concentration
1:44:23in the problems of
1:44:25uh random growth with redistribution
1:44:30so this is I think a very spectacular
1:44:32analogy
1:44:33to think that you know we spoke about
1:44:37concentration of wealth and actually the
1:44:40question of whether wealth is
1:44:42concentrated or not in a society is
1:44:46mathematically related to the problem of
1:44:49of knowing whether a polymer or a
1:44:52dislocation is pinned by impurity
1:44:55so more about that next week
1:44:59so I'm about some time
1:45:01maybe you have questions or
1:45:05comments or
1:45:08so if you want uh some those of you want
1:45:11to make a break before Valentina starts
1:45:13at 11 right Valentina
1:45:18oh she must be on her way
1:45:19I guess it's 11. yes and I'm coming
1:45:21upstairs okay okay so 11 Valentina and
1:45:25so if um in the meantime you want to
1:45:28chat I'm I'm totally free to do it so
1:45:31don't hesitate to also contact Valentina
1:45:34or or myself by email if you're
1:45:39if you have questions or if you want to
1:45:41discuss because the problem of course is
1:45:43that I usually come at 8 30 in the
1:45:46morning when things are normal and so
1:45:50people can get a grab of me and and can
1:45:52have a chat or a chat now
1:45:54but since we don't see each other this
1:45:57is not possible so I think email or
1:46:00or chatting now if you want is an option
1:46:06yes sorry
1:46:09uh I I'm not sure if I have understood
1:46:13what is the equivalent of the impurities
1:46:15in this specific problem
1:46:20Okay so
1:46:25B of x and t is the bounty in my uh
1:46:28treasure hunt
1:46:30and it's the potential energy
1:46:33the pinning energy in the case of the
1:46:37direction of the erected polymer or the
1:46:40vortex so if you want
1:46:42every time the trajectory of this
1:46:45physical object now which is not a
1:46:48trajectory uh you know an abstract
1:46:51trajectory this is a real physical
1:46:52object every time it goes nearby an
1:46:56impurity site it gets some actual
1:46:59physical energy which is D of x and t so
1:47:02this is the energy G if you want is a
1:47:05hamiltonian now
1:47:06so there's two parts in the energy one
1:47:09part is pinning by impurity so you want
1:47:11to be near
1:47:13uh impurities that lower your your
1:47:16energy that attract you
1:47:18but there's a penalty which is the
1:47:21elastic energy of the polymer of the
1:47:23object that doesn't like to be uh
1:47:26deformed if you want it would like to go
1:47:28straight
1:47:30okay so if you want what I'm saying here
1:47:33and maybe that's I should have said that
1:47:35more explicitly is that in my treasury
1:47:37hand problem T is time
1:47:40whereas here
1:47:42T is another spatial Direction so maybe
1:47:46I'm sorry that was really missing from
1:47:48my explanation in the in the pinning
1:47:51problem there's no time it's a static
1:47:54problem and what I'm calling T is
1:47:58is another spatial Direction
1:48:01and this is X but maybe it would be less
1:48:04confusing to to call formally this why
1:48:08and then I would get you know a pinning
1:48:13energy that would be
1:48:16X of Y and Y
1:48:19and d y
1:48:21X of Y
1:48:24and Y
1:48:26okay so B of X and Y and Y gives you the
1:48:30energy of this guy the energy of this
1:48:32guy the energy of this guy
1:48:34the attraction energy of these guys and
1:48:37this is really the the the the the the
1:48:41um price you pay in terms of energy to
1:48:45make your object
1:48:47not a straight line but rather uh
1:48:51bends all over the place is that more
1:48:54clear clearer
1:48:56yes totally thank you okay
1:48:59yes I was rushing a little bit at the
1:49:01end and I sort of
1:49:02be more careful with notations I have
1:49:06other question just about this
1:49:08so in the case uh where you can see the
1:49:12domain walls in a magnet with impurities
1:49:16um what is then the cost you need
1:49:18because I guess the domain walls don't
1:49:20have an elasticity that maybe it's the
1:49:22it's the length yeah
1:49:26right so so that's why I said that this
1:49:28problem is equivalent to pinning of
1:49:30one-dimensional objects two-dimensional
1:49:32objects
1:49:34you know membranes or the main walls
1:49:36they're more complicated because
1:49:39y here becomes a two-dimensional object
1:49:42okay
1:49:43a two-dimensional Vector so you should
1:49:46think of that as as now
1:49:49uh a full you know surface that is
1:49:53bending up and down
1:49:55and so in this case you would have a d2y
1:49:58two coordinates
1:50:01here
1:50:02and V would be a two-dimensional object
1:50:06which is the the local slope
1:50:08of the
1:50:11e2i only y let me call it
1:50:14y would be a two-dimensional vector
1:50:18and what is v v is the it's a
1:50:22two-dimensional vector
1:50:24which is the the two slopes of your
1:50:28membranes locally okay so
1:50:31um I don't know if you
1:50:33be in space but let me draw
1:50:38a two-dimensional object so there would
1:50:40be one
1:50:42gradient in that direction and another
1:50:45gradient in that direction okay so here
1:50:48I'm assuming that the surface moves up
1:50:51when you go uh behind the board yeah and
1:50:55and the norm of this two-dimensional
1:50:58gradient
1:50:59would be the equivalent of the velocity
1:51:01and V squared now is the energy
1:51:05associated with bending your elastic
1:51:09energy associated with bending your
1:51:11membrane
1:51:12okay you're creating more stuff you're
1:51:15creating more Surface by having the
1:51:17membrane that's not flat exactly as
1:51:19you're creating more length when you're
1:51:21uh considering a polymer that's not
1:51:24straight okay and every piece of extra
1:51:27surface that you're creating costs an
1:51:30energy
1:51:30and that's where it comes from
1:51:33okay thanks
1:51:35but of course in the case of a
1:51:37two-dimensional time so to say this
1:51:40would be a two-dimensional time in the
1:51:42optimization problem it's uh you know
1:51:46it's not natural to think of time as a
1:51:48two-dimensional object so uh these these
1:51:52two-dimensional objects uh lose the
1:51:54direct connection with uh uh the
1:51:58treasure hunt problem if you want the
1:51:59optimization problem
1:52:01okay okay yeah my question that now it's
1:52:04clear was that in the polymer you have
1:52:07the energy associated to bending the
1:52:09polymer but with the domain walls it's
1:52:11rather that you try to minimize the area
1:52:15yeah exactly but it's the same in the
1:52:17case of the polymer you try to minimize
1:52:19the length in the case of a membrane you
1:52:22try to minimize the the area
1:52:25okay
1:52:32sorry I have a question as well
1:52:35yes
1:52:37so could you uh give us maybe a list of
1:52:41references well we could find more
1:52:44details on the different topics
1:52:48so normally you have chapter one that
1:52:51has been uh given to you and chapter two
1:52:55the one that I'm discussing now uh is
1:52:58going to be given to you now I'm going
1:53:01to give it to Valentina tonight
1:53:03and uh it's going to cover uh everything
1:53:07on that I've talked about on uh
1:53:10redistribution it won't cover this part
1:53:13on
1:53:14um uh Hamilton
1:53:16because I haven't had time yet to write
1:53:20everything up but I can certainly give
1:53:23you uh references to all these these
1:53:25problems
1:53:26sure
1:53:28okay thank you
1:53:37okay good well if you have any comments
1:53:40any complain any anything that you find
1:53:42useful for the rest of the lectures
1:53:44please tell me or tell Valentina
1:53:47um again I mean I think we're both here
1:53:50to make you as happy as possible so
1:53:54um we're clearly open to suggestion
1:54:01okay well have a good today and um
1:54:05yeah yeah okay sure anyway AGB I just
1:54:08started so um I'm going to uh go more in
1:54:12details next week and as Valentina just
1:54:14uh
1:54:15fed you're going to have uh today about
1:54:18this
1:54:22okay
1:54:23goodbye then
1:54:32this conference will now be recorded
1:54:39okay good
1:54:44let me see the chart
1:54:46but anyway
1:54:48okay so welcome back uh everybody to our
1:54:51uh to the session so uh I don't manage I
1:54:56would like to upload the text of the
1:54:58today which is the same one that I put
1:55:00on the folder except that I just
1:55:02switched the order of the exercises and
1:55:05I decided to uh talking here about
1:55:09exercise number three or what was before
1:55:12exercise number three because uh I think
1:55:15it is more connected to some of the
1:55:18things that will come up uh in the
1:55:20future during the lectures
1:55:22so now this exercise uh in the new
1:55:25version of The Today is the second one
1:55:27and the third one which was about power
1:55:29laws and we leave it as as a bonus
1:55:32exercise and maybe if we have time uh at
1:55:34the end of this session we can comment a
1:55:37little bit on uh what's in there
1:55:40so before uh starting as you see I wrote
1:55:44just a little bit of summary about what
1:55:46we are going to do today but before is
1:55:48there anybody who has any questions
1:55:50about the previous today or about the
1:55:52homework
1:55:53or comments about the lectures
1:55:57and just remember that in case you have
1:55:59questions which thumbs up during the
1:56:01week you can just write them in the
1:56:03question and answer folder that could
1:56:05also be comments about the speed of the
1:56:08lectures the material and so on and we
1:56:10will process them uh during the week
1:56:14okay so let's start with this uh today
1:56:16number two and this setting here is
1:56:19going to be about uh statistics and
1:56:22about inferring properties of
1:56:26distributions from data
1:56:28so this might look a little bit
1:56:31disconnected with respect to what we had
1:56:33in the last lectures but actually it is
1:56:36not and in particular I will make some
1:56:38comments at the end of uh of the today
1:56:41at the end of the second exercise that
1:56:43will come up uh in the course of the
1:56:45lectures in particular uh toward the end
1:56:47of the course so stay tuned
1:56:50and uh and for what concerns the things
1:56:54that were discussed today during the
1:56:56lecture so as I said there will be the
1:56:58third today that will be about uh
1:57:00nanjivan poker plan shatanovic versus
1:57:03Ito and stochastic calculus so that will
1:57:06be covered in detail next week and then
1:57:09the fourth today is about what has been
1:57:12discussed at the end of the lecture of
1:57:13today so Hamilton uh Jacobi Bellman
1:57:16equations and problems of optimal
1:57:19control
1:57:20so we will revise everything uh in in
1:57:24the weeks to come
1:57:26okay but let's start with uh this today
1:57:28in here and as you see the idea of the
1:57:32two exercises that we're gonna do is to
1:57:34uh start some data and uh estimate
1:57:39or model
1:57:41distributions based on the information
1:57:43that we have from sampling them uh and
1:57:47collecting data so the second is what I
1:57:50summarized in here the idea is that you
1:57:52have a sample of data that I call as n
1:57:55so you have small n values of uh of
1:57:59realizations of your random variables
1:58:01and in the course of these exercises we
1:58:04are assuming that this data are obtained
1:58:08sampling independently and I will
1:58:11comment on this at the end but so far we
1:58:14stick to this framework of independent
1:58:16sampling from a distribution uh for the
1:58:21corresponding random variables which is
1:58:22to some extent uh unknown and the idea
1:58:25is to extract information about this
1:58:27distribution from the data that we add
1:58:30so there are somehow two parts in this
1:58:33study one is related to estimating and
1:58:36the other one is what I call modeling
1:58:39so let me give you a summary I hope that
1:58:42you can see
1:58:43uh what I write what I wrote in here
1:58:46especially the colors but you see that
1:58:49there are two things so estimation is
1:58:52related to the first exercise that we're
1:58:54gonna discuss that is about maximum
1:58:57likelihood whereas modeling is more
1:58:59related to the second part which is
1:59:02about maximum entropy so what is the
1:59:05idea in these two cases so in the first
1:59:07case
1:59:08you are in a situation in which you are
1:59:11given a model so what I call a model
1:59:13here is a functional form of the
1:59:16distribution from which you assume that
1:59:18your your data are extracted so you know
1:59:21what is the functional form of your row
1:59:23but this row depends on a set of
1:59:26parameters which in principle are
1:59:28unknown and the goal in here is to
1:59:30estimate these parameters based uh based
1:59:35on the data so there is somebody who is
1:59:37asking about where to find that it is uh
1:59:41okay so there is a little bit of an
1:59:43issue which is that with the ens
1:59:45connection I cannot enter into that
1:59:47folder unfortunately so I will have to
1:59:50guide you uh based on memory but if you
1:59:52open the folder you should find
1:59:55um I guess a repository which is called
1:59:57M2
1:59:59and then inside you find the folder
2:00:00which is called complex systems from
2:00:02physics to social sciences you click in
2:00:04there you have subfolders one of which
2:00:07is called today's and homework and then
2:00:10you go to week two and in there you find
2:00:13the text of the second day so let me
2:00:15know uh ah I see
2:00:18uh yes
2:00:21so if anybody actually
2:00:27let me do this
2:00:29let me send it to the meaning list
2:00:33uh
2:00:42okay so now I sent the text to the main
2:00:45English and everybody should have it
2:00:46thanks for uh pointing this out
2:00:51okay so uh as I was saying in this case
2:00:55what we need to do is to estimate the
2:00:57values of these parameters based on the
2:01:00data so the input that we have are the
2:01:03functional form of of our distribution
2:01:06and we may have some prior information
2:01:09uh about how these parameters or what
2:01:13are the properties of these parameters
2:01:14we will see this in the exercise and the
2:01:17output will be an estimator so
2:01:21a value for this parameter that we get
2:01:24out of the data that we have and so this
2:01:26estimator will depend on the particular
2:01:29sample that that we have and in
2:01:31particular will depend on the size of
2:01:33the sample and this is why I put this
2:01:35little n in Brackets which indicates
2:01:39this dependence
2:01:40so this is uh the way in which we are
2:01:43going to do this is through a maximum
2:01:45likelihood as I will explain in a minute
2:01:48and then there is a second part and the
2:01:50second part is is about modeling so in
2:01:53this case we assume that we don't know
2:01:56even what is the form of our
2:01:59distribution but we only know some
2:02:02constraints that this distribution has
2:02:04to satisfy for instance we know what
2:02:06should be the average of certain
2:02:09functions which I call G of K with
2:02:12respect to the distribution of the data
2:02:13and the values of these averages may be
2:02:16given to you uh a priori or you may get
2:02:19them directly from from the data and
2:02:22this makes a connection with this first
2:02:24part and what are these functions well
2:02:26in most of the cases they are just the
2:02:28moments of your random variable so you
2:02:31know for instance what is the average of
2:02:34your random variable and you would like
2:02:35to find a shape a form for the
2:02:38distribution that is compatible with uh
2:02:40with the average that you have and there
2:02:43are in principle many distributions
2:02:45which are compatible so a whole family
2:02:47of those and the idea is to use the
2:02:50so-called maximum entropy principle to
2:02:52extract uh or to give a good model for
2:02:55this distribution so in this case the
2:02:57input that you have are the values of
2:03:00these constraints and the output will be
2:03:02the functional form for your
2:03:04distribution which of course will encode
2:03:06the information that you have and so it
2:03:08will depend on on the constraints that
2:03:11you have so this will be uh the topic of
2:03:14exercise one and this is the topic of
2:03:16exercise two and then I will comment on
2:03:18what happens when you go beyond this
2:03:21Assumption of Independence
2:03:24okay so let's start with uh maximum
2:03:29yeah likelihood so let me summarize a
2:03:32bit
2:03:33what is the idea
2:03:35in here
2:03:37so this is uh maximum likelihood
2:03:41estimator
2:03:42what is the idea
2:03:49and first of all what is the definition
2:03:51so the definition starts from a basic
2:03:55formula in probability which is which
2:03:57goes under the name of uh bias formula
2:04:00and the idea is the following so suppose
2:04:03that you want to compute what is the
2:04:05probability to get a certain value for
2:04:10your unknown parameters so see in
2:04:12general you can have more than one
2:04:14parameter that is unknown so I I denote
2:04:16it as a vector so this will be
2:04:19Theta 1 up to C10
2:04:22and so you want to know what is the
2:04:24probability of this set of day of
2:04:25parameters giving the values
2:04:29the sample that you have so giving the
2:04:32data that that you have and using the
2:04:34laws of conditional probability you know
2:04:37you see that this probability will be
2:04:40proportional
2:04:41so the idea is that the probability of
2:04:44theta given x times the probability of X
2:04:46is equal to the probability of x given
2:04:49Theta times the probability of theta so
2:04:52this is what I'm going to write so here
2:04:54in the right hand side I will have the
2:04:56probability of the set of data that you
2:04:58have given an assumption of your on your
2:05:00parameter Theta and using the idea that
2:05:04the data are sampled independently from
2:05:07an unknown distribution row what this
2:05:10probability will be is just the product
2:05:13overall the data that you have
2:05:15of my row
2:05:17of X I even
2:05:20a given value of the parameters
2:05:22and then you have the probability the
2:05:24probability for these parameters which
2:05:26of course are unknown but you can
2:05:29assume that you know something some
2:05:31properties of those that you encode into
2:05:35some prior distribution
2:05:40the prior which depends on on Theta
2:05:45and so in this language of bias and
2:05:48probability this is uh called the prior
2:05:50this quantity in here is called
2:05:53the likelihood
2:05:56and what you have on the left hand side
2:05:58which is more or less what you want is
2:06:00the posterior
2:06:06and now what is the maximum likelihood
2:06:07estimator well this is essentially the
2:06:10value of theta which maximizes what you
2:06:13have on the right hand side and to make
2:06:16things a little bit simpler we will
2:06:17maximize the logarithm the so-called log
2:06:21likelihood which is the logarithm of the
2:06:23right hand side so the value of my
2:06:27estimator which is now a vector so Theta
2:06:30is a vector and then I will put a knot
2:06:33to indicate that this is the value that
2:06:35we estimate from the data
2:06:37this is
2:06:39the maximum
2:06:41of a function
2:06:43that is a function that depends on the
2:06:46data that you have
2:06:49and which is a function that in general
2:06:52depends
2:06:53on Theta
2:06:56and what is this function well this is
2:06:58the log
2:06:59likelihood that as I said is the log of
2:07:03the right hand side so if I take the log
2:07:04of the product this is just the Sun
2:07:07someone going
2:07:09from a I going from one to n of the log
2:07:12of my row
2:07:15of x i given Theta
2:07:18and then if I have some prior
2:07:20information I have to add to this the
2:07:22log
2:07:24of this
2:07:25P prior
2:07:29over Theta
2:07:31so to solve the problem and get our
2:07:33estimator what we have to do is to
2:07:35compute this log likelihood and then to
2:07:37maximize it to get the value of the
2:07:40fetus
2:07:41now let me make a few comments before we
2:07:44see this concretely in an exercise
2:07:48so the first comment
2:07:54is that the problem that we are dealing
2:07:57with or the setting is essentially the
2:07:59one that one has in general in in
2:08:03inference problems
2:08:07so we are in an inference setting
2:08:12where we are assuming that our data are
2:08:16extracted from from a true from a real
2:08:18uh distribution so we are assume
2:08:23that there is a real distribution
2:08:25that has the functional form
2:08:28that uh that we that we put into the
2:08:32into the model but which has some real
2:08:36values of this parameter that I call uh
2:08:40Theta star so this is what usually is
2:08:42called the hidden truth
2:08:49so we don't know what is the true value
2:08:51of these parameters but of course what
2:08:53we are doing in here is estimating this
2:08:56value from the sample of data that we
2:08:58have and just as a notation in the
2:09:01following I will denote
2:09:06with
2:09:08these
2:09:09expectation value and I will denote with
2:09:13these variance
2:09:15the expectation value and the variance
2:09:18with respect to the true underlying
2:09:20distribution so this is the average
2:09:22computed with respect to the row where
2:09:25instead of theta I put the true value of
2:09:29um of the parameters which is of course
2:09:31what we want to estimate
2:09:34so as I said from data we get
2:09:37some possible estimates
2:09:40for this set of parameters now let me
2:09:42assume just for the notation that we
2:09:44have one single parameter so I don't
2:09:46have to use vectorial notation
2:09:49so our maximum likelihood estimator will
2:09:52be some number and what you can show is
2:09:54that this number or this estimator is
2:09:57consistent in the sense that if you take
2:10:00your sample to be of infinite size so if
2:10:03you take small and going to Infinity you
2:10:07know that this will converge to the true
2:10:09value of the distribution from which you
2:10:13are something the data
2:10:14so you have this sort of uh if you want
2:10:17kind of low of large number for your
2:10:20estimator
2:10:21and you can ask of course how does this
2:10:25convergence happen as a function of your
2:10:28small n
2:10:32so how does it happen and what you can
2:10:33show is that it happens with
2:10:36fluctuations around the True Value which
2:10:40are the ocean so what I mean is that if
2:10:43you look at the rescaled variable which
2:10:45is rescaled by square root of n as you
2:10:48usually do in central limit theorem for
2:10:51example and you look at the fluctuations
2:10:53of your estimator with respect
2:10:57through the True Value
2:10:59what you can show is that asymptotically
2:11:01so when n
2:11:03is large this object will be distributed
2:11:07as a gaussian
2:11:09so we have a normal distribution and
2:11:11denote it like this and this gaussian
2:11:13has zero average
2:11:16and it has a variance that is
2:11:19related to
2:11:22a function e that is what people call
2:11:25decision information so this e in here
2:11:29is known as the Fisher information
2:11:36and what is
2:11:38this expression so let me try to squeeze
2:11:41it in here so e is a function of your
2:11:44parameter and it is
2:11:47minus the expectation value
2:11:50now with respect to the true
2:11:51distribution
2:11:54of uh of what of the second derivative
2:11:59the log of your distribution
2:12:03with respect to your parameter
2:12:09okay and if you find uh also in the text
2:12:12of uh of the today so to characterize
2:12:15the synthetically your distribution what
2:12:16you know is that uh in the limit of
2:12:20large n your variance will be controlled
2:12:22by the value of this feature information
2:12:24at the uh true value of the parameter
2:12:27Theta star now of course usually you are
2:12:30not in the very large and limit so what
2:12:33you can ask is uh what happens when your
2:12:37sample is maybe very large but it is not
2:12:39strictly infinite and in that case what
2:12:42you can show is that the variance of
2:12:45your
2:12:46um of your estimator is not exactly
2:12:49given by uh one over I but it is lower
2:12:52bounded by uh one over I so let me put
2:12:56it in here
2:12:58this is the last
2:13:00formula that I give you before the
2:13:02exercise
2:13:04so for
2:13:07finite and
2:13:10what you have is that the variance
2:13:14of your estimator
2:13:16is lower bounded by
2:13:191 over n e
2:13:22of theta
2:13:24okay so this one over n comes
2:13:27so that's not visible anymore
2:13:30ah you're right
2:13:34because of the screen not because of
2:13:39okay now it's better
2:13:43yes
2:13:44okay great
2:13:47so of course this is just telling you
2:13:49that if you look at the finite sample
2:13:51the fluctuations that you get are larger
2:13:54than the fluctuations that you have in
2:13:56the asymptotic limit but you still get
2:13:59an estimate of those looking at this
2:14:01official information
2:14:03so now we are going to look at all of
2:14:05this formula a little bit more
2:14:07concretely by solving the fresh exercise
2:14:10and I hope that things will become even
2:14:13more clear
2:14:18okay let me erase this
2:14:36maybe let me add the comments while I
2:14:38erase
2:14:39which is that I wanted to stress that
2:14:42somehow the framework that you have in
2:14:44here is the one of an inference problem
2:14:46because I guess that some of you
2:14:49have followed courses in the last
2:14:51semester there was a course for instance
2:14:53by floran jacquela that was about
2:14:56inference problems and nowadays this
2:14:59type of influence problems are becoming
2:15:01quite popular even in in research in
2:15:04particular in the limit of a large
2:15:07Dimension and this fits a little bit
2:15:09into this framework and in that context
2:15:12if you look at high dimensional
2:15:13inference and even problems related to
2:15:16machine learning you will see these
2:15:18concepts of Maximum likelihood which
2:15:20emerge constantly in the literature
2:15:25okay so now let's uh try to be more
2:15:28concrete and to look at an exercise and
2:15:33maybe before uh there was an example so
2:15:36in in the text of the today and the
2:15:38example is so I'm not going to solve the
2:15:41example I just wanted to make a comment
2:15:42so in that example what you want to do
2:15:45is to estimate the exponent of a power
2:15:49load distribution using a maximum
2:15:51likelihood and this is related if
2:15:53anybody had time to to do the arm works
2:15:56is related to the homework too so the
2:15:58idea of homework 2 was to use this
2:16:02framework to try to fit data which are
2:16:05asymptotically power law and estimate
2:16:09what is the exponent indeed based on the
2:16:11data and if you use this recipe of
2:16:14Maximum likelihood you end up with an
2:16:17expression for your estimated exponent
2:16:20mu that is of the form
2:16:241 over 1 over n sum I going from 1 to n
2:16:29of log of x i which are your data
2:16:33divided by XC so what exact C well what
2:16:38I'm doing in here is assuming that my
2:16:40distribution
2:16:42rho of x given the exponent mu is power
2:16:46low but it starts so it's power law only
2:16:48asymptotically so the power load starts
2:16:51from a given value of x which I call
2:16:55XD
2:16:59so you have to look at values that are
2:17:01larger than this XC and what I wanted to
2:17:03point out with uh with this example and
2:17:07with homework tool is just that if you
2:17:09use maximum likelihood you get this
2:17:11estimator which is called
2:17:13the hill estimator
2:17:17which depends
2:17:19on this particular value of XC so when
2:17:24you try to fit an estimate powers from
2:17:27data I there is a little bit of
2:17:30sensitivity with respect to this
2:17:32parameter that you have to choose so you
2:17:34have to play around change a little bit
2:17:36this XC if you don't have of course an
2:17:39exact power law but data which are only
2:17:41asymptotically Power low and you will
2:17:44see that your estimator changes but the
2:17:46good choice of XC is signaled by the
2:17:49fact that once you get it and you vary a
2:17:52little bit the value of XC you find that
2:17:55this quantity doesn't change much and
2:17:57therefore you have a good estimate of
2:17:59your exponent so if you want to to look
2:18:01at an example concretely this was given
2:18:04this was the ideal homework
2:18:07where we use this to estimate the power
2:18:10flow of the distribution of words in
2:18:13language that we saw in the homework
2:18:15number one
2:18:17okay but this is just a parenthesis so
2:18:20now let me go to
2:18:22uh to the two exercise and maybe I will
2:18:24erase this
2:18:25to have space
2:18:34so in this exercise one
2:18:39foreign
2:18:42we assume that our data are taken from
2:18:44some gaussian distribution of which we
2:18:47don't know either the average the mean
2:18:49and and the variance
2:18:52so our row of X will depend on this
2:18:54unknown parameter the mean which I call
2:18:56n and the variance let me call it B
2:19:00this is Sigma Square
2:19:03and this has a gaussian
2:19:04shape so this is one over square root of
2:19:072 pi v e to the minus x minus M Square
2:19:11divided by 2B
2:19:13so M and V are play the role of my Sita
2:19:18the formulas above are the parameters
2:19:20that we want to estimate and we have
2:19:22some prior information on both m and b
2:19:26so for m
2:19:28we know that or we assume that this is
2:19:32distributed a priori as a gaussian with
2:19:35some variance capital sigma
2:19:38and with zero average the prior will be
2:19:40e to the minus and square over 2 capital
2:19:44Sigma squared divided by
2:19:47square root of 2 pi
2:19:49capital Sigma Square
2:19:52and for the variance we assume
2:19:55well we have at the beginning we have
2:19:58good reasons to believe that this is
2:20:00distributed as an exponential
2:20:03so e to the minus B Lambda divided by
2:20:06Lambda
2:20:08okay and now we just want to apply this
2:20:12recipe to get estimates for this
2:20:13parameters so of course
2:20:15the first thing I hope you can see if I
2:20:18do this the first thing that we have to
2:20:20do is to
2:20:22um is to compute the log likelihood
2:20:26okay
2:20:28so what is the log likelihood in this
2:20:31case
2:20:31well what we have to do is to take the
2:20:34sum overall the data
2:20:38that we have
2:20:39of the log of this distribution and now
2:20:42I will use the fact that eventually what
2:20:45I want to do is to take the derivative
2:20:46of this object with respect to M and V
2:20:50so I write down explicitly only the
2:20:53terms which depends on m and b and all
2:20:56of the other terms I will collect them
2:20:58into a constant because they don't
2:20:59matter when I look at the derivative so
2:21:02if I do this I get the first term which
2:21:05is the logarithm of this so I get a
2:21:08minus sign which comes from the
2:21:10exponential so it's minus
2:21:13x i minus M Square divided by 2v
2:21:17then I have the logarithm of the
2:21:19normalization so I just keep the factor
2:21:21of B and this gives me
2:21:23-1
2:21:25log of V
2:21:27and then I have the logarithm of this
2:21:29prior distributions so the first one
2:21:34will give me a factor of minus M Square
2:21:37divided by 2 capital Sigma squared and I
2:21:41can neglect the normalization because it
2:21:43does not depend on M and on B and from
2:21:47here I just get minus V over Lambda
2:21:51and then all the rice is collected into
2:21:53some constant which depends
2:21:56on C Min Lambda
2:21:59okay so this is my likelihoods now this
2:22:01depends on two parameters and I have to
2:22:04take the derivative with respect to both
2:22:06so let's start from n let me take
2:22:10uh the first derivative of the
2:22:13likelihood
2:22:14with respect to n so why should I take
2:22:16the derivative because I want to
2:22:18maximize so I look at stationary points
2:22:22and if I do this what do I get so I have
2:22:25a minus in here that will cancel from
2:22:27the minus with the minus coming from the
2:22:30derivative Over N so I'm left with
2:22:33sum from I going from 1 to n of
2:22:37x i minus m
2:22:40I bring down a factor of 2 which
2:22:42consists with 2 so everything is divided
2:22:44by B
2:22:44[Music]
2:22:46and then the other term is this one and
2:22:49this will give me minus M over
2:22:53stigma Square
2:22:54okay now what can I do I can I have to
2:22:58solve for M so let me multiply this by V
2:23:03okay
2:23:05and let me do the following so this is
2:23:08the sum of a term which depends on I and
2:23:10a term which does not depend on ice so I
2:23:12can just
2:23:13sum over the second one and this will
2:23:15just give me m times small n which is
2:23:19the number of terms on which I'm summing
2:23:21let me divide by small n
2:23:25this is just algebra
2:23:27okay
2:23:28and now let me solve for n so if you do
2:23:32this
2:23:32what you get is that m is equal to what
2:23:36is equal to 1 over n
2:23:39sum
2:23:41one over n x i
2:23:43divided by a factor which is of the form
2:23:46one plus
2:23:48V over n capital Sigma Square
2:23:53so what you see is that in the numerator
2:23:56what I have is is essentially the sample
2:23:59average of my sample as n so I will
2:24:05just they note it as the average
2:24:08of X over the sample and then I have
2:24:11this factor in the denominator
2:24:14which contains the information
2:24:17on the prior distribution through this
2:24:20Sigma square and through the variance
2:24:24that we have to fix taking the second
2:24:27derivative
2:24:29sorry not the second derivative but the
2:24:31derivative with respect to B
2:24:33okay so this is the first this will be
2:24:36our estimator for n
2:24:39and now what we have to do is the same
2:24:41thing now taking the derivative of this
2:24:43guy with respect to V so let me just
2:24:46catch it
2:24:52we will get a second order equation so
2:24:54okay from here the minus chances and you
2:24:58get some
2:24:59I going from 1 to n
2:25:03X nine minus M let me know if you don't
2:25:06see
2:25:08what I'm writing so I have 2 over B
2:25:11squared then the logarithm which gives
2:25:13me 1 over 2 V
2:25:16and then what do I have I have this one
2:25:19which gives me a factor of
2:25:211 over Lambda
2:25:23so let me multiply this since I have to
2:25:26set this equal to zero as above
2:25:30let me multiply by MB Square
2:25:34so here we'll have a factor of v and
2:25:37here I will have a factor of
2:25:39D Squared
2:25:41and this has to be equal to zero so I
2:25:43have to solve a second order
2:25:47equation so I just write the result
2:25:57save some time
2:26:02you can do it while I erase
2:26:18easy
2:26:22so if you solve this you will see that
2:26:25your estimate for V
2:26:27use of this form and Lambda divided by 4
2:26:32then you have a minus one
2:26:34and then you have plus square root of
2:26:38one
2:26:39plus eight
2:26:41over Lambda N squared
2:26:45sum over I from 1 to n of x i minus here
2:26:49I have M so I replace it with my
2:26:52estimate
2:26:54to the power 2. okay and I have to
2:26:58choose the plus sign because I want the
2:27:01variance to be positive
2:27:02so here I have a minus one so I have to
2:27:04add something which is positive to it to
2:27:08get a positive variance
2:27:09okay so this gives us
2:27:13the expressions for uh for the two
2:27:16estimators
2:27:17and of course in this V here you have to
2:27:20replace it with with the head so these
2:27:23are coupled
2:27:25and now let's go to uh to the second
2:27:28point this was 0.1
2:27:34and the second point is about what
2:27:36happens when we take the limit of the
2:27:39sample
2:27:41small and
2:27:43going to Infinity
2:27:46so we have to take the limits of this
2:27:47expression so the first one is pretty
2:27:49easy so you see that the only dependence
2:27:52on small line is in the denominator and
2:27:54this is such that if you sign into
2:27:57Infinity uh you get your life only with
2:28:01one so this means that in this limit
2:28:03your estimate
2:28:06for the average
2:28:08is just
2:28:10the sample average
2:28:14based on your data so the dependence on
2:28:16Sigma disappears
2:28:18and what happens for the variance well
2:28:21something very similar happens so how do
2:28:23you take the limits here
2:28:24you see that when n is large this object
2:28:28is small so you have one plus something
2:28:30small to the square root so this you can
2:28:33approximate as one plus one alpha times
2:28:37the small thing this is just a tailor
2:28:39expansion and if you do this the one
2:28:42cancels and you will see that you cancel
2:28:45one factor of N and you're left with
2:28:48something very simple
2:28:50to expect which is that this is just
2:28:54this sample variance
2:28:58that I will denote as x minus
2:29:02let me see M hat averaged over all of
2:29:06your data
2:29:11and this is something that is good and
2:29:14that you may expect so what this is
2:29:16telling you is uh something that you can
2:29:20summarize as the expression that
2:29:23evidence
2:29:27takes over
2:29:33so what this is telling you is that you
2:29:36add some prior gas
2:29:37of what was uh what were possible values
2:29:42for M and for V given their distribution
2:29:45but in the limit in which your sample
2:29:47becomes infinite the information about
2:29:50this guess becomes not relevant so it
2:29:52disappeared from from your expression
2:29:55and the only thing that you're left with
2:29:57are are the data so in the limit in
2:30:00which n goes to Infinity evidence which
2:30:03means the information which is contained
2:30:05in your data becomes uh be only relevant
2:30:09scene to estimate this this parameters
2:30:13and you can forget about your your prior
2:30:15information because your data are enough
2:30:19to give you information about the
2:30:22structure of your distribution
2:30:24sorry
2:30:29uh yes
2:30:31thanks
2:30:34and uh what the thing is that this is
2:30:36exactly the same thing that you would
2:30:38get if you had no prior information at
2:30:40all which is consistent with what I just
2:30:42said so no prior information
2:30:45uh would be would correspond to the
2:30:48limit of this expression in which Sigma
2:30:50goes to infinity and Lambda goes to
2:30:54Infinity
2:30:55so that's basically your gaussian and
2:30:57your exponential becomes uh kind of flat
2:31:00so you have no information a priori on
2:31:03what is the distribution of your
2:31:05parameters and in that case the only
2:31:07thing that you have are data and so you
2:31:09get that your estimators are precisely
2:31:11just given by data
2:31:14okay
2:31:15so now we are going to use this
2:31:18no prior information to uh to comment a
2:31:22little bit on this uh feature
2:31:25information and on this uh bound which
2:31:29is called the grammar brow bound on the
2:31:32variance of your estimators and this
2:31:34concludes
2:31:35uh the exercise so where shall I do it
2:31:38let me do it well maybe this is
2:31:41uh useful
2:31:44let me do it up here
2:31:58if it is not clear so far please stop me
2:32:01anytime
2:32:08foreign
2:32:12so let's go to point three and point
2:32:14three I think it tells you
2:32:17the following so suppose that now
2:32:20we know
2:32:22that our average uh is equal to zero
2:32:27so we know that this we can set it to
2:32:30zero and we have no prior information on
2:32:33uh on our variance
2:32:36and therefore based on the equation that
2:32:38I just erased
2:32:40our maximum likelihood estimator for the
2:32:43variance is just
2:32:451 over n some
2:32:47over our data of x i Square
2:32:51where I don't have to shift by the
2:32:54average because I assume it to be zero
2:32:56and now what we want to do is to compute
2:33:00the left hand side of this inequality
2:33:03so you want to compute the variance of
2:33:06this estimator with respect to the true
2:33:08distribution and what is now the true
2:33:10distribution so we are assuming
2:33:13that this is gaussian
2:33:15it has a true value
2:33:17for the for the variance which I call
2:33:19now V Star
2:33:21so it is just one over square root of 2
2:33:23pi
2:33:24star times
2:33:27uh e to the minus x square over 2 D star
2:33:34okay so let's compute the average of
2:33:36this estimator with respect to this
2:33:38distribution so as I said before I
2:33:42denote it like this this expectation
2:33:44value so I have to compute
2:33:46the expectation value of this sum
2:33:52sorry
2:33:53the variance
2:33:57now I use the property of the variance
2:33:59the variance of a constant times your
2:34:01random variable is the constant Square
2:34:03Times the variance of your random
2:34:04variables
2:34:06and moreover the variance is linear so
2:34:08the variance of a sum is the sum of the
2:34:10variances so what I get
2:34:12is using that my distribution is is the
2:34:15option I get a factor of
2:34:171 over n so it would be 1 over n Square
2:34:20coming from this and then an extra
2:34:22factor of n which comes from the sum and
2:34:25then I have just the variance
2:34:27of my random variable X
2:34:31and this is what well this is 1 over n
2:34:33the expectation
2:34:35of
2:34:37sorry this was x squared
2:34:39so this is the expectation of uh the
2:34:43object Square so this will be x to the
2:34:46fourth
2:34:47minus
2:34:49the expectation of the object to the
2:34:52power 2. I'm just using the definition
2:34:54of the variance in here
2:34:57and now I want to compute this
2:34:59expectation with respect to uh to the
2:35:01gaussian distribution and I use to to
2:35:03compute this expectation of x to the 4 I
2:35:07use a property of the gaussian
2:35:09distribution which tells you that all of
2:35:12the moments which are higher than two I
2:35:15can factorize them into products of the
2:35:19expectation values of x square so this
2:35:21is what is usually called in many
2:35:24contexts as the weak theorem it's a
2:35:27property of of gaussian measures so if
2:35:31you do field Theory you will see this
2:35:32emerging many many times so this is a
2:35:35simple example the idea is that
2:35:38to compute the average of x
2:35:42to the power of 4 you can write it as
2:35:46the sum over all possible contractions
2:35:48in which you two you take two values of
2:35:50x and you compute the expectation and
2:35:52this is factorized from the contraction
2:35:55of the remaining two values of x
2:35:58so what you find is how many ways you
2:36:02have to pair these four elements into a
2:36:05couples of two well you have essentially
2:36:07three ways so either I pair this one
2:36:09with this or this with this or this with
2:36:11this
2:36:12so what weak theorem tells me or if you
2:36:15want just properties of the gaussian you
2:36:17can compute it is that this will be
2:36:20equal to three times
2:36:21the expectation value of x squared
2:36:24which is one pairing times the
2:36:26expectation value of the other pairing
2:36:28so this is just
2:36:30three times the expectation of X the two
2:36:33to the power 2.
2:36:35which combines nicely with this so in
2:36:38the end what we find using this trick is
2:36:42that our variance is what is 1 over n
2:36:48times
2:36:50twice
2:36:52the expectation of x squared with
2:36:54respect to our gaussian distribution and
2:36:56this is just the variance to the power
2:36:592. so this will be
2:37:01V Star
2:37:03to the power 2.
2:37:07okay so what uh I hope this is fine
2:37:10anyway
2:37:12or in any case just stop me but this is
2:37:16basically what you have uh on the left
2:37:18hand side of our uh grammar shroud bound
2:37:22and now let's try to compute for this
2:37:24particular example what is the right
2:37:26hand side so what is the fissure
2:37:28information and see
2:37:30what we get
2:37:34so let me erase maybe this part
2:37:49okay
2:37:52okay
2:37:59so this is the last point
2:38:03and the last point is asking us to
2:38:05compute now
2:38:08e of V which is our parameter
2:38:13that is as a shade minus
2:38:16the expectation value of the second
2:38:18derivative
2:38:20of the log of rho of x given B
2:38:25derived over B
2:38:28okay
2:38:30so before we do this
2:38:33computation there is a question in the
2:38:36exercise which is
2:38:37uh let's say let's try to give an uh an
2:38:42interpretation to this feature
2:38:44information and the idea is that you can
2:38:48interpret this official information as a
2:38:52measure of how much your model so your
2:38:56distribution is is informative
2:38:59so and indeed this is what the name is
2:39:02suggesting and why is it so well if you
2:39:06have no prior what you see is that uh
2:39:09this type of this is a second derivative
2:39:11and if you just have the first
2:39:13derivative what you are taking in here
2:39:14is basically the first derivative of
2:39:17your uh log likelihood so it is what you
2:39:20should set to zero in order to fix your
2:39:24maximum likelihood estimator
2:39:26and then you take the second derivative
2:39:28which tells you about the fluctuations
2:39:30so about how much this function changes
2:39:33if you change the value of V
2:39:36and it is that your distribution is very
2:39:39informative if uh this uh let's say
2:39:42fluctuations are large so this is
2:39:44telling you that uh
2:39:47it is quite easy to locate what is the
2:39:49true value of V because as soon as you
2:39:51move away from the true value of V
2:39:54your uh let's say distribution is
2:39:58changing is changing a lot and this is
2:40:00measured this type of information is
2:40:03measured by uh by an object like this
2:40:05and this is why you see that in this
2:40:09inequality the feature information
2:40:10appears in the denominator so the idea
2:40:13is that the larger it is the smallest is
2:40:17your lower bound and therefore the the
2:40:20more hope you have that the variance of
2:40:22your data is small so if you're inside
2:40:24they have a very small feature
2:40:26information this is telling you that the
2:40:29shape of your distribution is not very
2:40:31informative and so you will know for
2:40:33sure that the variance of your data will
2:40:35be larger than than some threshold which
2:40:39is sufficiently much larger than zero
2:40:43so this is why this is called uh
2:40:45information indeed and now let's see
2:40:47what uh what this gives us in this case
2:40:50so what we have to do
2:40:53is
2:40:55um to compute the second derivative
2:40:58so
2:41:00let me
2:41:03maybe just do it quickly
2:41:10what is it
2:41:12okay this I just said
2:41:18foreign
2:41:19[Music]
2:41:21if I do this
2:41:24computation in here
2:41:28now I hope that with the factors then
2:41:30I'm fine so what I get is
2:41:33okay so where is my likelihood I erased
2:41:36it
2:41:37oops I will just give you uh the results
2:41:40so
2:41:42my likelihood derived with respect to V
2:41:45so now this is as an extra factor of
2:41:48small land but just to link with the
2:41:51notation Above This was given in the
2:41:54case where you have no prior so when
2:41:57Lambda goes to Infinity
2:42:00by something like this
2:42:02minus n of the two bit so this came from
2:42:06the logarithm of the distribution of M
2:42:08and this and this comes from the
2:42:13gaussian term
2:42:14and then if I take
2:42:16a second derivative
2:42:20of V what do I get so now I have no
2:42:23space
2:42:24foreign
2:42:40coming from here this was a square sorry
2:42:43so I get a minus
2:42:45um
2:42:48x i Square over V Cube and then the
2:42:52derivative from here will give me a
2:42:55my n over 2 V Square
2:43:02okay
2:43:03[Music]
2:43:05and what is this well this is
2:43:07essentially n times
2:43:10the fissure information uh that I wanted
2:43:13to compute why and times because I'm
2:43:15summing overall values uh overall
2:43:19possible values of my data and this
2:43:21gives me this factor of n so
2:43:24in a nutshell
2:43:26uh sorry
2:43:28It's Not Yet TV this is n times
2:43:31the second derivative of One log
2:43:34over V Square
2:43:37okay so now to get EV what I have to do
2:43:40I have to divide uh by n and then I have
2:43:43to take the expectation of minus this
2:43:46object in here so e v
2:43:50times n
2:43:52is nothing but minus
2:43:55the expectation of this object up here
2:43:58so the expectation of the first term is
2:44:02is 1 over
2:44:05V cubed times the expectation of
2:44:10x squared
2:44:12and then I have a factor of small n
2:44:15and the expectation of the second term
2:44:16is easy this is just the constant so
2:44:19this is n over 2 V squared
2:44:23and so you see that this will give me an
2:44:26extra factor of B
2:44:27so I will have uh and I add a minus that
2:44:32I did an account for so I will have n
2:44:34over V Square minus n over 2B squared so
2:44:37the result is
2:44:39n over 2 b square
2:44:42and this is n times
2:44:44the uh the fissioning formation
2:44:48okay and so what do I see from this well
2:44:51what I see is that if I now compute the
2:44:54right hand side
2:44:56so what I just gave in here add to the
2:45:00true value of V
2:45:01I just get 2 V Star to the power of 2
2:45:05divided by n which is exactly what we
2:45:08got in the left hand side so this is an
2:45:11example in which actually the bound that
2:45:15you see up there is saturated
2:45:17in this case and whenever this happens
2:45:20people say that the maximum likelihood
2:45:23estimator is is efficient so this is the
2:45:27terminology that you get so let me write
2:45:29it here
2:45:32in this case your estimator is
2:45:38is efficient
2:45:41meaning that you get a bound which is uh
2:45:46which is no longer a bound so you really
2:45:48know Computing the feature information
2:45:50what is the variance of your finite and
2:45:53Sample
2:45:55okay so I hope this was more or less
2:45:58clear
2:45:59um
2:46:01apart from these factors when which were
2:46:04fluctuating but I hope now it's fine
2:46:07so are there any questions about this
2:46:10exercise
2:46:12and if not I think we can jump
2:46:15to the second case which is about
2:46:18modeling so computing
2:46:21distribution based on maximum entropy
2:46:28okay
2:46:30and then I will conclude
2:46:33with a comment on correlation
2:46:48okay so maximum entropy I think you have
2:46:51encountered it several times it is
2:46:55one of the ways as we will see now in
2:46:58which you justify
2:47:00the emergence of uh Gibbs distribution
2:47:03for instance
2:47:06and the idea is as follows so this is
2:47:10now maximum entropy
2:47:14this is the idea
2:47:17so the idea is that you won't to get a
2:47:20distribution that I call row maximum
2:47:23entropy of your random variable X
2:47:26by maximizing something which is the
2:47:28entropy
2:47:34that is a functional of your
2:47:36distribution
2:47:38that you recognize
2:47:41from any Physics course so this is a
2:47:44functional because it is a function of
2:47:46the full distribution which is itself a
2:47:49function so here I'm denoting the
2:47:51variable to which it refers and this is
2:47:54just minus
2:47:55the integral
2:47:58of row x times the log
2:48:01of row X
2:48:02[Music]
2:48:04and you want to maximize these objects
2:48:06but in a way which is compatible with
2:48:09with a constraints that you have so let
2:48:12me rewrite this
2:48:15so this is subject to
2:48:18constraints which I wrote in the
2:48:20following form
2:48:21so we know what is the value so this bar
2:48:24in here it's just an annotation it's
2:48:26just to tell you that this is a number
2:48:28and it is indeed octane as the average
2:48:33of the corresponding function
2:48:36over your distribution
2:48:37[Music]
2:48:39and the prescription to do this uh
2:48:43constrained maximization is
2:48:45as usual to use LaGrange multipliers so
2:48:50via the so-called
2:48:51LaGrange method
2:48:55um
2:48:58so I will review it now and uh and what
2:49:01do you get out in this type of
2:49:03calculations well you always get out
2:49:06shapes for this distribution that are of
2:49:09the exponential form and the reason is
2:49:11that you have a logarithm here and this
2:49:14is what gives you exponential shapes so
2:49:16let me show this very quickly with with
2:49:19an example
2:49:21so first of all how many constraints do
2:49:24you have well you can have many so let
2:49:26me say that my index K goes from 1 to
2:49:30capital K
2:49:32in general
2:49:35Okay so
2:49:37let's do then the first two points of
2:49:40the second exercise which is just an
2:49:42example of this
2:49:44so the idea is to show that if you are
2:49:47in such a setting the outcome of such a
2:49:50maximization is of the following forms
2:49:53you get a distribution
2:49:55foreign
2:49:57X which you can write as e to the
2:50:01some constant which accounts for the
2:50:04normalization and then you have a sum
2:50:06overall your constraints
2:50:10so from one to capital n a capital K
2:50:12sorry
2:50:13of
2:50:15some parameters which are your LaGrange
2:50:18multipliers as we will see times the
2:50:21function itself
2:50:24so why do we get this let's show this
2:50:27so what we have to do is to maximize
2:50:30this object and this LaGrange method
2:50:33means that you introduce a modified
2:50:36version of this functional
2:50:39that depends on
2:50:42on other LaGrange multipliers so now let
2:50:45me introduce a Lambda 0 and the vector
2:50:48mu and the vector mu collects
2:50:50all of these values in here so this goes
2:50:53from 1 to
2:50:55mu capital K
2:50:57[Music]
2:51:00and this new functional is the
2:51:02functional of before so your entropy
2:51:05then you add the constraints
2:51:08so the constraints are the sum of your
2:51:12LaGrange multipliers times what you want
2:51:15to enforce
2:51:16and what you want to enforce is
2:51:21that the average
2:51:23of your functions
2:51:27take the given value that you are given
2:51:31in as input and you have an extra
2:51:33constraint which I impose with a
2:51:36multiplier lambda zero which is that you
2:51:37want your distribution to be normalized
2:51:40so you want them to integral
2:51:42of the X row of X is equal to 1.
2:51:47okay so let's maximize this so let's
2:51:51take the derivative of this with respect
2:51:53to the distribution row so this is a
2:51:55functional derivative of this object
2:51:58which I denote like this
2:52:03Lambda 0 Mo with respect to rho
2:52:07so if anybody is not familiar with with
2:52:11functional derivatives we will use them
2:52:13also in the next per day by the way but
2:52:16the idea is it works exactly as a
2:52:19derivative where a you essentially
2:52:22drive with respect to the full functions
2:52:25so for instance I will show it with with
2:52:27a concrete example and it will become
2:52:29clear
2:52:30so the first term that I have to derive
2:52:32is just the entropy so this is row Times
2:52:36log row so the first derivative
2:52:39will eliminate this factor of row so I'm
2:52:43left with minus
2:52:44log of rho of x
2:52:49and then I take the derivative of rho of
2:52:52X it gives me a 1 over rho of X which
2:52:54comes us with this so I have an extra
2:52:56factor of -1
2:52:58and then all of the other terms they are
2:53:01linear in row of X so what I obtain is
2:53:05sum over all values of K of mu k
2:53:12g k of x
2:53:14from here and then from the
2:53:16normalization I have a factor of Lambda
2:53:190. okay
2:53:23and now in order to recover so this I
2:53:26have to impose this uh to be equal to
2:53:29zero because I want the maximum and you
2:53:32see I introduced Lambda 0 because I just
2:53:35wanted to call MU zero
2:53:37as Lambda 0 minus one so I absorb also
2:53:41this constant and if you solve these
2:53:44equations uh equal to zero for rho of X
2:53:48you get out exactly the type of
2:53:51expression
2:53:52that you see in here
2:53:56okay so this gives you the exponential
2:53:58shape that as a shade comes from this
2:54:01logarithm
2:54:02but there is something more which you
2:54:04have to do because of course now this
2:54:06multipliers are not defined
2:54:09so you have to fix the values of this
2:54:12LaGrange multipliers
2:54:14using the constraints
2:54:16that you have
2:54:20so let's do it quickly
2:54:23foreign
2:54:25[Music]
2:54:29comes from normalization so what I have
2:54:32to do is I integrate that expression for
2:54:35rho over X and I set it equal to 1 and
2:54:39this tells me that e to the minus mu 0
2:54:42is equal to the integral
2:54:44over DX
2:54:46of e sum
2:54:49over all my values of K of mu k
2:54:52times g k of x
2:54:55right
2:54:57so this is an equation for Mu zero that
2:55:00tells me that mu 0 will be a function
2:55:05of all of the other values of mu
2:55:08given by this implicit relations
2:55:11and then what is the equation that I get
2:55:12for each other value of mu well what I
2:55:16have to do is to
2:55:17rewrite this constraint in here plugging
2:55:21the expression for my maximum likelihood
2:55:24density
2:55:26and so what I get so let me do something
2:55:29which will resonate with you let me call
2:55:31this
2:55:32uh
2:55:36normalization in a partition function
2:55:39and what I will get
2:55:41out of the other constraints is that
2:55:45G average k
2:55:47is what is 1 over Z so e to the MU zero
2:55:52times the integral over DX of just
2:55:56G over x e to the
2:56:00my sum
2:56:06okay
2:56:09and this is
2:56:10what fixes all of your LaGrange
2:56:13multipliers
2:56:14and as you see this is basically what
2:56:16you do so this is very reminiscent of uh
2:56:19what you do when you do basics
2:56:22of statistical physics so the idea is
2:56:25that for example uh in statistical
2:56:29physics you may interpret your X as a
2:56:32configuration and the constraint that
2:56:34you that you may have is for instance
2:56:36the value of the average energy so let
2:56:39me write it as a comment
2:56:41foreign
2:56:46so you may choose your just one value of
2:56:50K
2:56:52and you may choose then your function to
2:56:54be
2:56:55the energy as a function of the
2:56:57configuration
2:56:59so I will now call it small age
2:57:03and then uh what you have is you have a
2:57:06certain value of the average energy and
2:57:09what you are looking for is a
2:57:10distribution which is compatible with
2:57:11this average energy and out of that
2:57:14formula what you get is precisely a
2:57:17distribution of the
2:57:19boatsman form so you will get that row
2:57:22foreign
2:57:25one over z e to the MU times
2:57:30h of X where new place the role of minus
2:57:35beta in the usual physics notation and
2:57:39what is Mu zero well you can identify
2:57:41some from this type of relation mu zero
2:57:44with essentially the free energy of uh
2:57:47of your of your system and these type of
2:57:50equation gives you the usual relations
2:57:52between indeed the energy as as a
2:57:57derivative of the log of the partition
2:57:59function so of your free energy so you
2:58:01recover everything from this framework
2:58:04in here
2:58:04and you can go beyond this so of course
2:58:07in this case you just have one
2:58:09multiplier to fix but you may have
2:58:12several constraints to impose and in
2:58:15this way you will get out Expressions
2:58:18that are some sort of generalized uh
2:58:21Gibbs ensembles and now these are also
2:58:23becoming uh very popular
2:58:26in particular in context which are a
2:58:29little bit different of uh of quantum
2:58:31mechanics and in the rebel systems so if
2:58:33you are interested I can give you some
2:58:35references
2:58:37about about this and the last comment to
2:58:42connect with statistical physics is
2:58:45related to the point three of this
2:58:47exercise
2:58:48which I leave you as an exercise so out
2:58:51of this formula you can show that you
2:58:54can recover some relationships which are
2:58:56very well known in in equilibrium
2:58:59statistical physics so for
2:59:00boltzmann-like measures which are known
2:59:03as fluctuation dissipation relationships
2:59:06so this is just algebra but it is nice
2:59:09to see the connection
2:59:12okay and now I just wanted to uh
2:59:17to conclude with the last part so I hope
2:59:20this uh this is clear
2:59:23and the last part is to connect a little
2:59:26bit between
2:59:27maximum entropy and uh and maximum
2:59:30likelihood and this is the point two
2:59:35of this exercise
2:59:37[Music]
2:59:38and what we do in this point two is to
2:59:42assume that now we are given so now we
2:59:45go back to the context of estimation so
2:59:48of Maximum likelihood and we assume that
2:59:51somebody gives us
2:59:53our distribution that is of the form
2:59:55above
2:59:57so derived in this way but with
3:00:01parameters which are
3:00:03unknown and that we want now to estimate
3:00:06Based on data
3:00:08so in this case the shape of the
3:00:10distribution is and then you will write
3:00:11it
3:00:12I have a normalization which depends on
3:00:15all of the other constraints so I
3:00:18indicate this dependence explicitly and
3:00:20then I have some
3:00:22K going from one to capital K of my mu k
3:00:27GK of x
3:00:30very good
3:00:31and now I want to use this type of
3:00:34distribution in my maximum likelihood
3:00:37framework and see what comes out as an
3:00:40estimate for this
3:00:41values of new k
3:00:44and this is a simple exercise in here so
3:00:47I assume that I have no prior
3:00:48information on this parameters in UK so
3:00:51what do I have to do I have to compute
3:00:53my log likelihood assuming that I have a
3:00:57sample and my log likelihood as we saw
3:01:00before is the sum
3:01:03overall of my data of the logarithm of
3:01:07this distribution computed and the
3:01:09particular value of x which I find in
3:01:11the sample
3:01:12and so this will be I have a first term
3:01:15coming from here
3:01:20which is new k g k of x i
3:01:25and then I have the constant term so I
3:01:27collect the factor of small n and I have
3:01:30mu zero
3:01:32of n
3:01:35and now to minimize this with respect to
3:01:39uh I have to minimize this with respect
3:01:42to the MU k and so the equations that I
3:01:46get
3:01:47the estimate for my new K is
3:01:51what
3:01:54okay so I will get
3:01:57from here
3:02:01okay let me write it implicitly actually
3:02:05so if I derive this what I obtain is n
3:02:09times the sample average of my GK
3:02:14now I'm deriving with respect to one
3:02:16particular value of K
3:02:19so this will select only one element of
3:02:22this sum and I'm left with the second
3:02:24sum over I
3:02:25and on the right hand side I have
3:02:28remember that this normalization depends
3:02:30on all of my parameters in UK so I have
3:02:33in here a derivative of mu zero
3:02:38of mu with respect to
3:02:41bmu k
3:02:43and this I have to impose it
3:02:47equal to zero so you see
3:02:50that this gives me the following
3:02:52equation so I cancel a factor of N and I
3:02:55bring one to the other side and I get
3:02:58these equations
3:03:00that is very very similar
3:03:03to the equation so this is now an
3:03:05implicit equation for my values of mu
3:03:09uh that will be fixed with maximum
3:03:11likelihood and the expression is very
3:03:13similar to this type of equation which
3:03:16fixes the values of your parameters with
3:03:20a maximum entropy except that now your
3:03:23left hand side is not some number that
3:03:25is given to you a priori but it is some
3:03:28number that you get out of the data so
3:03:30it is now the average of your function G
3:03:33with respect to
3:03:35um to the sample of data that you have
3:03:40okay so this is uh to make a connection
3:03:42uh between the two things of uh of today
3:03:47and I wanted to now conclude uh with a
3:03:52couple of comments
3:03:55okay so first of all why bab expression
3:03:57are the same well because you know that
3:03:59you can rewrite
3:04:01this object in here as the derivative
3:04:04of the log of Z with respect to Mu K you
3:04:08can show this equation so this will be
3:04:10equal to the derivative
3:04:12with respect to Mu K of what we call log
3:04:16of Z but log of Z is what is log of e to
3:04:19the minus mu zero so here you will get a
3:04:21factor
3:04:22minus mu zero that is precisely what we
3:04:26have uh on the right hand side
3:04:30okay
3:04:31so
3:04:33that's all for the exercise so let me
3:04:36now go to some final comments and then
3:04:39maybe we can also comment on the third
3:04:41exercise
3:04:42very briefly
3:05:01and this final comment is to give a
3:05:05little bit of perspective and connect
3:05:07with some of the things that will be
3:05:09discussed at the end of the course
3:05:18and it is about
3:05:20correlations so if you remember
3:05:23what we assumed and stressed at the
3:05:26beginning is that we always assume that
3:05:29the data that we have are obtained
3:05:31sampling in an independent way from a
3:05:34given underlying distribution so now we
3:05:37can ask
3:05:39what happens when instead we have
3:05:43data
3:05:47that are extracted from a process which
3:05:51has some correlations
3:05:56so what is the setting in this case so
3:05:59in this case we can assume that we have
3:06:01not just one random variable X
3:06:04but we have more random variables so let
3:06:06me collect them into a vector
3:06:10X so this will be
3:06:13now I label them with a superscript and
3:06:17let's say that we have I don't know
3:06:20m
3:06:22of these random variables
3:06:24[Music]
3:06:25and our sample will be given by
3:06:28realization of all of these vectors so I
3:06:32will have a set
3:06:33of numerical values
3:06:36for each of these entry of my vector and
3:06:40the different values are labeled by I
3:06:43and I have
3:06:44a small n of those
3:06:46okay so now I'm sampling n times each
3:06:50entry of the vector
3:06:52but I am assuming that some of these
3:06:55variables which make the vector X are
3:06:57correlated and in particular I assume
3:07:00that there are pairwise
3:07:04correlations for example
3:07:09between these variables
3:07:13and then I can try to repeat this scheme
3:07:16with maximum entropy this recipe to find
3:07:20the shape for the distribution of these
3:07:22objects and I will tell you what is the
3:07:24result that you should get and just
3:07:27comment on this
3:07:28so first of all we have to introduce
3:07:32as before and average
3:07:34over the sample but now we are we have
3:07:38capital M averages corresponding to each
3:07:40entry
3:07:42so what is this this is one over n some
3:07:45I going from 1 to n of
3:07:47all of my realizations with the variable
3:07:51x side coming from data
3:07:53and we can also introduce correlations
3:07:55between
3:07:58pairs of the components of these vectors
3:08:01computed on the sample
3:08:05and this will be as you expect
3:08:08the same sum as a move
3:08:15[Music]
3:08:16okay
3:08:17and if
3:08:19a kind of reasoning that we gave before
3:08:22for the simplest case
3:08:24what you get is that in this case
3:08:27the distribution given by maximum
3:08:31entropy which is now a distribution a
3:08:33joint distribution of all of the entries
3:08:35of your vector takes the following form
3:08:39so you have some normalization factor
3:08:41which is the same as e to the minus mu
3:08:44zero
3:08:45and then you can write it as sum over
3:08:49Alpha which goes from 1 to capital M
3:08:52of some multipliers that now I call H
3:08:55Alpha just to
3:08:57be reminiscent of some Physics notation
3:09:02so so far if you have just one value of
3:09:06M is precisely what you would get as a
3:09:09bow if you choose just one value for the
3:09:12function G that is a linear function so
3:09:14if you just know what is the average of
3:09:17X you will get a distribution that is of
3:09:20the form e to the MU times x following
3:09:24the scheme above but here since we also
3:09:26have correlations we have a second term
3:09:29which is the sum
3:09:31over
3:09:33all pairs
3:09:38xaxp with some other LaGrange
3:09:41multipliers which depend on both indices
3:09:43and so which I denote as j a b and how
3:09:48do we fix H and J well following the
3:09:51same idea as what you will find
3:09:54is that
3:09:58you have to use the average of the
3:10:01sample and you will have an implicit
3:10:04relationships
3:10:06relationship between your sample average
3:10:09and the shape of this distribution
3:10:12of the following form
3:10:15but now you integrate overall values of
3:10:17x
3:10:18and similarly
3:10:21your sample correlations
3:10:23[Music]
3:10:26will give you an equation
3:10:29implicit equation for the J
3:10:35that looks like this
3:10:38okay
3:10:42and I just wanted to comment
3:10:45that what you end up with in in this
3:10:48type of framework is
3:10:51some inverse using problem
3:10:54[Music]
3:10:59s
3:11:01well I call it using but you know this
3:11:04uplinks can also take arbitrary signs
3:11:09and so on but as you see if this is the
3:11:12shape essentially of the partition
3:11:13function of of a missing model with with
3:11:17some local Fields if I interpret each
3:11:20values of X as as an entry or as a spin
3:11:24if you want and the problem that you
3:11:26have to solve is is not the usual
3:11:28problem so in the usual problem you are
3:11:30given the couplings of your Remington
3:11:32and then you want to compute
3:11:33correlations out of these couplings you
3:11:36want to compute the partition function
3:11:37you want to compute the average energy
3:11:39and so on and so forth whereas in here
3:11:41we are given the shape of our
3:11:44distribution we don't know these
3:11:46parameters here we know some of the
3:11:49correlations from the data and what we
3:11:51have to do is to solve for the
3:11:53parameters given the correlations so in
3:11:56this sense it is an inverse easing
3:11:58because
3:11:59as I said uh you you do not know the
3:12:03couplings of your Remington and you want
3:12:05to infer them based on the statistics of
3:12:07the data that you have and this is a
3:12:09problem which comes out in many many
3:12:11settings so you will find it in
3:12:13statistics in biology and maybe if you
3:12:16follow the course of of
3:12:18um
3:12:20uh you you will also see this appearing
3:12:24in there several times
3:12:27and and perhaps let me add uh the last
3:12:30comment to connect with the last day
3:12:33about correlations so at a certain point
3:12:35in the last day I asked you so there was
3:12:38a discussion about real language versus
3:12:40random language and we commented on the
3:12:44fact that our model for random language
3:12:47it was giving us a slip flow
3:12:51which we could compute explicitly
3:12:54but somehow there were assumptions that
3:12:56were not very realistic and in
3:12:58particular one of the assumptions was
3:13:00that any sequence of letters uh was an
3:13:05acceptable word uh in in the random
3:13:08language case so the total number I
3:13:11don't know if you remember it but we say
3:13:12that this was a number of
3:13:14words
3:13:16of length
3:13:17of a given length L was just given by m
3:13:21which was the number of letters to the
3:13:23power L which means that we accept so we
3:13:26have just to count how many combinations
3:13:28how many strings of length L we add and
3:13:33we accept all of those as as admissible
3:13:36words in real language in a random
3:13:39language but of course in real language
3:13:41this is not the case so we know that a k
3:13:44l is not a word
3:13:46so there are some rules of selections of
3:13:49words and you can think about uh this
3:13:53rule has been encoded into correlations
3:13:56between uh between the different letters
3:13:58so this is just uh to to stress that in
3:14:02any context that you are thinking about
3:14:05and in any realistic models correlations
3:14:08will matter and and problems of this
3:14:11sort will appear anytime you try to
3:14:14model them based on on real data
3:14:19okay so I think it's it's time to stop
3:14:22so the third exercise is it goes back a
3:14:24little a little bit to things that were
3:14:27discussed in the previous lectures uh
3:14:29about freezing and power laws and
3:14:33estimating the maximum over a set of
3:14:36variables which are power law
3:14:37distributed so there are some nice ideas
3:14:40in there but I think you can look at
3:14:43them with the solutions and then if
3:14:44there are issues write on the question
3:14:47and answer file or we can discuss them
3:14:49uh during the next day
3:14:52so are there any questions about this
3:14:56otherwise
3:14:58I will uh
3:15:01just say something to conclude for those
3:15:04who are still online
3:15:07um so we have been said that there is
3:15:10the possibility
3:15:11to have some people in class during the
3:15:14lectures and during the today but the
3:15:17number of people that can stay in the
3:15:18room is only two
3:15:21and therefore so I think this might be a
3:15:24good thing for those of you who want to
3:15:26move a little bit and come to ens but we
3:15:30have to decide who wants to do this and
3:15:32if there are more than two people we
3:15:33have to somehow do coordination to to
3:15:36decide who comes each week so if you are
3:15:39interested and you would like to uh to
3:15:41come physically just send me an email
3:15:42and we will try to organize this for the
3:15:45next weeks
3:15:47okay any
3:15:49more questions or comments was it fine
3:15:53I can stop
3:16:01the recording

About this transcript

This page contains the full transcript of Complex Systems - Jean-Philippe Bouchaud - Lecture 3: Multiplicative Growth (II). Redistribution by EconophysiX Lab, generated from the public captions YouTube serves with the video. The transcript has 23,292 words across 3,978 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to translate the transcript into 100+ languages.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, translation to 100+ languages, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.