YT YouTube2Text

YouTube2Text → Transcripts → Lecture 01 : Network Data - Some Stories !!

Lecture 01 : Network Data - Some Stories !! — Transcript

by NPTEL IIT Kharagpur · 4,119 words · 682 segments · language en · Watch on YouTube

Open in transcriber Summarize with AI Download SRT/VTT

Full transcript

0:01[Music]
0:22[Music]
0:25welcome to the first lecture of
0:27artificial intelligence for economics
0:30in in this segment in this lecture and
0:32the and the the few lectures coming up
0:35uh I will try to introduce to you uh a
0:39few different topics which will get you
0:41warmed up hopefully so in the first
0:44lecture what I have planned is I will
0:47try to give you a few
0:48examples uh of data which we see all
0:51around
0:52us uh from from uh politics to
0:57finance especially network data that's
1:00what we'll deal with in this particular
1:03lecture and we'll try to see how we can
1:06interpret that and what stories they
1:09reveal let's move
1:11on
1:13first let's start with history let's
1:16start with marriage alliances we have we
1:18we know that in history uh marriage
1:21alliances have been a very common
1:24strategy when it came to uh forming
1:27political uh political
1:31marriage was a key key um tool for for
1:35political alliances or a marriage
1:38between royal families was a very common
1:42um
1:44um
1:46occurrence uh and and it played a it
1:48played an important role when it came to
1:50power
1:52sharing so let's roll back the clock and
1:55let's go back to
1:56Florence in fact 14th century 15th Cent
2:01Florence well these were the most
2:03influential families of Florence back
2:06then now and this is the this is the
2:08Florentine marriage Network so consider
2:12any two families let's say the the the
2:15uh salviati and the medicis so an edge
2:18existing between them means that one
2:21member of the salviati family has been
2:23married to somebody in the medic family
2:26so if two families are connected via
2:28marriage then they exists an edge
2:30between them in this network where the
2:33families are represented as vertices or
2:37nodes now can we take a look at this
2:40network and guess something about the
2:44power structure of
2:46Florence or the power distribution of
2:48Florence can we guess which family or
2:51which families were ruling Florence or
2:55where the most powerful in
2:57Florence okay
3:00before we do
3:02that by the way it turns out that it was
3:05the
3:06medes okay I don't know you can pause uh
3:10whether you can guess whether it's the
3:12medes uh by merely looking at the
3:16network maybe you
3:20can but why the medicis is there a
3:22mathematical Foundation which tells us
3:25by looking at the network by
3:26interpreting the network that the medes
3:27will turn out to be the most important
3:29family
3:30most important
3:32family uh before we get into the math
3:34let's get into the history first you can
3:37watch this Netflix show if you wish
3:40uh a a few words on the medicis the
3:43medic Dynasty or family they went to
3:47banking they it uh it was founded by
3:50kosimo medic u in the late 14 Century or
3:55early early 15th
3:57century the the the medes became
3:59extremely popular they they almost
4:02occupied many of the important positions
4:05in the
4:06assembly to the extent that Katherine uh
4:10became the queen of France in
4:131547 okay so they they were that
4:16powerful the medes uh also played an
4:20important role in patronizing um all
4:23these renaissa artists Michelangelo
4:26Rafael uh Leonardo DaVinci
4:32now let's formalize let's try to
4:35understand let's try to look at the
4:37network uh history tells us that yes the
4:40medes were uh the most powerful but can
4:43we simply look at the
4:45network
4:47and mathematically infer that the medes
4:50will medes are
4:52powerful can we make the history uh and
4:56the math coincide
5:00let's try to see let's try to
5:02formally understand if we can do so
5:05let's
5:07formalize before we get back to the
5:09Florentine marriage network uh let me
5:11Define a few things and then we'll get
5:13back consider this network a very simple
5:18one a few
5:20definitions in a network two nodes I and
5:23J are called Neighbors if there exists
5:25an edge between them for example 1 and
5:28three are neighbors four and five are
5:31neighbors nodes I and J are connected if
5:34there exists a path between I and J not
5:36necessarily neighbors for example 1 and
5:39four are not neighbors but one and four
5:41are connected because there exists a
5:43path between 1 and
5:46four the shortest path between node I
5:49and J is the shortest root or the number
5:52of hops so the it's the minimum number
5:54of hops required to read J from
5:58I degree of a node is the number of
6:01neighbors a node has for example the
6:03degree of four will be two the degree of
6:05five will be four so on and so forth
6:08sorry degree of five is three degree of
6:11four is two degree of three is three
6:14again
6:15okay
6:18great now that we know this let's define
6:22a particular metric called betweenness
6:24centrality so now we'll Define two two
6:27measures or two metrics if you may call
6:29them
6:30which uh in a way depicts the importance
6:35of any particular node in a
6:37network okay so what is betweenness
6:40centrality let's
6:42understand so if I have two nodes I and
6:45J any two nodes I Define p i j as the
6:48number of shortest paths between I and
6:50J okay number of shortest paths for
6:54example between 1 and 4 what is the
6:56shortest path it's 1 3 4 but I have
6:59another path 1 2 3 4 but 1 3 4 happens
7:04to be the shortest path so there is only
7:05one shortest path between 1 and 4 in
7:07this
7:09case p k i j is let's say the number of
7:14times a node K lies in the shortest path
7:17connecting
7:19in okay for example in the shortest path
7:23connecting 1 and four there is only one
7:26shortest path and three appears in that
7:28path so P3 31 4 is going to be
7:321 so
7:34P3 so if you if you look at three here
7:38so
7:41P 14 will be 1 and P
7:47314 will also be one
7:50right
7:53anyway between this centrality of a note
7:55K is the number of times K features in
7:57the shortest Parts between any two nodes
8:00in the
8:01network okay so this tells you that how
8:05many times if any two nodes have to
8:07connect to each other how many times
8:09they'll have to connect via VIA
8:12K okay so between N means between I and
8:16J how many between how many i's and J's
8:19K
8:21features okay let's formally Define this
8:24now uh let's have a uh let's define this
8:29set SK this is the set of all pairs of
8:33connected
8:37nodes okay set of all pairs of connected
8:40nodes other than
8:43K so what is SK uh so SK is pairs of all
8:47connected nodes other than K betweenness
8:50centrality is defined in the following
8:53manner I take PK that is how many times
8:56K
8:57features U in the shortest paths between
9:01I and J for all I belonging to
9:07SK and I divide PK by Pig so the
9:12numerator PK by P is the number of times
9:15K features in the shortest path divided
9:17by the total number of shortest paths
9:20okay and that divided by the cardinality
9:23of s so this is the betweenness
9:26centrality of K
9:34okay that's the betweenness centrality
9:37of
9:40K
9:43great let's compute the between the
9:45centrality of the different nodes in
9:48this particular
9:50Network let's consider 1 and three are 1
9:54and three connected yes they
9:57are what about p41 1 3 I want to I want
10:01to compute the between the centrality of
10:03node 4 now let's say so I'm I'm I will
10:07take all pairs of nodes other than four
10:10so that is my set
10:12SK so I take 1 and three 1 and three are
10:15connected so p13 is
10:181 uh and not only connected how many
10:21shortest parts are there between 1 and
10:22three it's only one because they're
10:24directly connected there exists an
10:27edge so uh p13 is is 1 does four feature
10:31in that shortest path answer is no so
10:33p413
10:35is0 what about 1 and two one and two are
10:38neighbors again so p12 is 1 so the
10:42number of shortest Parts is one because
10:43they're direct neighbors what about p412
10:47does four feature in the shortest path
10:48between 1 and two answer is no
10:50absolutely
10:51not similarly I can find out for all
10:55other
10:56pairs p uh so I can find out
11:00p
11:02j4 and p j for all other nodes i
11:07j i not equal to J not equal to 4 that's
11:11what I've done in this slide and once I
11:14do that I can find the sumission which
11:17is this which is what we have seen in a
11:19few slides before
11:22this okay and if we compute that we get
11:26that the betweenness centrality of 4 is
11:289 by
11:3115 we can proceed similarly for three it
11:35turns out that the betweenness
11:36centrality of three is 8x
11:4015 and proceeding for all of them it
11:43turns out that the betweenness
11:46centrality of four is the highest so in
11:48this in this network four happens to be
11:52the most important Network by the by
11:55byit if we consider between the
11:57centralities
12:00followed by the between the centralities
12:01of 3 and 5 followed by 6 7 1 and
12:062
12:10okay now let's introduce another uh
12:13measure of
12:15importance in in a network another
12:18measure which which depicts the
12:19importance of a particular node in a
12:22network it's called The cads Prestige so
12:25let's understand what is cads prestige
12:28the power of a no node comes from
12:30connecting to a powerful node and the
12:33powerful node derives uh its power from
12:35connecting to other powerful nodes and
12:36so on and so
12:38forth so it
12:40means let's say I have a I have a node
12:43I
12:45okay and ni I is the set of all
12:47neighbors of
12:51I so what is what is uh The Prestige of
12:55node I what is the cat's Prestige of
12:58node I or or player I or family I
13:02whatever you might call it the catch
13:04Prestige of this vertex I is given
13:07by the cat's Prestige of its neighbors
13:09divided by their
13:11degrees so I I scan through the set of
13:15all neighbors of
13:17I compute their see what their Prestige
13:20is divided by their degree and add them
13:23up why divided by a degree what's the
13:26rational for that so let's say if you
13:28and I are connected and you are
13:29extremely
13:30powerful and if you're also connected to
13:33other people then your influence gets
13:36dissipated amongst others so the so the
13:39fraction or share of the power which I
13:42derive by being associated by by with
13:45you uh gets diminished it is inversely
13:48proportional to the number of other
13:50associates youve
13:52got
13:53okay great so this is cat's prestige
13:59so for this
14:01network so yeah this is what I was
14:03mentioning The Prestige of noi depends
14:05on both The Prestige of its neighbors
14:07and the degree of the neighbors as well
14:08the time and resources that the
14:10prestigious node can share to an
14:11individual node reduces with increase in
14:14its
14:16degree okay let's now let's try to
14:18compute the cat's prestiges of this
14:21particular
14:22Network let's see let's first uh
14:25normalize let's say uh Prestige of one
14:29is one let's start with that then what
14:32is the cat's Prestige of
14:33two well it is two has two neighbors
14:36right 1 and
14:40three so the cat's Prestige of two is
14:42going to be cat's Prestige of one
14:44divided by the degree of one degree of 1
14:46is
14:462 plus cat's Prestige of three divided
14:50by the degree of Three Degree of three
14:52is 1 2 three so this is
14:56three what about cat's Prestige of three
15:00three has three neighbors 1 2 and four
15:04so cat's Prestige of three is prestige
15:07of 1 divided by the degree of one which
15:09is 2 plus The Prestige of two divided by
15:13the degree of two which is again two
15:16plus the degree of four uh Prestige of
15:18four divided by degree of four degree of
15:20four is again
15:22two similarly we can write down the
15:24equations for all the cats
15:27prestiges now we have a system of
15:29equations we have seven equations and
15:31seven unknowns we can solve them right
15:35let's solve
15:36them and this turns out to be the cat's
15:39prestiges of the different
15:41known okay
15:45but here we uh initially considered we
15:48had to cons we have a if we have a
15:50system of equations with uh no
15:55particular uh initialization that will
15:57yield to nothing right then we'll have
15:59infinitely many
16:01solutions but here the initialization
16:03was P cats equal to P1 equal to 1 and
16:06then we wrote down our uh system of
16:09equations for cat's Prestige and found
16:11it but we could have started with P2
16:13equal to 1 and then we will get another
16:15Vector of cats prestiges P3 equal to 1
16:18we'll get another Vector of cats
16:21prestiges right so we should remove this
16:24uh initialization
16:26sensitivity so just to do that we we we
16:28we follow the following
16:30algorithm okay so we first initialize
16:34one compute all the cats prestiges then
16:37initialize to compute all the cats
16:39prestiges and finally we take all
16:41average of all those
16:44prestiges okay great so we have we have
16:47we have U
16:49learned two important measures two
16:52important metrics which uh signify the
16:57importance of a particular node in a
17:00network what were they betweenness
17:02centrality and cats
17:07Prestige so let's roll back to
17:09Florence and uh let's understand what is
17:14the betweenness centrality let's go
17:16let's delve deep into the Florentine
17:18marriage Network and understand what
17:21will what is the betweenness centrality
17:23or cats Prestige of the different
17:27families it turn out that the cat's
17:30Prestige of the medic family is 47.5 and
17:33that's the
17:36highest uh sorry the betweenness
17:39centrality the cats Prestige is also
17:42highest for the medic family
17:45again okay so the Florentine marriage
17:49network if we look into the Florentine
17:50marriage Network and compute between the
17:53centrality and Cat's Prestige it turns
17:56out that they give us a picture that the
18:00medic family is the most powerful family
18:03in this network and it also turns out in
18:06history that the medicines were indeed
18:08the most
18:09powerful uh in in in
18:12Florence so we see how a little uh
18:16interpretation of the network gives us a
18:19peak into the
18:24history
18:26great so so much so for an example from
18:28history
18:30now a little bit of
18:32Finance
18:34quickly so this is from a paper by
18:37demirer debal and Le and
18:41gmas so let's connect it this is about
18:43connectedness of financial
18:46institutions so what are they doing I
18:49won't get into the technical
18:52details so let's understand so the study
18:55basically takes in 96 Banks okay okay
18:59and uh these are all chosen from the
19:04world's top 150 banks by
19:07assets uh 82 from uh 82 are from uh
19:11developed
19:12economies and the remaining four 14 are
19:15from emerging
19:19markets and all all of these Banks which
19:22are which are chosen are globally
19:24systematically important Banks gsbs
19:30first let's define something called
19:32volatility or uh or or let's call it
19:36volatility of a
19:38bank so total volatility of a bank they
19:42have defined it in this manner this is
19:45the definition which has been
19:47used let's say Sigma it square is the
19:50volatility of bank I at time
19:53t Okay time T is period T day T let's
19:57say it is simply on day T the V the
20:01volatility of bank I volatility means
20:05how much it's fluctuating it's simply
20:07given
20:08by this complicated expression where hit
20:13is the highest stock price lit is the
20:16lowest stock price CIT is the closing
20:18stock price and oit is the opening stock
20:22price of Bank I in
20:24DT simple I observe on day t or on time
20:30period T I observe the highest lowest
20:32closing and opening stock prices of
20:35Bankai and then I apply this complicated
20:38equation or expression and we get the
20:42volatility of bank
20:44I in time period
20:48T now we use something called variance
20:52decomposition we try to see how the
20:55return volatility of bank I is
20:57influenced by other Banks
21:00J okay so the total volatility which is
21:03Sigma I well technically it's Sigma I
21:06squ this can be decomposed into this
21:10Theta i
21:12j so what is Theta i j Theta i j is or
21:17as we'll Define it very soon Theta i j
21:20is the effect
21:22of effect
21:27of Bank
21:32J on
21:34the volatility of bank
21:39I okay and we can when we talk about
21:43this effect this effect could be an
21:45immediate
21:46effect or it could be a into the future
21:50effect for example I can talk about
21:53Theta i j h
21:59which basically tells you that this is
22:03the effect which bank J has on the
22:06volatility of bank I eight step
22:09forward that is what happens in Bank J
22:12how will it affect the volatility of
22:14bank I age periods from now AG periods
22:18from
22:19now great now we can do this by using
22:23something called uh V which is Vector
22:25Auto regression and a lasso regression
22:27technique the this is what the this is
22:29what the authors of the paper have done
22:31but I'm not getting into the technical
22:33details because this is an introductory
22:34lecture I just want to get you or give
22:38you a glimpse of what's going
22:42on
22:44great so let's see so this is firm J's
22:48uh
22:49contribution to firm I's e Step Ahead
22:53variance this is given by
22:56Theta okay
23:00great and G is the
23:02network G is the network of banks but
23:05let's forget about networks
23:08now so what is CH
23:14hji well this
23:16is uh Theta I ided summation of theta i
23:21j so this is basically the
23:25proportion of the total volatility
23:30which uh proportion of the total
23:31volatility of
23:33I which comes about due to the effect of
23:36J so is the proportion of I repeat once
23:39more this is the proportion of the total
23:41volatility of I which is being brought
23:44about by Pang
23:47J okay and by construction of course so
23:51that that is CJ
23:52j2i C J2 I and we we are taking it for E
23:57Step Ahead okay this age could be
23:59anything 10 day ahead 3 day ahead
24:01whatever whatever your uh interest of
24:03the empirical study
24:06is so clearly this summation will by
24:09construction this summation will be
24:11equal to 1 and this summation is going
24:12to be n if you sum this
24:15up over all all all JS it will simply be
24:19one as you can
24:24imagine now we Define something called
24:26total directional connectedness what is
24:29this so if you consider any firm I or
24:32any bank
24:34I the total directional connectedness
24:38is uh the total amount of effect other
24:43banks are
24:44having on bank
24:47I on an
24:50average okay so let's say there are uh n
24:54Banks
24:56cji cji is the effect J has an i and if
25:00you sum it over all JS J not equal to I
25:03so that is the effect all other banks
25:06are having on bank I or firm
25:09I okay so C8 star I is the total
25:14directional
25:15connectedness to firm I from all other
25:20firms similarly the total directional
25:23connectedness from firi c h I2 star
25:28is C i2j summed over all JS so this is
25:33the total effect which I has on all
25:37other banks on the volatility of all
25:38other banks on an
25:41average okay
25:44great what is systemwide
25:46connectedness well systemwide
25:48connectedness is every every bank or
25:52every firm has a total directional
25:56connectedness to that firm
26:00the systemwide connectedness is the
26:03average of those total directional
26:05connectedness to the
26:09firms okay in this case I'm Computing CH
26:13so I've taken H so H is the E Step Ahead
26:16systemwide
26:17connectedness okay remember H could be
26:21anything H could be 1 2 whatever your uh
26:24interest
26:27is in in this study now uh the authors
26:30have proceeded with h equal to 10 so if
26:33we look at H equal to 10 and if we do
26:36the uh if we look at the volatility
26:39connects this is what we
26:41see well this seems like a very
26:44complicated Network we can't seem to
26:47make a head or tail of
26:49this but I think we can interpret the
26:51stable a little more carefully now look
26:53at this so what story comes out from
26:57this
26:59let's look at Africa
27:02first Africa is getting influenced by
27:05who the most well 45 is the total
27:08influence let's say then it's the total
27:11volatility of all African Banks so I'm
27:14aggregating over continents
27:16now so if this is the total volatility
27:19of the African Banks it is coming from
27:22where the total directional
27:25connectedness or the total influence
27:28to Africa is being brought about by
27:31Europe and North America the
27:33most okay what about
27:36Asia Asia again is getting influenced by
27:39who again Europe and America the most
27:43the remaining influences are not much
27:44see 30 21 4
27:48nothing what about Europe Europe is
27:51getting affected by who the most North
27:53America the most 581 is the total
27:57volatility 43 of that so that's a huge
28:00proportion is coming from North America
28:02similarly when it comes to North America
28:04who is influencing North America the
28:05most
28:07Europe
28:09okay so it seems North America and
28:11European banks are uh ruling the
28:15world they are impacting the volatility
28:18of banks all over the
28:19world what about Asian
28:22Banks well the total volatility uh or
28:25the total input into Asia is 481
28:29418 that's the amount of total
28:31directional connectedness into Asia to
28:35Asia what about the total direct
28:37directional connectedness from Asia it
28:41is
28:43214 214 see so which means Asia is
28:48getting influenced
28:51more rather than influencing more right
28:55that's the picture
28:56we uh Loosely
28:59get so two two key takeaways from this
29:03from this study we see that in this
29:06network network of Banks North America
29:08and Europe are large and they are
29:11transmitters of future volatility
29:13uncertainty to the rest of the
29:15world okay so all everybody's connected
29:19the banks are connected to each other
29:21North America and European Banks affect
29:24the volatility of other Banks globally
29:26to a much larger extent
29:29and they also form a clustered amongst
29:32themselves they are also connected to
29:34each other massively
29:35remember uh Europe is affected by North
29:39America massively and Europe is uh North
29:42America is affected by Europe
29:44massively so they are extremely
29:46interdependent and they also have
29:48tremendous volatility spillovers
29:50globally in other
29:52zones Asia on the other hand has
29:55noticeably large total directional
29:57connectedness
29:59into okay more into than from Asia so
30:03it's a net receiver of volatility if if
30:07if you may think about it that
30:09way okay so this was a regional thing
30:13now let's look at let's look at time
30:17wise if we compute this uh 8 Step Ahead
30:21volatility
30:23spillovers the the total directional
30:26connectedness it turns out
30:28that in September 1 2008 that's before
30:31the Leman
30:32crisis this is how the connectedness uh
30:35scenario looks like whereas in this
30:38situation which is post November 21
30:41after Leman went
30:43bankrupt the connectedness is all the
30:46more which means that the banks were
30:49getting were failing together
30:52now they were impacting each other much
30:55more after the crisis than before the
30:57crisis
30:59in fact it turns out if you compute the
31:02systemwide connectedness of the bank
31:05volatilities remember systemwide
31:06connectedness this is what it is this is
31:09systemwide
31:11connectedness so if you compute
31:13systemwide connectedness of all the
31:15banks
31:16globally it turned out that the system
31:20white connectedness went
31:22up and it peaked during the lemon crisis
31:25and then again it started coming down
31:27which which means that when there is a
31:29global
31:30crisis the volatilities of the banks
31:33move together which makes the crisis all
31:37the more
31:39grave and that's what we saw during the
31:41Leman crisis when Leman went bankrupt uh
31:46every other bank there was a there was a
31:48spillover effect and every other bank
31:51started uh falling down and we slipped
31:55into uh a recession and and a complete
31:59economic
32:01meltdown thank you I think I have given
32:04you a a starter to look into Data uh
32:08first was interpreting a network of
32:12marriages from 15 Century
32:15Florence and the second one uh I talked
32:17a little bit about connectedness of
32:19financial institutions or
32:22Banks I hope this got you started in the
32:25next lecture I will talk about something
32:28completely different I will talk about
32:31um

About this transcript

This page contains the full transcript of Lecture 01 : Network Data - Some Stories !! by NPTEL IIT Kharagpur, generated from the public captions YouTube serves with the video. The transcript has 4,119 words across 682 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to translate the transcript into 100+ languages.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, translation to 100+ languages, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.