YT YouTube2Text

YouTube2Text → Transcripts → PANEL DISCUSSION: Orchestrating agentic AI: Control, scale, and enterprise value

PANEL DISCUSSION: Orchestrating agentic AI: Control, scale, and enterprise value — Transcript

by World Summit AI · 7,240 words · 1,037 segments · language en · Watch on YouTube

Open in transcriber Summarize with AI Download SRT/VTT

Full transcript

0:00All right, everyone. Moving right into
0:03our next panel conversation that we have
0:06coming up here. We're asking this
0:08question about whether you have an AI
0:11agent or a set of agents. We we want to
0:14get to that point, right, where we have
0:15a whole team. They're all working in
0:17tandem with each other. But the thing
0:20that we need to realize when that's the
0:22case is are these agents all very nicely
0:26working one with the next? Do they need
0:29a manager? In real life, if we had a
0:31whole team of people working together,
0:32we'd have a manager above them, right?
0:34But for us, we're going to need today to
0:37look at this topic from the sense of
0:41many different industries. So, we have
0:43the next panel really amazing. We have a
0:45whole different set of industry leaders
0:47for you who are going to come talk about
0:48this topic. We're going to throw them up
0:50on screen right away here. Our wonderful
0:53moderator, David Nash, is going to come
0:55to the stage. And unfortunately team, if
0:58we could have all their names here, it
1:00would let me give you all their perfect
1:01job titles and everything, but otherwise
1:03we might have to have our wonderful
1:05moderator come on up and tell us all of
1:08the names of the speakers that he has on
1:10his iPad right in front of him. Here we
1:12are. Or on the stage behind me, David
1:14Savage, come on up.
1:16>> Welcome to stage from Harvey Nash. Here
1:18he is.
1:20>> Will you help introduce all of this?
1:21>> I think if we get them on stage and we
1:23get them up here, do it more. All right.
1:25Welcome panelist.
1:26>> Absolutely.
1:27>> Thank you.
1:27>> Please give my panelist a good hand.
1:29>> Agentic AI.
1:31>> Happy to have you.
1:32>> Everyone finding their picture.
1:35>> Margo, thank you for the um
1:37introduction. I think we know what we're
1:39talking about here. Maybe it's slightly
1:41more in hope rather than expectation. I
1:44suppose we'll probably get into that as
1:45we go, especially as we try and deploy
1:47agents at at production and at scale. I
1:51think as a way of introduction whilst
1:53your names and your job titles are on
1:54screen. Hi, do you want to tell us a
1:56little bit about who you are first of
1:58all and maybe as we come down the line
1:59just one very quick kind of thought
2:02about why or what the biggest barrier
2:04towards adopting agents at scale might
2:06be.
2:07>> Yeah, sure. Uh my name is Hiker. I'm the
2:10chief data and AI officer at ABN Amro,
2:13one of the largest Dutch banks. Uh and
2:16I'm really driving the AI journey within
2:18our organization. And so that's really
2:20orchestrating whatever we do. So we see
2:22it as an companywide transformation. So
2:25it's not in my court. I'm I'm I'm more
2:27the orchestrator making sure that
2:28everyone plays his role.
2:30>> Um and for me the the part on agents is
2:33really how can we scale it? So I think
2:35that's a common theme here. But it's how
2:37do we get from well AI to geni? How do
2:40we scale it and how do we eventually
2:42with agents in particular how do we
2:44control it and how do in particular one
2:46regulated market it is really important
2:48to to have that oversight and making
2:50sure that the agents are doing what they
2:52are supposed to do but I think we will
2:53touch based on that one there
2:54>> I imagine scale might get mentioned a
2:56few times [laughter]
2:58>> sorry Kim
2:59>> sure hello my name is Kim I'm a field
3:00CTO of castai the company behind Kimchi
3:03and the whole AI journey was born as an
3:07pain out of our main product line casti
3:10is well known for Kubernetes
3:11optimization. So we built this product
3:13by using entropic heavily and surprised
3:16what's happened our our bill was
3:18exploding. So we looked into this issue
3:21and check out what can we change there
3:23how does open weight models could help
3:25us and at the end how to orchestrate how
3:27to curate and also how to run these
3:29environments and these agents in a
3:31secure way. So Kimchi is the outcome of
3:33this. So happy to be in this panel and
3:35speak about this.
3:36>> Perfect. Why?
3:38>> Yeah. So I'm Wayman Freddy. Um little
3:41bit weird name combination Chinese and
3:44Italian. Um so I lead AI architecture
3:48and innovation for IHG. Hopefully all of
3:53us here stayed in IHG property. Now if
3:57you ask me what is the most uh
4:00challenging thing for us um I think you
4:04know just something just changed uh in
4:07the past few weeks. It is really about
4:11how do you control the agents outside of
4:14your boundary your enterprise boundary
4:17and we're talking about those personal
4:21agents. Yeah,
4:22>> right. Muse, instinct, dots, right? They
4:26are totally outside of our enterprise,
4:29but they are representing our customer
4:32visiting our website and committing
4:34transactions.
4:36>> I'm sure that we will get into that
4:38shortly. K, let me ask you something
4:40slightly different. Introduce yourself.
4:42Absolutely. But instead of the same
4:43question that we've asked as we've gone
4:44down as a quick way of introduction,
4:46what do you think most seues get wrong,
4:48seuite executives get wrong about
4:50Agentic AI? Yeah, it's a good question.
4:52I'll probably start with the way I was
4:56wrong. I'll
4:57>> introduce yourself first and then I was
5:00the chief digital officer of Pernicard.
5:02You might know better the brands like
5:04Absolute Vodka or the Irish Whiskey
5:06Jameson.
5:07>> People might get familiar tonight
5:08>> probably.
5:09>> Yeah. Yeah.
5:10>> And um so for six years I was in charge
5:13of deploying uh AI at scale. So what we
5:17focused initially on was the biggest
5:20population in the in the company, the
5:22sales reps. And so uh we developed and
5:24we deployed in more than 28 countries a
5:27solution uh agents we would call them
5:30today that would give the sales rep a
5:32recommendation every Monday morning of
5:34what stores, what supermarkets to visit
5:37this week and each of the store or the
5:39the bars uh saying okay this is the
5:43action you should be taking. for
5:44example, promoting that brand or that
5:47other one.
5:48>> It did pretty well uh because with AB
5:50testing, we could measure a few millions
5:53of euros of uh of benefits. Uh however,
5:56what I overlooked uh is the fact that
6:00it's a large population making a lot of
6:02small decisions, but actually the
6:04biggest decisions in the company made at
6:07the executive committee in the boards.
6:10we're still using uh Excel and
6:12PowerPoint and it's still the case today
6:15in most companies I've been discussing
6:16too. So one of the thing for me which is
6:19key today is to look at uh the highest
6:23stakes the highest level of decisions uh
6:26whether you're going to invest on this
6:28brand or that brand how you allocate
6:30millions of euros between Turkey, China
6:32or France or Netherlands. And this is
6:35for me where there's a huge potential of
6:38value because everybody's looking for
6:39value from AI. It's not that easy to uh
6:42to find. And so that's the the thing uh
6:45for me which is overlooked the same way
6:46I overlooked it five years ago when we
6:48started on AI journey.
6:50>> Mhm. Okay. Well, look, let's dive into
6:52some of these topics. We we'll start I
6:53suppose by talking about what's changing
6:54a little bit here and travel given that
6:57many people will have traveled and
6:59booked hotels to be here for this
7:01conference. Well, let's let's talk about
7:02that because before we sat down for this
7:06panel, you were talking about the fact
7:07that you we we are seeing a change in
7:10the control of terms.
7:11>> Yeah.
7:12>> And as a way of of context, you know, in
7:15your introduction, you talked about
7:16personal agents booking travel. So, what
7:20does that mean for the relationship that
7:21you have with the customer if an agent
7:23is beginning to control that process?
7:25>> Exactly. So before I get that I I really
7:29want to check you know how many of you
7:33played with Muse instinct or dots those
7:38personal agents. Okay few of you this is
7:41just happened in last month and this why
7:45I felt like the whole control landscape
7:47changed. So you you you probably heard
7:50about that matter
7:53Facebook released a personal agent
7:56called Muse within nine day days nine
7:59days
8:01around 4 million download and people
8:04start to use it right and then then
8:08immediately instinct similar thing
8:11personal agent just sweeping right now
8:14in US everyone start to use it and then
8:18And of course followed by OpenAI
8:20thoughts, right? So majority of those
8:23personal agents actually what they do
8:26negotiate some of the bill, right? I
8:29personally probably saved $200 already,
8:32right? Just let them deal, you know,
8:35negotiate those deal for me. And the
8:38majority of them actually is using for
8:40the travel planning and the booking. So
8:43in in our case right like within the
8:47past few weeks we seen closer to a
8:50billion visits to IHG's
8:54digital property of those bots. So
8:57number one thing we didn't even know
9:00where it come from. Yes we know it's a
9:02bots but are they legit? Actually they
9:05are legit right? They represents our
9:08customer not only search and explore but
9:11they also committed transaction in terms
9:16of booking right with the legit payment
9:19methods. So that is really a new
9:22challenge for us is those things are out
9:26of control. One is we barely know who
9:28they are. Second is they are crawling
9:31our website. Do we have control over
9:34what should be crowded, what's not, not
9:37mentioning the information they
9:40scrapping from our website.
9:43We don't even know if they get a full
9:45picture.
9:46>> Okay. So, normally when when there's a
9:48new technology, there's a fairly slow
9:49adoption curve. You know, people showing
9:51their hands here. Not many people have
9:52yet utilized these tools, but just about
9:54everybody use platforms that's the
9:56platforms that they are built on.
9:58>> Exactly. Right. How how does your
10:00business respond to that? If you
10:01suddenly have this huge surgeact
10:03message,
10:04>> do you do you have the
10:06>> option to slow down and say, "Hang on,
10:08pause." Or realistically, in a
10:10competitive market environment, you just
10:11have to accept that things are going to
10:12go wrong, right?
10:13>> Totally. Totally. Right. So, Amazon is
10:16the company they block those spots.
10:20That's the only company. But the rest of
10:22the retail if you're consumerf facing
10:25because they bring the real customer
10:27transaction to our property. So we have
10:30to meet where customer they are right.
10:34If they are leverage and delegate that
10:36bots the agent to book the hotels we
10:40have to be there. So within 24 hours we
10:43have to put our terms together start to
10:46talk to the agent provider. Right. One
10:49is you need to tell us what's the
10:52pattern of your traffic. Second is we
10:55need to get agreement on what's the best
10:57way to share our information to our
11:00guest. Y
11:01>> right. Third one is what action we can
11:05agreed on you can take right and the
11:07fourth one is also the measurements.
11:11So those are the terms I'm just saying
11:14even like we suddenly happen we don't
11:16have a control but you have to be on top
11:19of those terms. So I suppose Kim look
11:22listening to that if if we accept that
11:24organizations are going from a handful
11:25of agents to multiple agents whether
11:27that's coming from personal usage or
11:28even the agents that you're building in
11:29your own environment what breaks first
11:32you know if we're talking about scale if
11:33we're talking about cost what what are
11:35the what are the early red flags? So
11:36what breaks what breaks first is um I
11:40would say mostly it's
11:43the monitoring part but can something
11:46break what doesn't exist because most
11:48people don't have visibility what they
11:49spend they [clears throat] don't don't
11:50visibility about how much agents do I
11:52have what are these agents doing how
11:54much do I spend for
11:57>> oh I'm breaking [laughter]
12:00>> we might have to start that again in
12:02case anyone missed yeah [laughter]
12:04>> okay so first of all what will break
12:06visibility, but what can it really break
12:09if it doesn't exist? Cuz people don't
12:11know what is the spend, how much agents
12:14they have,
12:15who's responsible for these agents. So,
12:18um, we are just flying in the dark.
12:21Nobody knows. Um, and then what breaks
12:24mostly reliability. How are we doing
12:26tests? So, these project starts in a
12:28lab. Lab people, they are just
12:30interested to be successful. So, we are
12:32testing a bit with agents. we see oh
12:34there's accuracy of 95% good enough good
12:38enough ship it to production and uh
12:40because 95 sounds good but in production
12:44in the reality then we see okay it's not
12:45just one agent the result of an agent is
12:48moved over to a next agent or next agent
12:51so we see instead of having just one
12:52agent um we have maybe three so the
12:55accuracy is going down to 86% and if I
12:58have 20 I'm talking about 30 36%
13:03and 36% it's nothing. Do you want to
13:06ship something? Would you trust a guy
13:08who tells you ah I'm 36% sure that this
13:11will be the right solution for you? I
13:13would not do this and I think this is
13:16how a lot of people starting this gamble
13:19and at the end for sure what breaks last
13:22or let's see it's not breaking maybe the
13:24heart breaks of the CFO at the end of
13:26the months if the feels like being at
13:28the Oscars putting out the the bill of
13:30entropic from this envelope and having
13:32all these pills and alcohols and I don't
13:35sure what else uh close to them cuz we
13:37know the story from this uh Indian
13:39company which received a $500 million
13:42bill for Just one month of entropic
13:44usage,
13:46$500 million. Really? What are these
13:49guys doing? So
13:50>> hopefully they cured cancer for that
13:51amount of money.
13:52>> I hope so. I hope they are still alive.
13:54And I'm pretty sure someone got fired
13:55after. So um and we all want to protect
13:58our jobs. So um don't don't adopt just
14:03what you have in labs to your
14:04production. Think about it. What does it
14:06really mean? And have the right
14:07guardrails in place. Having avoid agents
14:11in loops. avoid wild west everybody can
14:13do what they want you need to protect
14:15this you need to control this you need
14:16to have visibility otherwise it will be
14:19a nightmare
14:20>> I'd imagine hike in your environment 95%
14:23would not be good enough
14:24>> correct [snorts]
14:26that's a good guess yeah I think the the
14:29the advantage and disadvantage of being
14:31in a regulated market as banking we are
14:34is that we can go with a little bit
14:36slower pace so we we do see clients that
14:38having agents and are already connecting
14:41through us with agents. We can have our
14:43agents connecting to our clients. So
14:45that is let's say technically possible
14:47but we are really really careful about
14:49it.
14:49>> Are there areas where you do accept that
14:52something might break first where you
14:53can accommodate some element of failure?
14:56>> A very simple example is making
14:57appointments. Uh that's something that
14:59we do that with an agent. Yeah, it's a
15:01very safe one but we want to be very
15:03careful. But the way we we try to
15:05control it is by separating separating
15:08out the different type of agents. So we
15:10have an conversational agent
15:12>> internal external that is really doing
15:14all the conversations with whoever there
15:16is. Then we have a reasoning agent that
15:18is the agent that is doing the real
15:20thinking about what needs to be done.
15:22They're also checking what are the the
15:23policies the standards around it because
15:25an agent is always pleasing. So if you
15:27have the conversational agent agent if
15:30you ask the right questions you can
15:31bypass some of the the things but
15:33because they have to hand it over to an
15:35reasoning agent there we really see what
15:38whether the right things are being asked
15:40whether it's within the policies within
15:41the guard wheels that were mentioned
15:43>> and then the third level of agents are
15:45the task agents and they're really
15:47performing the the particular task that
15:48they get from an reasoning agents. So we
15:51do that with an MC with MCP servers.
15:54making sure that the the context is also
15:56properly in place. So that's the way we
15:59uh overall control the the agents uh
16:01side.
16:02>> So you talked about the agents handing
16:03off to the agents. Who defines when they
16:05hand off to the human?
16:08>> Well,
16:09foremost the agent the human is always
16:12responsible for a task. Uh and and we
16:14need to have explanability uh towards
16:16our customers. So in particular, let's
16:18say in a mortgage process, we will never
16:20say the computer says no. There's always
16:22a human in the loop. uh to to check the
16:24final decision that has been made. So if
16:26it is a really more impactful uh uh
16:30choice for our customers, there's always
16:33a human in the loop.
16:34>> Sure.
16:34>> By definition, uh and overall, if
16:37there's no human in the loop, the human
16:38is still responsible for Of course,
16:40sorry, this is slightly off script, but
16:41I've heard people talk about the need
16:42for a chief virtual people officer,
16:44someone who defines that actual
16:46relationship when you dial an agent up,
16:47when you dial an agent down, someone
16:49with the organization who has that
16:51oversight when it comes to that that
16:53structure. Do you have someone in the
16:54bank who who is overseeing not just who
16:57has responsibility, but how much of
16:59that?
17:00>> Not yet. But we we are working hard on
17:03the topic of what does the human
17:05oversight actually look like. uh for
17:06instance we are using a lot of uh
17:08software development with with agents
17:10that's that's a very easy easy one uh
17:13but a very powerful use case because
17:15every step in the software development
17:16life cycle from setting your
17:18requirements writing your solution
17:19intent writing the code doing the
17:21testing blah blah blah the whole shebang
17:23can be done by by agents where every
17:26step is done by uh controlled by a human
17:31the task of a human becomes quite boring
17:33after a while because you have to well
17:35check the code But if the code is let's
17:37say 95% good and you only find 5% and
17:41that's your day job well soon you will
17:43not be checking the code anymore. So we
17:45are thinking about okay what are the
17:47alternatives to make sure that the code
17:49is checked properly so the human
17:50oversight is done but also effective
17:53>> because you can say yeah there was human
17:54oversight but if if it is just a
17:56stamping machine
17:57>> then uh you you don't have an effective
17:59control anymore and that's the
18:00discussion that we have like what is it
18:02really mean to have human oversight and
18:05and I I say the jury is not out yet
18:08>> that's that's a fair point right because
18:09a lot of people say that that this will
18:11enable there to be a lot more
18:12productivity hours in our staff
18:15because they're going to have more time
18:16because the agent's going to take XY Z
18:18off their desk. But in reality, a lot of
18:20people are talking about it's just
18:21shifting work and the work that it is
18:23shifting is frankly less interesting
18:24work. To your point, they're marking the
18:26agents homework.
18:27>> Well, and and the fact that is less
18:29interesting to me is well, not the most
18:31important thing, but what's more
18:32important, how do they make sure that
18:34that you still do your job with the
18:35right level of due diligence? And that's
18:37the my biggest concern because if 95% is
18:40right, you're only fi finding 5% of
18:42errors in the code or whatever you look
18:45at and that's becomes well okay uh and
18:49becomes better over time. So at the end
18:51you don't control it anymore. At least
18:52that's my conviction that in in a few
18:54years from now there won't be any human
18:56oversight on code.
18:58>> But still you need to have control on
19:00code. And so how do you do that? That's
19:02the the puzzle that we are having. But
19:04it's a broader question about what is
19:06human oversight and how should how can
19:08we still make it really effective.
19:10>> Yeah. Perry. Sorry. Go on.
19:12>> Do you want me to try? So go first. We
19:14have a similar practice, right? So one
19:16each agent will have a unique ID, right?
19:19Universal ID, unique ID in our system.
19:22So we can trace end to end. Second is
19:26agent also has owner, human owner. Just
19:29like a business system has a business
19:31owner, we have a real human's name
19:33attached to that.
19:34>> So something goes wrong goes to that
19:36person.
19:37>> That's interesting because I was going
19:38to ask Perry kind of if if if the if the
19:41workflow is shifting, who's then
19:42responsible for it? It seems that you
19:44have an answer there. I mean, is that
19:46something that you've seen replicated in
19:47other businesses?
19:48>> Yeah. So for me the accountability uh
19:51for what comes out of an AI
19:53recommendation or an automation uh it's
19:56not an issue when there is one optimal
19:59easy solution so I'm going from point A
20:01to B there is one optimal route I mean
20:04no no issue the issue comes when there
20:08is not one uh answer so and when you
20:11cannot have maybe a human in the loop
20:14automated cars they are in a critical
20:16situation they have to decide to crash
20:19on the A side or the B side having some
20:22different consequences.
20:24Here there is a necessity uh to make
20:27trade-offs. So I took a critical example
20:30of the selfdriving car but it's the same
20:33in in companies. Uh you want at some
20:36stage some automations to be uh decided.
20:39So for example do I taken my previous
20:42example the salesforce to go for more
20:44sales more margins. Do you want my
20:47supply chain to go for optimization of
20:49cost, optimization of CO2 footprint?
20:52>> And so what's important in all those
20:54cases is those trade-offs.
20:58How do I balance the different uh
21:00decisions and for me this is one of the
21:02key uh addition that we need to make to
21:05artificial intelligence. I call it uh
21:08tradeoff intelligence where uh you need
21:11people sometime at the highest level of
21:14the company to decide on the trade-offs.
21:17So do I accept to spend 5% more uh to
21:21optimize my CO2 footprint by 20 30%. Um
21:25and you want also uh all the decisions
21:28which are made or recommended by agents
21:31uh to follow those uh those parts. So
21:34for me uh balancing
21:36sometimes the creativity of AI uh the we
21:40call it also inaccuracy sometime uh with
21:43some control but also having that
21:45trade-off intelligence which is added uh
21:48to the agents and also helping u high uh
21:52stake uh decisions uh people in
21:55executive committees in boards to make
21:57those tradeoffs um in an easier way. For
22:01me it's key uh and a key addition to
22:04artificial intelligence today.
22:06>> Let's just quickly kind of talk a little
22:07bit I suppose about guard rails and
22:09who's building because where you believe
22:12that we should let experts build but
22:14when we were chatting before we came
22:16over to the tent you said that actually
22:18a lot of the real business savings that
22:21an organization like yours sees is from
22:23personalized
22:24agents rather than the big enterprise
22:27agents.
22:28>> That's right. So tell me why it is that
22:30experts should build if actually a lot
22:32of the savings are coming from those
22:33personalized agents especially when
22:35maybe you're not seeing the return on
22:36investment from from the ones that your
22:38your teams are building.
22:40>> So one is you know we all know right um
22:43I'm pretty sure everyone sitting here we
22:45know only a few areas that clearly can
22:49demonstrate the return of the AI
22:51investment. Majority is still hard to
22:54measure. We all know that but actually
22:56in real life you know you see when a
22:59business side of we call the domain
23:03experts right for example they are the
23:05one every month they have to close the
23:08account right month end closing or
23:11procurement right um procure to pay
23:14those people know where the big pain
23:17points they know the process they know
23:20what's the ultimate solution should be
23:23right given time the powerful AI tools
23:27right assisted by engineers often land
23:30in lots of productivity gain and
23:34savings. So that's what I've seen so far
23:36is you know actually is the power users
23:40from business side assist by engineers
23:44actually produce quite a measurable
23:47results.
23:48>> Is that quite expensive though? you know
23:51experimentation is expensive
23:53>> actually it's not it is not right so
23:56think about that that normally software
23:59uh projects we all know budgeting is all
24:01in millions right but those are like you
24:05know to me is like mostly token cost
24:08right engineer we have in-house engineer
24:11already the work they have to get done
24:14by business so to me it's like a lowest
24:17cost but then you see the result is very
24:21Yeah, I didn't want to talk about the
24:22cost. It's more about uh the trade-offs
24:25that we are making between letting
24:27general productivity tools uh having
24:30within the whole organization. On one
24:32hand, we are talking about AI
24:33citizenship, AI democratization and
24:36making sure that we enable the whole
24:37organization to work with it. On the
24:39other hand, we really have to control
24:41side of it because my biggest nightmare
24:43is that with end user compute which we
24:46saw with excels and and all those kinds
24:48of things that it will be amplified with
24:51AI. So it is it's the balance that we
24:54are looking for and on the other hand
24:56what we are doing is really having a
24:59couple of big bets where we really think
25:01this is where the organization will
25:03benefit the most. So on one hand it's
25:06about yeah incremental changes and and
25:08seeing how you can AI your processes but
25:11on the other hand it's also about having
25:12a couple of big bets and say okay this
25:14is really transformational green field
25:16starting from how should the process
25:18look like knowing what we can do and
25:20then it it's not trying to squeeze in in
25:22in the current process but re reimagine
25:25your new processes and that is something
25:27that needs to come from top down because
25:29bottom up it will be very hard
25:30>> I was going to say how do you define
25:31those big bets who who signs off on
25:32those big bets
25:33>> it's the executive board. So I'm I'm
25:36working on them as the responsible for
25:39AI. So I'm working with the organization
25:40to figure out where do we see the value
25:42pools, where do we see also the market
25:44trends, where we expect that the the the
25:46big changes will be. That is a
25:48discussion in our executive board and in
25:50the in the end there is a a choice made
25:53by them.
25:54>> Mhm. And then you have a top- down
25:56choice that is cascading down in the
25:57organization to make sure that you also
25:59build then around those transformational
26:01areas a team around it with the business
26:03itself of course uh but also with the
26:05group innovation is there HR is involved
26:08uh technology data uh so everyone is
26:11there a multi-disiplinary team because
26:13that's the only way in my opinion to
26:15really get that transformation in there
26:17because the businesses for the business
26:18it's very hard to reimagine their their
26:20world because they lived
26:23>> the way they did it for the last 20
26:24years so to say
26:25>> innovation has ideas but they don't know
26:27the business enough well etc etc so you
26:30need that common uh well
26:32multi-disiplinary team in order to run
26:34it
26:34>> Kim this feels like a good jumping off
26:35point to return to your 500 million
26:37right if this is if this is the world in
26:39which we inhabit how does the CFO make
26:42sure the costs do not just explode
26:44>> exactly because also I want to jump on
26:46this because I have the question if you
26:48say uh the board is doing this big bet
26:50at the end how do you measure this
26:52because I see a big problem in how we
26:55measure if AI really helps us and if we
26:58if we really save something or get any
27:01value out of it because we we have a lot
27:03of discussions about people talking
27:04about um price per token. The price per
27:06token says nothing. I can have a I can
27:09uh I can route a task into a cheap
27:11model. Uh if the quality is the same
27:13great I'm saving something. If the model
27:15generates garbage it doesn't help me
27:17anything.
27:18>> And I think this is the problem where we
27:20all are right now. How do how are you
27:23measuring this right now?
27:25>> Okay. Uh good question. Well, last year
27:28my target was really about getting a
27:30number of use cases into production. So
27:32it was like well we did care a little
27:34bit about the business case but not so
27:35much. We we kept the tokens under
27:37control. But now we made the pivot more
27:39to okay where do we really see the
27:41value. So every use case that we get
27:43live has a business case and the
27:45business case can be about cost
27:47reduction, it can be on growth, it can
27:48be an MPS, it can be on capital
27:50allocation or on risk
27:53mitigation. So it's on those five
27:54metrics that we say every uh use case
27:57has to fulfill in order to get funding
28:00for it. Um we work together with the
28:02CFOs, the controllers to make sure that
28:04we also measure it backwards. So that's
28:07an that's a challenge I must say and for
28:10the bigger bets uh it's it's what we
28:13expect. We also have a business case
28:14around it. Um but because of the whole
28:17environment is changing it's sometimes
28:18hard to really pinpoint is this really
28:21an AI thing or is it something else that
28:22we are doing so it becomes a little bit
28:24blurry but we do measure it and it is
28:27one of the the metric that we have but
28:29not only on cost reduction and that's
28:30important to me. We have five metrics
28:32that we see what is the value that it
28:34brings.
28:35>> Yeah that's perfect. Well, what I say we
28:37have we are all all the companies are in
28:38the same problem right now. We need to
28:40identify how to measure things. What is
28:42the right metric? Also, we started all
28:44with token cost. Token cost at the end.
28:46Now, we realize that token cost says
28:47nothing. We need to measure success at
28:50the end. How does we how can we define
28:52success in coding? It's easier. I can I
28:54have a task. The task is done. I can
28:56route them into different models. And
28:57then I see okay, is the quality the
28:59same? Which model at the end was the
29:02best? Best not does not mean just the
29:05cheapest one. It's also timing is
29:07interesting. A model the one flow takes
29:09a half a half hour the other just five
29:12minutes. So um I think a lot of these
29:14challenges are out there. Um the
29:16industry
29:18has some let's call it solutions out.
29:21There are some best practices created
29:23right now. We are we are dipping our
29:26feet into this right now in the field.
29:27We are figuring out what works what
29:29works not. at the end also it's related
29:30to the company's
29:33>> vision structure and so on
29:35>> but maybe on the cost side I mean I just
29:37said on the on the impact side we
29:39measure on the five five things but on
29:41the cost side what we do is of course we
29:43have our tokconomics we have our what we
29:46call the AI gateway to make sure that we
29:48really have optionality on what model to
29:50run for what use case but also do we use
29:53ventropic do we use open AI or do we
29:55well we have a partnership with Mistral
29:57run it on our own GPUs
29:58>> we don't have long left. You mentioned
30:00solutions. Perry, just very quickly,
30:02you've been in this position where at
30:04Penoricad you can work across a number
30:05of different market brands. If we were
30:07to give people insight on what they
30:09should be doing today to try and
30:11implement solutions and successful
30:12strategies, what would you point people
30:14in the room towards? Yeah. So, for me,
30:16what's important uh the technology is
30:20complex, but it's not the most complex
30:23part. The most complex part is to get
30:24adoption within a company about new ways
30:27of working, new ways of making decisions
30:30etc. So for me um the the important part
30:34is to measure uh the the value and for
30:36me AB testing has been for hundreds of
30:40year the way medical companies are
30:42measuring whether a drug is successful
30:44or not. So having a group using not
30:47using is one of the the things. So
30:49value. The second thing which is
30:50important is adoption. Um so uh the the
30:54project we did for the salesforce uh in
30:57uh in pero record 10% of the people
31:00adopting the recommendation would mean
31:0210% of the value generated. We aimed at
31:0585% how did we get there? By making sure
31:09that the job got more interesting.
31:13>> So having people in sales team uh they
31:16don't like to do go through excels to
31:18find out the best outlet to uh to visit.
31:21they like to have the the personal
31:22contact etc. So make sure that they
31:24spend more time on this. At the
31:26executive level, a lot of executives are
31:28sitting to really boring board
31:31presentations where uh they're seeing
31:33slides after slides and then the good uh
31:36the bad and the ugly scenario and hope
31:40being hoped to choose the good one but
31:42super frustrated on this. So on this
31:45example you need to give solutions where
31:48people can play with the the scenarios
31:50where can simulate they can press the
31:53the button please give me the optimal if
31:55I'm getting this or that and so um if
31:59you are doing that on top on the highest
32:01stakes what I was mentioning at the at
32:03the beginning here you are generating
32:05value
32:07engagement and at the end of the time
32:09the return on investment everybody's
32:11looking for and so this is what I'm
32:12doing in my new So I'm just picturing
32:14I'm packaging all of this in a solution
32:18that we call a paradigma with a Y to
32:21help um board uh executive committee to
32:25make better decisions to be involved uh
32:27in the decision by taking just the data
32:29that they need. So not expensive token
32:32spend and I think this is where there is
32:34a lot of value which is overlooked the
32:36same way I was overlooking it when I was
32:38in per record. And so there are here
32:40probably a few millions of yours to to
32:43find every you know those decisions in
32:46boards where uh there's the question uh
32:49what about this and people say okay I'll
32:51come back two weeks later with the
32:52answer two weeks later there's other
32:54things to discuss. So sometime decisions
32:57are delayed by months which are worth
32:59millions of euros. So each week you are
33:02leaving on the table 100k, 200k, 1
33:05million euro and this is what they want
33:07to solve in companies. I I it was put to
33:10me last week that though that uh a lot
33:12of people are quite lazy. Humans can be
33:14quite lazy. So hopefully when you ask
33:16them what they would like to see, well
33:17how they would like it to augment their
33:18work, they said something slightly more
33:19optimistic. And uh anyway, look, we've
33:21only got a few minutes left for um some
33:23Q&A and you've probably got much better
33:24questions than I have as your experts in
33:26the room. Has anyone got a question for
33:27our panel? There's some hands here.
33:29There's there's a hand down the middle.
33:30Make it easy for the mic runner.
33:35>> Thank you. Yeah, sorry. On on the end.
33:38>> Hello, I'm Winnie from uh Stock. I'm the
33:41head of data and egg and um our company
33:43sell all the baby high chairs and
33:45strollers. Our iconic one is street
33:47trap. So in my department currently I'm
33:49having a lot of use cases coming through
33:50and then I think on the stage the
33:52speakers are talking about all these
33:53measurement bit. I'm interested to know
33:56what is your filtering when you receive
33:58all these different use cases coming to
34:00you. What is your filter and how you
34:03going to say I'm going to prioritize
34:05this you prioritize that and um coming
34:09to my second question is then how you
34:12decide what to build yourself and also
34:14how to where you going to buy it from
34:16someone else or you're going to
34:18collaborate or partner with someone
34:19else.
34:20Yeah, maybe I can take the the
34:22prioritization because I've done it for
34:24six years now. Uh well, I was doing it.
34:27Um first is not easy to know what is
34:30complex to do and not complex to do
34:32because an idea which is coming from a
34:34person working in marketing in supply
34:36chain etc. might be a really complex
34:39thing or something that can developed in
34:41two or three days. So that's one of the
34:44key criteria and you need a filtering by
34:46some experts that say okay this is a
34:48multi-user uh multissources of
34:50information that will make update in a
34:52critical system that needs accuracy.
34:54That's a complex part. I just need to uh
34:58create a draft for an answer to an RFP.
35:01That's a pretty easy one. But you need
35:04some expert to to help. So uh on the
35:06prioritization of ideas first you need
35:08to help the people who have ideas to be
35:10able to get a little more flesh on them
35:13so that somebody can say it's a complex
35:15not complex you need people from finance
35:18to be the guard raids in the future
35:20making sure that the company will invest
35:22in the right uh topics.
35:24>> Yep. Um and then it's a usual uh
35:27consultant prioritization. What is the
35:29value versus the cost but with something
35:31which is has some flesh on it so that
35:33you can really prioritize at the board
35:35level for example but with a
35:37pre-qualification before. So I would
35:39recommend to have all the ideas that
35:42come and really good ideas coming
35:43company to to come together but people
35:46to help uh putting flesh and uh
35:49identifying value versus uh cost or
35:52complexity uh to to get on
35:54prioritization.
35:54>> Yeah maybe to add one thing to it what
35:56what we try to build is really reusable
35:59components. So if you have an particular
36:01use case, you you you first think that's
36:03the feasibility part. What are the
36:04elements that you already have in place
36:06or how can you click it together as as
36:08Lego blocks uh in order to make it
36:10easier to roll it out. So
36:12>> yeah, but I I I want to point out a
36:14thing. This is some debate I always get
36:17into with our leadership, right? We're
36:20going through something so major we
36:23don't even know what's going to happen,
36:25right? often Silicon Valley said, you
36:28know, [clears throat] you probably are
36:30facing 10x or 100x opportunity right
36:33now, but you're not looking at it. If
36:36you're using your common senses, common
36:40logic, common intelligence, if the whole
36:43room full of people nodding their hands
36:45is saying that makes sense, maybe you
36:48are missing out a giant 10x opportunity.
36:52>> But that's another category. I'm not
36:54saying go that direction, right? But
36:57keep that in mind as well.
36:58>> We've got 90 seconds left, so we've
36:59probably got time for one more quick
37:01question.
37:02>> Yeah, sorry that the mic. Sorry,
37:03apologies. Mike's here. Yeah.
37:05>> Okay. Hi. Uh, I guess my question is
37:07primarily going to Mr. Fanders because I
37:10also work in a private bank as a data
37:12scientist. So, in my mind, there are
37:14like three sets of possible use cases
37:16for AI in finance. One, the problems
37:18that AI agents can solve. two uh
37:21problems worth solving because they're
37:23going to generate extra uh cost
37:25reduction or revenue and the third is uh
37:28use cases where we can trust AI agents
37:30to act and I feel like it's really hard
37:32to find the overlap of these three sets.
37:35What is your approach to solving this
37:36problem and uh what did you end up with
37:39as examples for a banking sector and AI
37:42agent application? Thank you.
37:43>> This is going to be hard to answer in a
37:45minute.
37:45>> Yeah, exactly. So the the three things
37:47that you're mentioning is uh
37:50you mentioned three things.
37:52>> Yeah. Uh AI can solve it. Uh it's worth
37:55to solve it and we can trust them to
37:58carry out the process of auto. Well, the
38:00the the the the f the the first thing
38:01that we see is that there are a lot of
38:03uh administrative and and manual
38:05processes still going on in the bank.
38:07And in particular, when when I look at
38:09wealth management, uh a relationship
38:11manager is is taking quite a lot of his
38:13time to prepare a an conversation with
38:16their clients because the information is
38:18in different ones. Their their risk
38:20profile is somewhere else. They have to
38:21bring it together. Maybe an a tool defi
38:24defin defines hey this is your well
38:27suggested investment but still you need
38:29to have a narrative around it so the
38:31investor has to make it themselves well
38:33that whole part is all administrative it
38:35can be taken over for 50 60% by AI so
38:38it's solving a problem because then you
38:40can as a relationship manager really
38:42focus on the the the client relationship
38:45where the human is is important and the
38:47rest of the administrative task you can
38:50leave it to agents
38:51>> you know it's it there answer all the
38:53questions here.
38:54>> Are you all sticking around at the
38:55conference for the next two days?
38:57>> Hope you'll have an opportunity to to
38:59talk to people. Look, thank you so much
39:00for your time today. Please give a hand
39:02to our panel.
39:04[applause]

About this transcript

This page contains the full transcript of PANEL DISCUSSION: Orchestrating agentic AI: Control, scale, and enterprise value by World Summit AI, generated from the public captions YouTube serves with the video. The transcript has 7,240 words across 1,037 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to translate the transcript into 100+ languages.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, translation to 100+ languages, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.