YT YouTube2Text

YouTube2Text → Transcripts → Plan and prepare to develop AI solutions on Azure | AI-103 | Episode 1

Plan and prepare to develop AI solutions on Azure | AI-103 | Episode 1 — Transcript

by Microsoft Learn · 4,575 words · 565 segments · language en · Watch on YouTube

Open in transcriber Summarize with AI Download SRT/VTT

Full transcript

0:00[ Music ]
0:05ROB FOULKROD: Imagine this, a brand-new project lands
0:08on your desk, ship an AI-powered customer support agent.
0:12The requirements are stacked, the timeline is tight,
0:14and expectations are high.
0:16It needs to understand voice when a customer calls in,
0:19it needs to search your policy doc
0:21so that answers are actually grounded in your business,
0:24and it needs to call your internal APIs to do real work,
0:27open a ticket, update an account.
0:29So where do you even start?
0:30Which Foundry resources fit the timeline
0:33without blowing the budget?
0:35How do you keep that agent grounded, governed,
0:37and safe every step of the way?
0:39And how do you ship it without handwaving all the hard parts?
0:43That's the reality most teams are walking into right now.
0:46And it's exactly what this course is built for.
0:48Welcome to "AI-103: Develop AI Apps and Agents on Azure."
0:56Hi, I'm Rob Foulkrod,
0:57a Lead Technical Trainer here at Microsoft.
1:00I've been a Microsoft Certified Trainer since '99,
1:02and I have spent my career helping professionals ship real
1:05solutions across developer tools, DevOps, data, and now AI.
1:11Across this course, I'm going to walk you
1:12through the same path I'd walk a customer
1:14through if they handed me this customer support agent project
1:17on a Monday morning.
1:19We'll move from the foundations all the way
1:20to production-ready agents on Microsoft Foundry.
1:24Here's how the course is laid out,
1:26four topics building on each other.
1:28In Topic 1, we'll learn the Foundry's platform,
1:31design chat experiences, call tools to form responses
1:34and ground answers in your own data,
1:37and apply responsible AI from day one.
1:41In Topic 2, we'll move from apps that respond to agents that act,
1:46building Foundry agents, extending the grounding
1:49with them with tools,
1:50and orchestrating multi-agent solutions.
1:55And Topic 3 will give those agents a voice and a vocabulary,
2:00the language and speech services, multimodal audio
2:03and chat, real-time conversations, voice live,
2:06and translation across languages.
2:09And, finally, apps that will reason over visual data,
2:14generate image and video content,
2:16understand multimedia documents with content understanding,
2:20and index everything with Azure AI Search.
2:23By the end of this course, that agent won't feel intimidating.
2:27So let's get started.
2:29Every AI agent you've ever interacted
2:31with starts the same way underneath,
2:32with a generative AI app calling a model, shaping a prompt,
2:37and grounding the answers in real data.
2:39So, before we get into agents that act, we need to be fluent
2:42in the apps that respond.
2:44That fluency is the runway for everything
2:46that follows in this course.
2:48Before you write a single line of code, the decisions you make
2:51in the planning phase shape everything,
2:54which platform you build on, which models you can reach,
2:56which tools your team uses,
2:58and how responsibly your solution behaves once it's
3:01in front of real users.
3:03Getting that foundation right is what makes the rest
3:05of the work easier.
3:07We'll start with what AI actually is in this context,
3:10and explore Microsoft Foundry
3:12and the Microsoft Foundry Tools you build with it.
3:15From there, we'll look at the developer tools
3:17and SDK available to you,
3:19and we will wrap responsible AI principles that'll guide
3:22everything in this project.
3:25In this session, we're going to ground ourselves with what AI is
3:28on this platform, we'll explore Microsoft Foundry
3:31and the Foundry Tools, and we'll walk you through the SDK
3:35in grounding our thinking in responsible AI.
3:40This course is going to be really focused
3:42on this idea of Microsoft Foundry.
3:45And so the first thing that we're going to have
3:46to do is start to look at what Foundry is
3:50and what the components are that we're going
3:53to find inside of Foundry.
3:56So Foundry itself is going to be -- as you can see right here,
3:59is this unified platform.
4:01It's a single spot where you can go to build the agents
4:05that you're going to need.
4:07Those agents are going to be reliant on a number
4:11of different models so we'll have an opportunity to preview
4:15and select a huge number of models.
4:20The model alone is the brain of the whole thing, but it's going
4:23to need arms and legs.
4:25And those arms and legs are going
4:27to come from various tools.
4:29And we've got a number of options available to us.
4:31We have kind of our tooling in terms of being able to reach
4:36out to external resources, we'll see things
4:38like MCP a little bit later, but there's also a collection
4:42of AI tools that we've had
4:45for quite some time that'll be available to us.
4:48Those will be in here as well.
4:49More on those in a minute.
4:52But the last thing, models have a finite amount of information,
4:56they've been trained on public information.
4:59And that public information has a cutoff
5:01of potentially a couple of years ago.
5:05So, if we need private information,
5:08we need your policies, we need your inventories,
5:12we need those tools.
5:13Those models are also going to need
5:15to have access to various knowledge.
5:19So those tools as well are going to be set up, configured,
5:23and accessible inside of your project.
5:26Right? So what we're going to build is a Foundry project.
5:30When we go to build the project, it will, beneath the covers,
5:34build the resource for us.
5:36So, in essence, the resource comes first,
5:38it's the landing place.
5:40This is where your network resources are,
5:44the compute resources.
5:45That's really the platform in which we will put our project.
5:50Then the project is going
5:52to contain potentially multiple models, multiple agents,
5:56multiple tools, and knowledge.
5:59Right? Now, before there was Foundry
6:04and before generative AI, there were a number
6:08of AI tools inside of Azure.
6:11These tools have gone by a number
6:13of different names over time.
6:16They have been called cognitive services,
6:18they have been called Azure AI services.
6:22And they're still relevant today.
6:24Right? Now, a number
6:26of generative AI solutions have maybe eaten away at some
6:31of the functionality that we saw in the original classification
6:34of tools, but there's still a number of tools
6:37that are still very valuable today.
6:39And we'll be looking at those tools throughout the course,
6:45primarily in Sections 3 and 4 of this course.
6:49But, here, we want to know where we can find them
6:51and where we can get our hands on them.
6:52The tools that we have available here are going to be things
6:55like the Azure Language in Foundry Tools.
6:59That's our natural language processing.
7:01This is things like, "Hey, can we detect the language
7:04that we're running on, can we find key terms or entities
7:10in that text, can we do things
7:13like lay the groundwork for translation?"
7:17We've also got capabilities in here for speech.
7:20Now, there's a number of generative AI solutions
7:24that have speech as a capability.
7:27However, picking that model is more complicated
7:31than just "I need speech."
7:34So there will be times
7:35where you're picking a tiny little model that's really good
7:38at one thing yet the agent still needs maybe to be able
7:43to understand speech or to respond in kind.
7:48So we've got these tools that we can kind of bolt on top
7:50of those additional models.
7:54So speech and translation are in there.
7:56The idea of document intelligence, to be able to read
7:59from PDFs and forms and invoices and receipts,
8:02and extract that information, and multi-model understanding.
8:07So we can look at those documents and say, "Okay,
8:10I know what's on that thing,
8:11I can describe what's on that document."
8:13These are all some of the Foundry tools
8:15that we'll be working with over the course.
8:20So let's walk into a quick demo
8:21so that we can get a feel for the environment.
8:26All right, so this is going to be a quick dry run to make sure
8:29that everything is going well.
8:30So, everyone, we are going to be taking a look at Foundry here.
8:33We'll look at it in a couple of different views.
8:36First, we'll take a look at the web UI
8:39where we will create a project and we'll create a model.
8:42Then we will get into what it looks like inside
8:46of Visual Studio Code, specifically the extensions
8:49that are inside of code, we won't write any code
8:51in this section, but we will get to that very, very soon.
8:56So, for starters, we are going to navigate to Foundry.
9:01Foundry, you can see right here,
9:02ai.azure.com is where we're going.
9:05There's a couple of different Foundry UIs right now.
9:09The nextgen here is the new UI
9:11that we're going to be looking to.
9:12You can see that little switch right there
9:14where you can turn it on or off.
9:17And we will first start by looking at creating a project.
9:20Now, I've got a few projects here already,
9:22but we will go ahead and --
9:24through the little Project Section right here, go ahead
9:27and create a brand-new project.
9:31And, from there, it'll give us a suggestion.
9:33It'll always be our username typically as a suggestion
9:36and some random number.
9:37My username right now is demo, so we'll grab that, and then,
9:43specifically, the resource.
9:45So we talked about earlier that we have a resource which is kind
9:49of the bucket that holds all of the components
9:53for a particular collection of applications, and then one
9:57or more projects inside of that resource.
10:00So this is the child inside of our resource.
10:06We'll specify where we want that to go.
10:07Right now, we will pick East US 2, which is fine.
10:12We will drop it into a brand-new resource group,
10:16and then we will go ahead and click "Create."
10:20That'll take just a couple of moments to build.
10:25And, a couple of minutes later, we now have a project.
10:28So let's go.
10:31It then should open up to our new project.
10:34You can see right here our demo-0687.
10:37We'll use this one today.
10:39In maybe later sections, we'll switch up.
10:42Next is what does this look
10:44like from inside of some of our tools.
10:47So I'm going to swing into Visual Studio Code here.
10:49I've got that open, kind of an empty project at this point.
10:52It is empty.
10:53Right? Yeah.
10:55And we're going to take a look
10:56at the extensions necessary for our app.
11:00We're going to swing into our extensions and we're going
11:02to look for our Foundry extensions.
11:06And you're going to see there's really two
11:07of them available to us.
11:10There is the AI Toolkit and the Foundry.
11:12As of right now, we're kind of in a little place of flux
11:14where we have the two of these.
11:16Later on, we will see probably just the one Foundry Toolkit.
11:20So I still install this one.
11:22It will install both for us.
11:25Only take a moment to do so.
11:30All right.
11:31And now that it's installed, we see the two of these down here.
11:35The first being -- the first
11:37at the bottom here being the Foundry and then the Toolkit
11:40where we're going to spend most of our time.
11:43All right, so there we've gone through kind
11:45of the demo that I need to see.
11:48Hope that was helpful.
11:49And let's wrap all this up in a sec.
11:54Now that we've seen our way around, let's take a look
11:57at the ways that we're going to be interacting with Foundry
12:02when we're not in the Foundry Portal itself.
12:05And so we've got a lot of tools, really, that are available.
12:09On this slide here, we're talking about one in particular.
12:12We'll be talking about Visual Studio Code.
12:14That'll be our primary vessel
12:16that we'll be using throughout the entire course.
12:18However, whatever your favorite IDE is,
12:23editor, they're all fine.
12:25We'll look at Visual Studio Code.
12:27We will look at the Toolkit Extension as a way
12:30of interacting with Foundry
12:32when we don't have the Foundry Portal open.
12:36And, of course, GitHub Copilot when we install the toolkit,
12:40there's a number of skills that are pre-built in there
12:44that are going to help you write your code faster.
12:47Okay. And then there's a matter of the SDK.
12:51You can interact with these models by way of REST API.
12:55Right? It's ubiquitous, you can use it anywhere.
12:59But REST is a little bit clunky for us on the developer side
13:03to be using on the regular.
13:04So, often, we will have that SDK layer
13:08that wraps over the top of it.
13:10So we have a couple of options,
13:12and we will explore what those options are in later sections.
13:16But, here, we just want to introduce that we do have things
13:19like the OpenAI SDK, which are typically used
13:22for talking directly to models.
13:24Right? When we want to start a chat with that model,
13:29get a response back, the OpenAI SDK is perfectly fine.
13:33Right? But, as we start to layer over the top of that,
13:38when we want to have not just a chat with a model,
13:44but have a full agentic platform, well, that's when --
13:48-- the Microsoft Foundry SDK comes in handy.
13:56It can shorten the development time when we need
14:00to start including things like additional tools
14:03and grounding can sometimes be a little bit easier
14:06with the Foundry SDK.
14:08So you can imagine that,
14:09if you're targeting the model directly,
14:13maybe the OpenAI SDK works, but, if you're working
14:16at a higher level, often we'll jump into the Foundry platform.
14:22And then we just talked about the other Foundry tools
14:27that were available to us, language
14:29and speech and translator.
14:30Each of those have their own SDK.
14:33Right? And so we can use those directly,
14:37or we'll see there are some tools that allow us to work
14:40with them from inside the agentic structure.
14:48And, finally, there is this idea of responsible AI.
14:55And there's six principles that Microsoft developed and refined
14:59over the years, the principles of fairness, reliability
15:03and safety, privacy and security, inclusiveness,
15:07transparency, and accountability.
15:09Each one of these are structural points that we need tools
15:14for all throughout the platform.
15:17So we will see throughout the course where we can plug
15:21in each of these components.
15:24Right? Whether we're looking at our choice of data grounding,
15:27whether we're looking at refining our prompts,
15:30whether we're looking at building guardrails or UX
15:33or ongoing operations, those are all components that we're going
15:36to be seeing throughout the course to reinforce these ideas.
15:45If you want to give a shot, there is an exercise in here
15:48to walk through the Foundry project to go ahead and build
15:51and test a generative AI model, and to make sure
15:54that we're comfortable with the various endpoints
15:57and working inside of Visual Studio Code.
15:59This one is a good one if you are brand new to Foundry
16:03and you want to get those tools ready
16:05for the rest of the course.
16:06If you've done this a little bit, you're going to do this
16:09for each section coming up, so it's an okay one
16:13if you're a little pressed for time to skip over.
16:17All right, folks, so let's explore Foundry a little bit.
16:20We're going to spend a lot of time there
16:22over the next few sessions.
16:24So I'm going to start by going to ai.azure.com.
16:30And ai.azure.com is going to be the portal
16:33that we're going to be in quite a bit.
16:35And, when we go there first, the first thing you're going
16:38to notice, and I want to make sure
16:39that you're really clear here
16:41because we've got a couple different portals,
16:44and so you can see here that I've got this little toggle
16:46up at the top for the New Foundry.
16:49And that's what we want to be in.
16:51We want to make sure that every time you're doing any
16:53of the examples, anything that we're seeing through the course,
16:57that you're in that New Foundry.
16:59Now, when I do this, I'm immediately going
17:01to be asked for a project.
17:04So, in the idea of this new portal,
17:08there's always a default project.
17:11And so it asks me now, "What's the default project?"
17:14And it shows me all the projects that I have,
17:17of which there are none at this point.
17:19So we're going to have to build a project.
17:21So I'll create here, "Create new project."
17:25I go to give it a name.
17:26This number could be whatever.
17:30And we'll go into "Advanced Options" here.
17:33And you can see what you're deciding.
17:36So we're deciding on two things.
17:38Number one is the Foundry resource.
17:41And the Foundry resource is just going
17:42to take whatever the name is with the word
17:46"resource" at the end.
17:47And you're also picking a region.
17:52And we haven't talked about this a lot yet.
17:55In future sessions, when we start talking about --
18:04Later in this session --
18:06we haven't talked about this a lot yet, but,
18:08later in this session when we start taking a look at --
18:12oh, it's the next session.
18:13Try again.
18:13Sorry. We haven't talked about this a lot yet, but,
18:18in the next session when we start talking
18:20about regional deployments, it's important to know
18:23that whatever we pick right now, this is the region
18:27where your models will be deployed.
18:30If you pick the wrong region, it's not really a big deal,
18:33we just deploy another project.
18:36But, if you are considering, and we'll talk about them
18:39in a little detail later, regional deployments,
18:42this is a big decision here.
18:44East US 2 is going to be fine for me.
18:46That's a fine description -- or subscription.
18:50And a new resource group is going to be fine.
18:53So I'll click "Create" here.
18:55And it'll take just another minute or so to go and build.
19:04All right.
19:04And so now we are seeing it has completed building the Foundry.
19:07We get a nice little introduction page here.
19:09I'll do the introduction so I won't worry
19:11so much about this page.
19:15And what we have in front of us here is the New Foundry.
19:19And the New Foundry has a series of tabs across the top
19:24for the different portions of kind of the lifecycle
19:28that you're going to be working through.
19:29And what we're on right now is Home.
19:30And Home, while not incredibly interesting,
19:33you will spend some time here just snagging these endpoints.
19:37We'll need these on a consistent basis so it's convenient,
19:40you just click "Home" and, boom,
19:42you can grab these endpoints at any point in time.
19:46Not much else on this Homepage.
19:48We can see kind of some news sections on here
19:51and then a quick start code.
19:53We'll take a look at that and a couple other links.
19:54But, generally speaking, the only thing you're doing here
19:57on Home is grabbing these endpoints.
20:01But then we have Discover and Build.
20:04So Discover is the area where you can search for new things
20:08and then Build once you have deployed them.
20:11This is where you're going to be able to work with them.
20:17So, in the Discover, we will see
20:19that we have then a panel down the side.
20:22And this will be pretty common as well.
20:25The items here on this panel are going to change depending upon
20:28which of those tabs we're on, whether it's Home, Build, Work.
20:33So Home -- the panel will change depending upon
20:37which of those tabs that we're on, whether it is Home,
20:40Discover, Build, et cetera.
20:44But, here, in Discover, we have an overview.
20:46And it's just saying, "Hey, here are some
20:47of the models that are possible."
20:48Very similar to that if we were to grab models.
20:52The next section, we will spend a bunch of time trying to decide
20:55which of these models to deploy.
20:57I'm just going to grab one really quickly here.
21:02So I'm going to search --
21:03-- for our GPT4.1, click on it.
21:10Again, we'll spend more time later trying to figure
21:13out which models to deploy.
21:15But I'm just going to grab this one right here, quick deploy.
21:19We'll use the default settings.
21:31And, instantly, we are now on the Build.
21:34You can see the little bolding right there.
21:36We're on the Build tab.
21:38Right? Let's go back to Discover for just a second just
21:41to see some of the other things that we could take a look at.
21:44Again, this is all exploration and kind of planning
21:47of what you are going to be building.
21:49So, here, inside of Agents, there are a few templates
21:53that you could pick from.
21:54Right? This is a -- we're not going to spend a bunch
21:56of time here in this course talking about these,
21:58but I do suggest that you dig down into those
22:01and take a look some of those starter templates.
22:03They're a really good spot for jumping off for new code.
22:07We'll talk in depth about a bunch of tools.
22:09If you just wanted to scan those tools up front,
22:11we've got that available.
22:13And end-to-end templates are available here as well.
22:16So, again, we won't spend a lot of time here
22:20in the course going forward, but I think you should
22:25when you're popping into the Foundry.
22:29Right? So that's our Discover side of things.
22:32Let's swing into Build.
22:36And over here in Build, really kind of the head
22:40of all of this is Agents.
22:42That's really the next topic over so we're not going
22:44to spend a bunch of time there.
22:47But we can then see here the models that we've deployed.
22:50And you saw just a second ago I deployed GPT-4.1,
22:54so we've got a version of that available to us.
22:58And, from here, we could then go and work on that model
23:02in the playground and kind of test it all out.
23:04So we can see here we will spend a lot
23:08of time building instructions, "What can you do?"
23:20And a almost non-instruction model here will say, "Well,
23:24I can -- I don't know, I can do lots of stuff.
23:27I am a large language model.
23:29Here are some ideas you can try."
23:33Other things that we're going to be looking at as we go,
23:37we will spend some time taking a look at fine-tuning.
23:39So, once you've deployed a model, you may find
23:43that it's not exactly what you want from that model
23:45that you might want to change its behavior a little bit.
23:48We can go through a fine-tuning process.
23:49We will see that.
23:52Models are only as good as the tools
23:57and the knowledge that they have.
23:59So, before we go to add those into our agents,
24:03we'll often spend time going ahead and building those tools
24:06or adding that knowledge.
24:09In a little bit, we will start talking about things
24:12like the six principles of responsible AI, we will start
24:16to see things like guardrails and evaluations
24:18as being part of that effort.
24:24And so Build is the spot where we're going to spend a lot
24:27of time here creating our resources.
24:30Where we won't spend as much time is on the Operate side
24:33of things, but it's still important, right,
24:36that the Foundry is giving us this overview
24:38so that we can see all things that are associated
24:42with our models and our agents.
24:47So, over here, in the Assets side of things, we would see all
24:51of our agents, all of the models that we've been deploying
24:56across projects, all of the tools
24:59that we've deployed are all here.
25:03When we're looking at things like compliance,
25:06this is a one-stop shop here for our policies, guardrails,
25:09security posture, security and governance.
25:13You will note -- and we haven't talked about it a lot,
25:16but you will note, when you go to deploy models there,
25:18you can only have so much throughput per model type.
25:24So this allows you to see
25:25across projects how many models you have deployed and how much
25:30of your allocation have you deployed of them.
25:34And then, finally, on the Admin side of things, we will see all
25:40of the projects that we have available to us.
25:43Right? So that's the Operate.
25:45Then off to Docs.
25:46Okay. Now, one thing that we did talk about earlier,
25:51if we swing over here into Build --
25:55-- and we look at our models, we have a section
26:02in here called AI Services.
26:07And this AI Services, this is that tools section
26:11that we talked about earlier that we're going
26:12to have generative AI models, we're also going
26:15to have those purpose-built machine learning models
26:21that can do things like voice live and speak to text.
26:24In Topics 3 and 4, we will spend a lot of time with those,
26:31but know that, when you deploy a single project,
26:34you're already ready to go.
26:35You don't have to do a secondary deployment or bring those,
26:38they're all here and ready to go.
26:44All right, so we've got a lot more to do, so let's go back
26:47to the deck and finish this up.
26:52All right, let's see how we did.
26:55Which web portal should you use to work with assets
26:59in a Microsoft Foundry project?
27:03In this case, we're looking at the Microsoft Foundry Portal.
27:05And we spent a little bit of time in there earlier.
27:09Okay? Which component
27:10of Microsoft Foundry provides pre-built services
27:13for common AI tasks?
27:17That is indeed the Foundry Tools, those additional tools
27:20that help supplement large language models.
27:25Which extension should you use in Visual Studio Code to work
27:28with Foundry projects?
27:33And, here, if you're using Visual Studio Code,
27:35the recommendation is the Microsoft AI Toolkit.
27:41So, here, we laid the groundwork for everything that's going
27:44to be following in this course.
27:47We took a look at the Foundry projects
27:49and the creation there of projects.
27:51They are the ones that are going to hold the models
27:54and the agents and the tools, the knowledge that we're going
27:56to be building on top of for the next 20-some sessions.
28:01Included in there are a bunch of built-in tools, language
28:04and speech, vision, document understanding,
28:07to supplement what some
28:08of our generative AI solutions are capable of.
28:13Visual Studio Code is going to be our primary tool
28:16in this course using the Toolkit and the Foundry.
28:19And then we're going to have a series of SDKs, both the Foundry
28:23and the OpenAI SDK in this topic.
28:25And we'll go a little bit further in the next one.
28:28And six responsible AI principles are going
28:32to guide us along the way.
28:34We'll spend some time as we tick off those requirements along
28:39the course.
28:42In this section, we just started.
28:44We grounded ourselves in what AI is on this platform,
28:48explored Microsoft Foundry and the Foundry Tools,
28:51and walked through the developer tools and SDKs that we're going
28:54to be using, and grounded our thinking in responsible AI.
28:59Thank you for participating in this "Develop AI Apps
29:02and Agents on Azure" course.
29:04I hope you enjoyed learning about planning AI solutions
29:07on the Microsoft Foundry platform as much
29:09as I enjoyed presenting them.
29:12I encourage you to remain curious
29:14and continue exploring new capabilities.
29:16That's how we're going to stay relevant
29:18in this fastmoving world.
29:19There are many ways to continue your learning.
29:22And I invite you to search for your next favorite topic
29:25on Microsoft Learn at aka.ms/learn.

About this transcript

This page contains the full transcript of Plan and prepare to develop AI solutions on Azure | AI-103 | Episode 1 by Microsoft Learn, generated from the public captions YouTube serves with the video. The transcript has 4,575 words across 565 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.