YT YouTube2Text

YouTube2Text → Transcripts → Building AI Applications with the Laravel AI SDK

Building AI Applications with the Laravel AI SDK — Transcript

by Laravel · 24,247 words · 3,696 segments · language en · Watch on YouTube

Open in transcriber Summarize with AI Download SRT/VTT

Full transcript

0:03And we are live. Hello everyone. Good to
0:06see you all. Uh we have very special
0:10guest today. We have the Taylor Otwell.
0:13Uh Taylor, how are you doing this
0:14morning?
0:15I'm doing good. I'm rocking and rolling.
0:17How are you?
0:19I'm doing pretty good. It was a It was a
0:20good weekend for me. Hopefully everyone
0:22in the chat. Uh I know we have already
0:24had quite a bit of people um in the chat
0:27already. Uh so a lot of people waiting.
0:29So if you are here and you are uh ready
0:33to see what the AI SDK has in store for
0:37us and also just to hear Taylor's
0:38thoughts on the AI SDK, why he built it.
0:40I'm excited for that personally. I
0:42haven't gotten a chance to talk to
0:43Taylor yet about the AI SDK
0:45specifically. Uh but if you're here, let
0:47us know where you're watching from and
0:49um I guess Taylor, what's a good What's
0:51a good question? What's something that's
0:52been on your mind like outside of work?
0:54Like maybe TV show, what something like
0:56that? Oh man, I don't even know. Um
1:00everything's been a blur since I did the
1:01[clears throat] AI SDK and then I was in
1:03um
1:04India and Dubai for Laracon India. So
1:07like I feel like I've been all over the
1:08place. Yeah, I feel like this is like
1:10maybe the first week I feel like I'm
1:11kind of getting back in the groove.
1:13Um There you go. Yeah, so I don't know.
1:15I don't even know what's going on in the
1:16world.
1:17>> [laughter]
1:18>> I guess what was your Whenever you
1:20travel somewhere, what is like the the
1:23food or like thing that you try to make
1:25sure you you do?
1:28Mhm.
1:29I try to just like have
1:32especially if I'm going somewhere like
1:33India, um try to have like whatever food
1:36that the people there really like. So
1:38Okay. For example, like the first time I
1:40went to India,
1:41I was the at the hotel and like the room
1:44service guy, I got back kind of late to
1:46the hotel
1:47and he was like, you know, do you want
1:48any food? Like what do you do? It was
1:49like almost midnight. And he was like I
1:51was like, just bring me like whatever
1:52you would eat at this time. Like
1:55whatever your favorite thing to have
1:56would be, just like bring that and I'll
1:58just try it. Uh so I just try to do
2:00stuff like that.
2:01Um
2:02just so I can get a feel for like what
2:03people are like, you know, what they're
2:05really doing in these places.
2:07I love it. I guess that it for everyone
2:09watching. I already see a bunch of
2:11all over the place. So we got
2:13Evan from Switzerland, Bakersfield,
2:15California. Glad to see you here.
2:17Albania, Kuwait, UK. I love just like
2:21the uh how how global everyone is.
2:23India. Um good day from Pakistan. I
2:26guess everyone what time is it for you
2:28right now? I always love to see that.
2:30And then also kind of similar to Taylor,
2:32what is what is the thing that you
2:35always tell people to try whenever they
2:37are visiting your region? So food that
2:40you always tell people to try.
2:43Oh, that's a great question actually. I
2:44think like in Arkansas and maybe other
2:46states in the south, a lot of people
2:48like to try the barbecue. You know, and
2:49when I say barbecue, I don't mean um
2:52Like Texas style.
2:52>> like
2:53Yeah, yeah, like barbecue with like
2:55barbecue sauce, like pulled pork,
2:58brisket with barbecue sauce, stuff like
3:00that. Around here it's a lot of like
3:01pulled pork. But um
3:03Yeah, in Texas it's more like the
3:05brisket, you know.
3:06Yeah, that makes sense. I
3:09I have not been to Arkansas. In In
3:10Arizona, I guess I would tell people to
3:12try any hole in the wall Mexican
3:16restaurant. Like that would be my go-to
3:18to
3:18>> Yeah. There are a lot of good Mexican
3:20places here, too. It's so good.
3:22>> Yeah, it's it's great. Unless you're
3:23like in Scottsdale, then there's some
3:24like good burger places that I would
3:27recommend. Uh good old Greg's sausage
3:29roll. I have not had one, unfortunately.
3:31I've heard good things.
3:32Uh 9:00 p.m. Dubai time. Well, thanks so
3:34much for staying up staying up late. Uh
3:37we got 14:00 here in Brazil.
3:39Uh 10:03 p.m. in Pakistan. I think that
3:43might be Oh, never mind. 1:03 a.m. right
3:45now in PH.
3:48All awesome. I love it. Macedonia 16:03.
3:51Glad or 18:03. Glad to have you all
3:53here. Any Ghanaian developers in the
3:54room? Says Ovak for you.
3:57Um Joshua, what is going on gents? I'm
4:00excited for this. Well, we're excited to
4:01have you here, Joshua. Glad to have you
4:03here. Um and Liam says, I'll buy you a
4:05sausage roll next time you're in the UK,
4:06Josh. Okay, sounds good to me. I am
4:09always down for some food.
4:11Uh I did have Taco Bell when I was in
4:13the UK, so and it was Well, that's
4:15actually pretty good.
4:17Man, I've got a Taco Bell that's like
4:18right outside my neighborhood. I mean,
4:20I'm talking it's like across across the
4:21street, basically. I didn't know I
4:23didn't know this about you. If you If
4:25you love Taco Bell, then that's awesome.
4:27I love it. It's so quick. I can like run
4:28over there and get like lunch and be
4:30back home legit like 10 minutes. Even if
4:34I eat at the restaurant, you know. Hey,
4:36>> [laughter]
4:36>> what's your What's your go-to current
4:38Taco Bell menu item right now? Oh man,
4:40like if I just want to be like really
4:42efficient in and out, I'll do just like
4:44two bean burritos and a drink and
4:46>> Okay.
4:47crush them and then dip out of there.
4:48I'm It's like 5 minutes.
4:51What do you get? I I get
4:54um like either the cheesy bean and rice
4:56burrito. Like that's kind of like one of
4:58my go-to. Or I always love like anything
5:01chicken from them. So like the There's
5:03the the cheesy chicken chipotle
5:05flatbread thing. I like that.
5:07>> Yeah.
5:07Um but yeah, like cheesy bean and rice
5:10burritos I've always had since I was
5:11like in high school, so those are kind
5:12of like my
5:13you know, uh go-to for nostalgia's sake.
5:17Yeah, for sure.
5:19And then uh Baja Blast. I used to do
5:20full sugar. Now I'm Baja Blast Zero.
5:22Mine's Mine daughter does the Baja Blast
5:24Zero, too.
5:25Yeah. I I I tried it full sugar again
5:28like for the first time in like I don't
5:29know, 10 years and I was like, "Wow,
5:30this is This is a lot."
5:33I can feel it.
5:35>> Awesome. Well, glad to have everyone
5:36here. It's It's crazy to see how many
5:38people are here mostly I to see Taylor
5:41and also to hear about this AI SDK.
5:44Um so, if you do have questions in the
5:46chat, be sure to drop them. I'll try my
5:48best to be watching, but mostly the goal
5:50of this live stream is one to get
5:52Taylor's thoughts on the AI SDK, what it
5:54is, how you would build with it within
5:57Laravel, and maybe like why it was
5:59built, and then two, hopefully like the
6:01towards the latter half of this stream
6:03being able to to build something.
6:04There's There's some projects that I've
6:06I've built um that we can kind of show
6:08off the AI SDK and just like what's
6:11capable of, but I think like the the
6:14perfect part is being able to say,
6:15"Okay, what actually is capable when
6:17you're building things?"
6:18Um and then the why. So, again, if you
6:20do have questions, again, everyone's
6:21kind of shouting out where they're from.
6:23I love it. Keep doing that. Um it's
6:24awesome to see people from all over the
6:26world being able to come together and
6:28say, "Hey, this is some cool things that
6:30we're building with this cool software
6:32that we love."
6:33Um so, Taylor, powered by Taco Bell, uh
6:37what was the
6:39reason like when did I I I think like
6:41the first thing I saw you uh start
6:44talking about the AI SDK? It was
6:46probably I don't know, probably in
6:49October, November of last year. I could
6:51be wrong on the timeline, but why did
6:52that come into [snorts] fruition? That
6:54was one of your projects, I think, that
6:55you kind of picked up and you're like,
6:56"I'm just going to I'm going to dedicate
6:59my life to this."
7:00Yeah. I mean, obviously everyone was
7:02just kind of tinkering around with AI
7:03stuff
7:04as they have been for the last few
7:05years, and um
7:07you know, it's just like such a
7:09important part of the dev workflow at
7:11this point. It felt like we needed some
7:13sort of first-party opinion on
7:15interacting with AI providers. I mean,
7:17just like we have like opinions on
7:19sending email or like queuing jobs, and
7:21just like
7:22this [clears throat] is becoming such a
7:23common part of like what people
7:26do when they build apps, um
7:28that made a lot of sense to bring
7:30something first-party and like have like
7:32an offering here. I usually try to think
7:34of like if I'm going to build something
7:35into the framework,
7:37I kind of think to myself like
7:39is this applicable to like 70, 80% devs,
7:43you know, in in the Laravel world. Like
7:46if I'm debating whether something needs
7:47to be in the framework. Yeah. Um and
7:49then like
7:51for a package like the AI SDK is maybe a
7:53little bit more lenient. Like is this
7:54applicable to maybe half of Laravel devs
7:56like in the world? Like probably so,
7:58right? With AI and probably increasing
8:00dramatically over time. So, made sense
8:02to have something and yeah, I did build
8:04it myself and it was the first package
8:05I've written at Laravel in a couple of
8:08years. The last stuff I wrote was a
8:11Livewire Vault and Laravel Folio.
8:14Yeah. And then as we were like building
8:16Cloud, building Nightwatch, I didn't
8:18really write a lot of packages at the
8:20time and now I'm kind of like back in
8:21the driver's seat with the AI SDK. I
8:24just kind of like one, [clears throat]
8:26most of other people here were busy with
8:27other stuff, right? So, it's [laughter]
8:28like So, someone has to build it, but
8:31everyone's busy. So, it's like, well,
8:33I I it'd be good for me to like get back
8:35in the code and like building something
8:37fresh and it sounds fun. So, like I'll
8:39tackle it and plus I had some ideas for
8:40what I wanted to do with it.
8:42Um there was already some cool stuff
8:44being done in the community around AI
8:46stuff with like TJ Miller's work on
8:48Prism, which we kind of built upon in
8:50the AI SDK. I kind of see like that is
8:53like almost like a query builder and
8:55eloquent type of relationship like
8:58you know, I think the AI SDK I tried to
9:00pull like
9:02a little bit more opinions on top than
9:04people might typically build in a in
9:07like a kind of a generic AI SDK.
9:10Yeah. Um
9:12yeah, that's that's awesome. I guess
9:13like for for those in in the chat some
9:15some quick questions um as well. So, uh
9:20um
9:22uh will be paid or free? It is already
9:24out, silent, and it is it is free. It's
9:26a free package, uh free package to use
9:28for the AI SDK. I guess for people who
9:30are just joining us as well, thank you
9:32so much. We're going to kind of take the
9:33first little bit talking a a little bit
9:35to Taylor about his thoughts about why
9:37built it, what's it for, um and then the
9:39second half of this stream we'll be able
9:40to jump into some code and see, okay,
9:42what what does this look like
9:43practically building um in today's day
9:46and age. Uh but uh for those joining,
9:49what is your favorite outside of the AI
9:51SDK? Cuz that's a little bit of a curve
9:53or or
9:53home run type of question. Uh what is
9:56your favorite Laravel package, either
9:58first-party or third-party? Love to hear
9:59it in the in the chat.
10:02Um
10:03Taylor, what's your favorite package
10:04that you've built outside of the AI SDK?
10:07Oh, man. If it's like something that's
10:09in Laravel, it'd probably be Eloquent.
10:11Um it's also the hardest thing I've
10:12pretty much ever built. Uh if it's like
10:15a package, I'm a pretty big fan of like
10:19um I think Reverb is cool.
10:21Um I think Echo is cool, like the kind
10:23of the real-time stuff in Laravel.
10:26What what else do we even have?
10:28Um
10:30Let me Let me Let me look at the docs.
10:31We've accumulated quite a few packages
10:33in there.
10:33>> There's quite There's quite a bit out
10:34there. I mean, I think Octane is pretty
10:36cool. Uh to some extent, like the bulk
10:39of Octane
10:40is sort of handled by other people,
10:42right? Like the FrankenPHP team and
10:44Swoole and stuff like that. And but uh
10:47just the ability to like boot the
10:48Laravel app and kind of keep feeding
10:50your requests really quickly.
10:52Um
10:53yeah.
10:54Uh Pete Bishop said, "My favorite
10:56package is Pennant. Cannot live without
10:57Fortify." It's crazy how many packages
10:59there there are that the Laravel
11:01ecosystem has, especially when on our
11:03Laravel YouTube channel during the
11:05advent season of last year. So, for 24
11:09days and then a special guest message
11:10from Taylor on the 25th day, uh we had
11:14one video for every package. And there
11:16were some packages that we we didn't
11:18make a video for. So, enough to have at
11:20least 24 videos is pretty crazy.
11:22Um and some of those could have been
11:23split up as well. Uh yeah, I I think my
11:26favorite personal favorite is Cashier,
11:28and I didn't really get into Cashier too
11:30much because when I started my Laravel
11:32journey, I started with um Spark. And so
11:35I didn't really have to touch cashier
11:37too much. It was kind of already done
11:38for me.
11:39Uh
11:39>> Yeah.
11:40But over the last year using cashier, I
11:42was like, oh, this is actually
11:43incredibly easy.
11:45Man, yeah, we're working on some other
11:46some new cashier stuff and some new
11:47fortified stuff actually.
11:49Um
11:50that hopefully we can show pretty soon.
11:51But like man, cashier is like
11:54that is a
11:56I don't think like back when I first
11:58wrote cashier like integrating with
12:00Stripe and all that, it is a lot of work
12:02to like catch all the web hooks, store
12:04all that. So
12:05Yeah, I think a lot of people find that
12:07package pretty valuable. Yeah, and
12:09especially if you look outside of the
12:11Laravel ecosystem, most people, you
12:13know, have their own set way of okay,
12:16here's how to not make Stripe incredibly
12:19difficult to work with and everything
12:20like that. So you know it's you know
12:22it's valuable.
12:23We got some people were saying Reverb,
12:25queues and jobs. We'll call that as part
12:28of a package. Working with queues and
12:29jobs is is incredibly easy. Yes, love
12:31for Octane.
12:33Prism is insane. I agree. We got inertia
12:36love, Laravel Nova
12:39favorites love. Octane,
12:42sale, Octane, Reverb, Fortify.
12:46I love it. I love it. Reverb is also
12:48cool. I I do love I do love Reverb.
12:50Makes everything makes everything easy.
12:53So Taylor, when it comes to the AI SDK,
12:56what is how is this different than maybe
12:59like working with AI
13:02in Laravel. So how is this different
13:04than like a boost for example or even
13:06MCP? Where does this
13:08SDK kind of fall into?
13:11Yeah. So like right now we have kind of
13:13three AI related packages in Laravel.
13:17I think MCP was actually the first one.
13:19So MCP is
13:22MCP stands for model context protocol.
13:25MCP, right? Model context Protocol.
13:28And it is basically a way to like I
13:31think of it as a way to like expose a
13:33standardized API for LLMs and AI things
13:37to talk to. So like I have an app on the
13:39internet and I want to allow like
13:42ChatGPT to do things or some other tool
13:44to do things with that, then I can use
13:46MCP to expose functionality and OpenAI
13:48started coming out with this new kind of
13:50like I don't even know if it's out of
13:51beta yet, but eventually like sort of an
13:52app directory of things you can like
13:54interact with from ChatGPT and things
13:56like that. That's all kind of powered by
13:58MCP.
14:01So then there's
14:03Boost, which is
14:06actually a little local MCP server that
14:08runs on your machine
14:10that Claude Code and Cursor and other
14:12like agentic dev tools like Open Code
14:15can plug into.
14:16And what it does is it just like makes
14:18available some tools to those agents so
14:20that they can query the Laravel docs,
14:21they can run Tinker commands, they can
14:23inspect your database schema. And so the
14:26idea is it improves like the quality of
14:28the Laravel code that things like Claude
14:31Code and Cursor can write because they
14:33have access to the latest docs. So like
14:35if I release a feature, you know,
14:37tomorrow in Laravel because we release
14:39every Tuesday,
14:40LLMs have like no idea that feature
14:42exists, right? Because they've been
14:43trained on old data, but Boost actually
14:47allows us to feed them like the latest
14:49info. So
14:50they always have access to the most
14:52up-to-date docs.
14:53If you're writing Laravel and you're
14:55using Claude Code or Cursor, which at
14:57this point you probably should be using
14:58some sort of like AI assistance,
15:01I think installing Boost is just like a
15:03no-brainer cuz it's like a free package
15:05that is probably only going to make the
15:06quality of your code better. So I would
15:09definitely install that. So that's the
15:10second AI package we released. And now
15:12the
15:13AI SDK is the third, which is sort of
15:15like
15:16a streamlined unified interface for
15:19working with different AI providers like
15:21OpenAI, Gemini, Anthropic. So, I want to
15:24generate some text or I want to stream
15:26some text or I want to generate an image
15:28or audio, I can do that with the AI SDK
15:32through a unified interface and kind of
15:33try different models, try different
15:34providers without going and like
15:36researching the What What is the you
15:38know, what's the HTTP endpoint for
15:39generating image on Gemini and now what
15:41is it on open AI?
15:43It's just a really streamlined way to
15:45work with AI.
15:46Yeah.
15:47Yeah, I love it. Yeah, someone someone
15:50asked the question
15:51for someone that does not know what
15:52Laravel AI SDK is, how would you
15:54describe it in a few words? I think
15:55Taylor just did that, but I think if I
15:57could put it into other words and maybe
15:59Taylor you can correct me. It's like an
16:00eloquent way of interacting with all
16:02these things that now in this world of
16:05AI you would expect you have to interact
16:08with at some point in time.
16:10Yeah, I think so too and then which I'm
16:12sure we'll get into a little bit later
16:13is sort of this um
16:15integrated with a lot of Laravel's the
16:17rest of its stack. So, like if I
16:19generate an image, I have a method to
16:21like store that image using the file
16:23system stuff in Laravel or if I want to
16:25like queue the generation of an image,
16:27it's actually integrated with the queue
16:28system in Laravel. So,
16:30it's an AI SDK that really leans into
16:33like the full stackness of Laravel.
16:36I love it. I love it. When you think of
16:40and some people are saying your your
16:42video quality isn't doing too good. I
16:44don't know if there's
16:45a way for you to bump it up, but if that
16:48means
16:48>> know. you start stuttering. I don't
16:50know.
16:51But,
16:52for everyone who's saying that
16:54uh I'm on standard definition, so I can
16:56go up to Do I go up to full high
16:58definition? Is that recommend? Full high
17:00definition? Uh maybe.
17:03All right, do I look good now? Do I look
17:05better? Let's see.
17:06Um I think I think you look better.
17:10>> Should look Should look
17:11video enhance, says someone.
17:13Yeah. Exactly. Taylor will get on that.
17:16Let's
17:17Yeah,
17:19sweet. But yeah, let's uh
17:21let's take a turn into Okay, all of this
17:24AI stuff is fun to work with, but I'm
17:26curious of maybe what you're seeing
17:29within
17:30um you know, current applications,
17:32enterprise applications, etc. Or just
17:36why would people start adding AI into
17:39their apps? What are the things that
17:40people are generally going to be using
17:41it for? Um what are some practical
17:43applications that you can now build with
17:45the AI SDK?
17:47Yeah. I mean I I think some of like the
17:49most common things that you see people
17:51build with it are sort of like almost
17:53text summarization type features. So
17:56like on Night Watch for us, which we've
17:58already launched, was like you have an
18:01issue in Night Watch, right? like an
18:02exception has occurred or you have like
18:04some slow query. You can actually click
18:06a button in Night Watch to let the AI
18:08write like a summary of that. So and it
18:11will tell you like okay, here's kind of
18:12what happened, here's some suggested
18:13solutions, here's where you might look
18:15next or whatever. I think that sort of
18:18like category of AI usage is super
18:20common. Like I have a I have a bunch of
18:22text that I need to basically like
18:23classify or summarize.
18:25It's super easy to do that with AI um
18:29and pretty fast and affordable.
18:32Um you know, I think another like chunk
18:35of AI is around like audio and
18:37transcription and sort of real-time
18:39applications. That's another like
18:41popular
18:42um
18:43use case for AI. I mean this is kind of
18:45I don't know if this is like
18:47people consider this like full on AI in
18:49the normal sense of the word I phrase,
18:51but I mean I think it kind of is related
18:53is around like embeddings and vector
18:54search, which is part of the AI SDK
18:56where we can generate embeddings from a
18:59given like string of text, vector
19:00embeddings, store them in a database,
19:01and then do semantic search querying,
19:03and then AI powered re-ranking of those
19:06results for like the most semantically
19:07similar results. [snorts] So
19:10Um you know, I think those are just some
19:11of the more common use cases. I think
19:13that then there's like image generation,
19:15which I think is a little bit more like
19:17you know, I don't know if it's a little
19:19bit more like the consumer side of like
19:21um the use cases for AI where you're
19:23generating sort of fun images and things
19:25like that. But at Laravel we've been
19:27using it mainly I would say for like
19:29text summarization, issue summarization,
19:32things like that.
19:34Yeah, definitely. And and the neat part
19:35I think with the AI SDK kind of like it
19:37you said is now you have this one
19:39central point. I know in the past within
19:42things, you know, there there are great
19:43packages like Prism that I I know you
19:45said that the AI SDK is is built on top
19:48of and expands in a lot of ways. Um, but
19:51it's great that you can kind of have
19:53this one central place to say, "Okay,
19:56when I'm working with images or when I'm
19:58working with audio, the infrastructure
20:00behind it, the the code behind is
20:02exactly the same." Um, I'm curious your
20:04thoughts and I and when we dive into
20:07building something within the code, I
20:09I've already used this within things
20:11like Cloud Code and Boost and everything
20:12like that. It's it's crazy how much
20:14better it gets because it's just one
20:16package for the LLMs that you're working
20:19with to say, "Okay,
20:20hey, this audio is going to be generated
20:23the same way that I generated this this
20:25chat." Do you think that's intentional
20:26that you built like how what are your
20:28thoughts on that when it comes to this
20:29new world of coding in this in that
20:31sense?
20:32In terms of like how much effort do I
20:34put into sort of like the elegance of
20:35the API?
20:37Yeah, either either that or is that
20:39elegance of the API matter now?
20:42Yeah, I know. It's something I've
20:44thought about myself. Um,
20:46I think it does sort of matter in the
20:48sense of like the LLMs still do well
20:51with things that are like easy to parse
20:53and understand and summarize and read
20:56about. I don't know if it's like,
20:59you know, obviously the LLM doesn't
21:00derive the same joy from it that like
21:02the human consumer might. Now, I used to
21:04put like a a of effort into these sort
21:06of like whimsical fun APIs because as
21:08humans were writing the code it's kind
21:10of fun to like use these tools that are
21:12sort of like charming in a way.
21:14Um and maybe it still is, but yeah, I I
21:16mean it's still important I think to
21:18have good API design just for the
21:19discoverability of that LLM to be able
21:22to figure out what the heck's going on,
21:24you know?
21:25Yeah.
21:25>> Um and I think like we we've tweeted
21:27about this a little bit, but like
21:29frameworks that sort of lean into
21:31conventions and structure I think do
21:34well with LLMs. I think we've seen that
21:36with like Laravel and things like Rails
21:38where like the LLMs are doing a pretty
21:40good job because there's lots of
21:41training data, there's these very
21:43conventional structure to the projects
21:44where it's like there's a models
21:45directory, there's a controllers
21:46directory, so they kind of like it's
21:48very discoverable, you know?
21:50Yeah. Yeah, I I I completely agree and
21:52it's it's it's neat to see the, you
21:55know,
21:56I I feel like the joy from those clean
21:58APIs comes now of of seeing it when
22:01you're when you're looking at your maybe
22:03code being generated or you're just
22:04looking back and it's very easy to know,
22:06okay, this is exactly what's happening
22:08rather than having to jump through 50
22:10different files to say, okay, what
22:11actually is happening. It feels like
22:13you're even if you're not handwriting
22:15all that code it still feels like you
22:16understand what's happening in that in
22:18that case, and I think the LLMs do a
22:20great job at
22:22also saying,
22:23you know, we're using this one package,
22:25so we're going to be doing it the exact
22:26same way another another
22:29iteration of the application.
22:31Yeah, totally.
22:33Uh for those just joining, thank you so
22:35much for for being here. Uh I am joined,
22:37my name is Josh, I'm joined by Taylor
22:39Otwell, you probably know Taylor
22:41uh just from everything, the creator of
22:45Laravel and Beam and now the creator of
22:47Laravel AI SDK, which is what we're kind
22:49of diving into here on the stream. So if
22:51you're just joining us, let us know
22:52where you're joining from, uh what time
22:54it is there for you, and then what's
22:55your favorite Laravel package. Uh but if
22:58you haven't gotten the chance, feel free
23:00to like and subscribe this video if
23:02you're on our YouTube platform. There's
23:04a whole bunch of awesome videos that we
23:06try to put out as consistently as
23:07possible, but we're going to continue to
23:09take a look at the Laravel AI SDK and
23:11then hopefully be able to have a chance
23:12to to build with it as well.
23:15Um Taylor, what is your favorite part
23:16before we jump into like each aspect of
23:19the AI SDK?
23:20Uh just from an overall standpoint, what
23:22is your favorite part of the AI SDK? One
23:24that you're like when you built it
23:26you're like, "Yeah, this is really cool.
23:27I like that. I like that."
23:29>> Yeah. Yeah. Yeah. It's a good question.
23:32Um I really like the kind of like agent
23:34class concept, which was a big part of
23:36like um my initial sort of like thinking
23:39behind the AI SDK. And you know, for
23:42those who haven't seen it, basically
23:44when you use the AI SDK, you make agent
23:46classes using like a make agent artisan
23:48command and it's basically a class that
23:50sort of encapsulates the system
23:52instructions, the message context, the
23:54tools, maybe the schema of um sort of
23:57like what you're doing um with the AI
24:00provider. So, you might have for example
24:02like, I don't know, I think in the in
24:04the docs we use a lot this sort of like
24:06sales coach or like lead extractor agent
24:08that's like reading transcripts of sales
24:10calls and giving advice and things, but
24:12I I think it's like this very
24:13Laravel-esque way of working with AI
24:17where everything feels sort of like nice
24:18and tucked in this little class that I
24:20can reuse throughout my application. So,
24:22maybe I use it in an artisan command in
24:23one place and I use it in a HTTP
24:25controller in another place and it's
24:27just sort of all like self-contained.
24:29Also very easy to like test. Um
24:33where you know, it had you can fake
24:35agents, you can fake other AI
24:36interactions. I think the testing story
24:38was something I tried to like make
24:41decent. I think testing with AI is
24:42something that's often like overlooked.
24:44One cuz testing AI can be hard. It's not
24:46deterministic. Um
24:48you know, there's all kind of quirky
24:49things that can happen. So, I tried to
24:50make that pretty streamlined.
24:52Yeah.
24:53No, I love that. And I I I that's an
24:55interesting standpoint that I've I've
24:57personally loved as I've worked with it
24:59just in like the past couple of weeks
25:01before launch and then also even just
25:03after launch I've I've built probably a
25:04couple two or three different
25:06applications just familiarizing myself
25:08and one of the things that I really love
25:10about the agent SDK specifically is that
25:13is that aspect where it feels like I
25:16don't know similar to a job where you
25:17would or an action I guess would be a
25:19better
25:20use case but where where now I'm
25:23abstracting that from every single job
25:26or controller that I'm using from that
25:27specific logic and it lives in one
25:29place. I can change the as we all know
25:32for if you've built anything within AI
25:35you have a prompt and sometimes you
25:37might want to change that prompt it
25:38sucks to have to change it in 10
25:40different places that you're using that
25:41exact text prompt so being able to have
25:43it in one place is is really awesome.
25:46Yeah.
25:48Awesome. I want to jump into the the
25:50landing page real quick and just kind of
25:52talk through some of the some of the
25:54aspects before
25:55getting into
25:57the the rest
25:59of the actual code. I'm going to
26:02plop this open right here.
26:06Perfect.
26:07So
26:08Oh.
26:10Lost you for a second. There we go.
26:12Okay. So for those who don't know that
26:14the Laravel AI SDK is out it is out it
26:17is laravel.com/ai.
26:19So laravel.com/ai
26:23and the it's the AI toolkit with
26:25batteries included. It's as simple as
26:28running this composer require command
26:29composer require laravel/ai.
26:32I guess one question for you Taylor is
26:34what was the you kind of mentioned it
26:36but what was the thought behind putting
26:38this in in one single package versus
26:41like hey here's a package for
26:43image here's a package for voice. What
26:45what's your kind of thoughts or idea
26:46behind that?
26:48You know I kind of just wanted people to
26:50have a complete toolkit for working with
26:52AI and not have to pull in a bunch of
26:54different stuff and kind of like
26:55overthink it. Uh so it's for me it's
26:58just kind of nice to be like composer
26:59require laravel/ai and boom I just sort
27:01of like have everything I need to do AI
27:05SDK related things whether that's audio
27:07or images or whatever. Um it doesn't
27:10really like you know significantly
27:13increase the size of the package to add
27:15these things. All the bulk of the work
27:16is handled by people like Jim and I and
27:18open AI and all that actually.
27:20>> Yeah. I do this stuff so it's just kind
27:22of nice to have this like one
27:23self-contained package to have a kind of
27:25a complete tool kit.
27:27I love it. Um so yeah just like Taylor
27:30said one SDK for every capability.
27:32I wanted to walk through some of like
27:34the elements that we have here on the
27:35bottom of the page. Hey that's you
27:36that's us today.
27:38But there's these kind of like
27:40uh you know there's seven different
27:43or eight different aspects and this is
27:46only again a small part of it but
27:47there's prompts, context, tools,
27:50provider tools, structured output,
27:52streaming, async, and attachments. I
27:54kind of wanted to get your thoughts and
27:55walk through of like when you might use
27:58each one cuz I feel like especially in
28:00today's age one of my thoughts is that
28:02as you are learning how to build with
28:04with Laravel of course maybe your LLM as
28:07you're building this within Laravel or
28:08adding AI SDK so that you can build and
28:11so for those who are installing the AI
28:13SDK if you run and you are using AI
28:16agents which like Taylor said it's it's
28:18a good thing to be using them I think
28:20you know. If you run boost install PHP
28:22artisan boost install after you install
28:25the Laravel SDK it will ask you to to
28:27install
28:28the AI SDK skill which will help the
28:31LLMs learn how to use this better.
28:33But like I think Taylor it's one of
28:36those things where
28:37the more you understand what can be
28:39possible behind the scenes the better we
28:41can help prompt our LLMs how to do that
28:44and so that we know what's actually
28:45happening. What do you what
28:48just in maybe like a couple sentences
28:49maybe we can go through each one of
28:51these and I just want to hear your
28:52thoughts on when you would use prompts,
28:54what is context and when to use it tool
28:57etc.
28:59Yeah, [snorts] prompts are sort of like
29:00the basic way to ask the LM to generate
29:03some text in response to like text you
29:05give it. So
29:06you know, if you've ever used chat GBT
29:08and you ask it a question and you get it
29:09some text back and that is like you're
29:11prompting the LM.
29:13Very similarly like in the AI SDK this
29:15is how you do that in code, right? So
29:18we're going to send the LM some sort of
29:20like text a question
29:22maybe like a command like in this case
29:24we're saying analyze this transcript and
29:26we're kind of passing along this file
29:28and it's going to give us back some
29:30text. So this is like a super common use
29:32case for AI, right?
29:35Similar to that summarization aspect
29:36that you had that you had said before.
29:38Yeah, exactly.
29:40And it doesn't have to be like you can
29:42see even in this case it's not actually
29:43like we don't have to use prompting just
29:45for chat so to speak, right? We can just
29:47like give it a task almost as a prompt
29:50and we we don't intend to ever revisit
29:52like the conversation, right? It's not
29:54an ongoing conversation but we're just
29:55prompting the AI
29:57or the LM with a specific task and we're
29:59getting a response and we're storing
30:01that or we're sending it to the user or
30:02whatever but it's not really an ongoing
30:04chat. And to your question like the
30:07context tab are you this is actually
30:10good. You can go you can look at this.
30:12This is a good example of like that
30:14sales coach agent. This kind of shows
30:16you how you can kind of like encapsulate
30:18the system instructions within this
30:20class and then we can kind of reuse this
30:22throughout of our application.
30:24So that we can just new up the sales
30:26coach agent and then give it a prompt
30:28and we can do that in a variety of
30:30places throughout our app without sort
30:31of like re-specifying those system
30:33instructions. So that's super
30:35convenient. What one question I had here
30:38just I think it's a good place to talk
30:40about it is so
30:42you kind of mentioned it's one central
30:43place to be able to interact with
30:44everything. So we have all these tools
30:46available to us that we see up here, but
30:48you're we're not saying
30:50send a prompt to open AI or even send a
30:53prompt to a specific model or even send
30:55a prompt to [clears throat] you know
30:56Gemini. How does all that work behind
30:58the scenes? Like do you have different
31:00sections providers? What happens? Yeah,
31:02so a lot of that is driven by your
31:04config AI file
31:07and that's where you specify I don't
31:08even know if that's on this page but we
31:10definitely talk about it kind of like
31:11multi-provider and failover and stuff
31:14but similar like in your Laravel app if
31:17you use Laravel you probably know you
31:18have like a config database file or a
31:20config file systems file. When you
31:23install the AI SDK you get a config AI
31:26file and that's where you configure
31:27okay, I want my default AI provider to
31:30be open AI or I want it to be anthropic
31:33and you can actually customize that
31:34based on the types of things you're
31:36doing. So you can have a default
31:37provider for text like our prompting
31:39here but you can have a different
31:40default provider for images. So maybe
31:42for images I want to use Gemini and Nano
31:44banana but for text I want to use
31:46anthropic.
31:48And so if you specify that in your
31:49config file you actually don't have to
31:51specify it here when you actually
31:53prompt. You know it's just going to use
31:54whatever defaults you have. Of course we
31:56can override that in the code but
31:59it's kind of nice you know you can just
32:01sort of like use it you don't have to
32:02remember okay
32:03what is the like code name for Claude
32:05Haiku - 401 whatever that is.
32:09402 405 yeah. Yeah, exactly. You can
32:12just use the
32:13use the features.
32:15Yeah, and it seems like that's the
32:17eloquent way in a lot of ways where like
32:18you're building out database queries in
32:21the same way where I could be building
32:22in SQL light in local production or
32:25local instance and then when I put it
32:27into production none of my code changes
32:29just my
32:29>> Yeah, yeah it's basically the same
32:31concept almost like a ORM type concept
32:34concept for uh AI providers.
32:36I love it.
32:38What about context? What's that?
32:40Yeah, so context is kind of how you
32:41actually would build something more like
32:44a chat where, you know,
32:46what what's kind of interesting about
32:47like dealing with LLMs is when you ask
32:50LLM for some text,
32:52it, you know, ChatGPT and like Claude
32:55give the illusion that there's this
32:56ongoing state, you know, with the LLM
32:59where it's like aware of all the
33:01previous messages, but that's not
33:03actually the case by default. You have
33:05to feed it the whole string of messages
33:09that came before your prompt so that it
33:11knows, and you have to do that every
33:13time, actually, that you call the LLM.
33:15It has no concept of like historical
33:17interactions with you.
33:19Yeah. Um so this is where you do that.
33:21So this is where like if you are
33:22actually are building more of like a
33:24chat-like interface where the user is
33:26sort of like having an ongoing
33:27conversation with an agent, you need to
33:30store those messages in a database, and
33:32then this is where you can pull them
33:33back out and give them to the LLM. So
33:35we're going to in this example, we're
33:36like pulling all the recent messages for
33:38a given user, maybe the most recent 50
33:41in this case, and we're passing them
33:42sort of in chronological order
33:44uh to the LLM. So this is an example of
33:47how to do that manually, but the tab
33:49you're on now kind of shows us this is
33:51built actually into the AI SDK, so
33:54like the previous tab showed, we allow
33:56you to do that manually, but if you use
33:57this remember's conversations trait like
33:59you have here,
34:00the AI SDK will actually just do that
34:02for you. Um
34:04And the AI SDK actually is in beta, and
34:06I would say this is one of the areas
34:08that I'm like most working on between
34:10now and stable release. There's some
34:12kind of more features around this, but
34:14the the the kind of like bare-bones
34:16skeleton of it is here.
34:18Where if I use this remember's
34:19conversations trait, it's going to
34:21automatically store conversations in a
34:22conversations table. It's going to put
34:24the messages in like an agent messages
34:26table, and it's going to automatically
34:28pull them back out when you continue the
34:30conversation.
34:31It's just super nice because it's a lot
34:33of kind of like boilerplate code to
34:35store those messages, get them back out,
34:37make sure they're sorted in the right
34:38order, pass them to the LLM. So, to just
34:41be able to use this trait in like one
34:42line of code is like, oh, I just have
34:43like conversation state. Um that's
34:46great. Yeah. To be able to not have to
34:48do uh any of this and store yourself and
34:51then you're in this aspect of it. It's
34:53just kind of obfuscated for you.
34:55Yeah. I guess there was a question that
34:57I saw in the chat that was, what was the
34:59most challenging part of building
35:00Laravel SDK? Is this it or is there a
35:02different part?
35:03No, I mean, a lot of it was kind of like
35:06I I wouldn't say there was any one area
35:07that was just like, oh, wow, this is
35:09just uber hard. I would say it was a lot
35:11of just like sitting and thinking about
35:13what are the right like APIs and like,
35:15how do I want to design it? Kind of what
35:16we were talking about earlier of like, I
35:18really like to design things that feel
35:19like really thoughtful and really like
35:22really like well-designed and easy to
35:24use and intuitive. And the process of
35:28arriving there is usually just like a
35:29lot of sitting there and staring at the
35:31screen, you know, like being like, or
35:33even just like writing pseudo code of
35:34like, oh, what would be like
35:36what would feel the best, you know? And
35:38sometimes I'll actually go into like an
35:39empty file and just start typing code as
35:42if I'm using the SDK, even code that
35:44just doesn't even exist to try to
35:46discover like, oh, this feels nice or
35:48this doesn't feel nice and
35:50um so, yeah, I I mean, to get back to
35:52the answer, I don't think it was any one
35:54particular thing. It was just figuring
35:56out what's the most like delightful
35:58considered APIs to give people. Yeah. I
36:02love that. And to your to your point
36:03before where you're like, you were kind
36:05of
36:06or the the posed question, does it
36:07matter? Like, I think it that that means
36:10it does matter in a lot of sense because
36:12it it's becomes this unified platform
36:15where you know what's available to you
36:17even if you might not be handwriting
36:19every single piece of the code, you know
36:21what's available and it feels unified,
36:23which I think is the the biggest thing.
36:25Yeah, absolutely.
36:26Jason, you mentioned uh Jason Torres,
36:28okay, fine, I'll go and install it. You
36:30had your uh Santa app that I that was
36:33hosted on Laravel Cloud. I'm curious of
36:35if you swap out the AISDK, it'd be cool
36:37to see.
36:38Um all right, Taylor, what is uh tools?
36:42I know we kind of have talked about this
36:43a little bit on the streams before when
36:45it comes to like MCP and what's the
36:47difference between tools, stuff like
36:48that. What is tools in the context of
36:49the AISDK?
36:51Yeah, tools are one of the coolest
36:53parts, I think, of building agents and
36:54they kind of give you the most
36:57uh interesting possibilities for the
36:58kinds of things you can build because
37:01um it lets you like basically
37:03supercharge LLMs with things that they
37:06couldn't do otherwise. So, and this
37:08example is a very basic example, but
37:10I'll I'll share some other cool examples
37:12in a second where maybe you want the LLM
37:14to be able to generate like a
37:15cryptographically secure random number.
37:18I don't think that's actually built into
37:19LLMs, so like if I ask ChatGPT, "Hey,
37:22give me a number between 1 and 100."
37:24Yeah. That's probably not actually a
37:26random number in the true like
37:28mathematical sense of the word.
37:30But I can actually give
37:32um using the AISDK, I can define a tool
37:35that when I prompt the LLM, I give it a
37:38list of tools that it has available to
37:40it. So, it's going to see, "Hey, I have
37:41this tool that can be used to generate
37:43cryptographically secure random
37:45numbers." And it can invoke that tool,
37:47which will then invoke this code on your
37:50end, this handle method that's like
37:51right in front of us, will actually be
37:53called. Uh but the LLM will have
37:55actually decide to call it. I mean, it's
37:57pretty it's pretty wild, actually. Um
37:59and so, a lot of times if you look at
38:00like model evals when OpenAI or
38:03Anthropic comes out with new models, it
38:06will actually have like an eval for tool
38:08use and how good it was at using tools.
38:11You know, and LLMs these days are are
38:13pretty good at this.
38:15So, this lets us actually augment the
38:18LLM with cool stuff.
38:20So, I mean, a real-world use case for
38:23tools that probably a lot of us are
38:24familiar with is if you've ever used
38:26Claude code or open code or cursor,
38:29you'll see that it like um
38:31it inspects files. Like it reads files,
38:33it writes files, it runs bash commands.
38:36Those are actually tools
38:38that the builders of Claude code and
38:40open code have built, right? So, they
38:42have like a run bash command tool that
38:45they have given to the LLM to run bash
38:48commands. And so, actually the first
38:50time when I demonstrated the AI SDK out
38:52in San Francisco a few weeks ago, I
38:54actually wrote like a little tiny nano
38:56version of Claude code. I think I
38:58actually called it nano code. Where I
39:00gave it like seven tools. I gave it like
39:02a read file tool, a write file tool, or
39:04a bash tool.
39:05And it actually worked. Like you could
39:07chat with it, it would update files. You
39:09could say, "Hey, make a new eloquent
39:11model and update the routes to like
39:13return all the data using the model."
39:15And it like actually works.
39:17Um so, tools let you do all sorts of
39:19random stuff. And um I think we'll get
39:21into a second. We actually give you some
39:22tools built in. Um but this is how you
39:26can like
39:27start to integrate the LLM with your
39:29system.
39:31>> [snorts]
39:31>> Yeah. No, I love I love that. And that's
39:33that's essentially what uh Boost is
39:34doing a lot of ways, too. Like you
39:35mentioned like Claude code, but Boost
39:37has has
39:38um a specific number of tools that Would
39:40you say then, Taylor, tools are a great
39:42place to to keep the things that your
39:45app likely does over and over or that
39:48your LLMs through your app are doing
39:50over and over again?
39:52Yeah. Yeah, I think so. Um and some of
39:54the tools we give you are are some
39:56common tools that people like those are
39:58like search. So,
40:00um
40:01go to provider tools. What do we have
40:02here?
40:03I think this is Yeah, the This is web
40:05search, web fetch,
40:06uh and then file search. So, like
40:09people use tools a lot for this kind of
40:10thing. Um so, if I throw like the web
40:12search tool or the web fetch tool onto
40:14this agent, and these are two tools that
40:16are just built into the AI SDK. You
40:18don't need to write them yourself. It
40:20will allow the LLM to actually search
40:22the web for the latest data or fetch a
40:24web page from a given URL. And then with
40:27file search, it will allow to search
40:28like vectorized PDFs or other data to
40:32find things that otherwise didn't have
40:34access to. So if we have maybe uploaded
40:37I don't know all of our company policy
40:40PDFs to a file store, we can now search
40:42this and have like a chat agent that can
40:45search over this data.
40:47Um and that's data that the LLM
40:48otherwise wouldn't have access to.
40:51I love it. Uh cool kind of question
40:53around these lines. How should we
40:55organize our tools? I guess when you're
40:56building out an application with this,
40:59what do you think are the thoughts of
41:00one, how many tools MC can or should we
41:03use?
41:04Um and then how should we organize our
41:06tools? I guess
41:07uh you know
41:08the SDK gives you a way to organize them
41:11within this tools namespace. But what
41:13are your thoughts on this?
41:15Yeah, I I know it's been like kind of a
41:18pretty hot topic of discussion around
41:20like the amount of tools that you should
41:21expose to
41:23to an LLM. I I would say like
41:26you know
41:27when you you shouldn't have like 50, 60,
41:2970 tools probably exposed to the LLM
41:32because you get what's called context
41:33bloat where
41:35kind of what I was saying where you have
41:36to send all of the messages to the LLM
41:38to provide historical context. You also
41:40have to send all of the tool
41:41definitions, what they do, and you have
41:44to send that on every message as well.
41:45So if you have too many tools, you can
41:47sort of consume a lot of tokens and a
41:49lot of context. Um
41:52I think that really comes into play on
41:54some of these local coding agents where
41:55you start to plug in like different MCP
41:58providers and they have maybe a dozen
41:59tools and you've got like maybe five of
42:01those and now you've got like a hundred
42:02tools.
42:03And uh it can kind of get pretty out of
42:05control pretty fast. I think if you're
42:07using the AI SDK to build something in
42:08your own application, you probably
42:11aren't running into that quite as much
42:12where you have like a hundred tools and
42:14something that's probably pretty rare.
42:15So um
42:17Yeah, I mean I I don't know what the
42:19exact number is, but it is a thing you
42:21have to watch out for. Let's put it that
42:22way. Yeah. Yeah, I am curious. I
42:25actually don't know the answer to this,
42:26but I am curious of if if there's going
42:28to be a way or if there's already one
42:30way to be able to say
42:31to to conditionally
42:34load up tools would be would be very
42:36interesting. I know that there's
42:37probably And what's cool with the AI
42:40SDK, you can do that a little bit in the
42:41sense of like
42:43>> [clears throat]
42:43>> you have the agent class and so whatever
42:44you return from that tools method can be
42:46sort of like determined at runtime.
42:49Okay. So maybe maybe one user maybe you
42:51maybe you pass it a user into the
42:53constructor of the agent and they're
42:54using that user to say, okay, well, this
42:56user has access to this list of tools
42:59and then maybe this other user doesn't.
43:00So you can do a little bit of like
43:02dynamic tool registration here.
43:04I know people are sort of working on
43:06sort of like other things around tool
43:08discovery.
43:09Um
43:10So yeah, it's pretty ongoing I think
43:11evolving topic in the AI world. Yeah, so
43:14that does make sense that that
43:17a great use case for like a tools array
43:20on a particular user would be maybe
43:22they're a paid user and would have
43:24access to specific tools that you might
43:26not have for free users for example.
43:28>> Exactly. Yeah.
43:30Awesome. For those just joining again,
43:31thanks so much for being here. We have
43:33Taylor Atwood talking through the label
43:34AI's SDK. We'll get to actually building
43:37things and like what that actually looks
43:38like practically here in a little bit,
43:39but I want to give a good overview bare
43:42bones for those watching in on demand of
43:44what it is, what do you how we think
43:47through different pieces of this because
43:49there's so many cool options that you
43:51can build with this.
43:52Even one app probably would only touch
43:54two or three of these. It's really hard
43:55to find a singular app that could have
43:57all of these things provided. If you do
44:00have questions, be free feel free to
44:01drop them in the comments. I am checking
44:03it as much as possible. I can't promise
44:04that we'll get to all those questions,
44:06but uh glad to have all of you watching
44:08here. Feel free to like and subscribe to
44:09help us uh show more content to you in
44:12the future, as well. Uh so, Taylor, what
44:14is structured output? So, I've heard of,
44:17you know, we kind of talked about in
44:18prompts that you get a text back when
44:21you're saying, "Hey, give me this." What
44:22is the difference between that and
44:24structured output?
44:26Yeah, so when you prompt an agent sort
44:27of by default, um you just get sort of
44:30free-form text back in the response. So,
44:32very much like you would get from like
44:33ChatGPT. You know, it doesn't really
44:36have like a defined structure or schema
44:38to it. You're just getting back a
44:39paragraph or two of text.
44:42Um what structured output lets you do is
44:44actually say, "Okay, when you respond to
44:46me, I want you to respond in this
44:48structure." Um and I think if you scroll
44:50down a little bit, you can see like in
44:51this case, we're saying, "Give me back a
44:53score, and that needs to be an integer,
44:55and it's like a required field." So,
44:57when the LLM responds, it's going to
44:58give you JSON that matches that schema.
45:02Which is super convenient, because then
45:03you can parse that JSON, you like know
45:06what to expect in terms of what data is
45:08on it, and then you can reliably store
45:10it or do whatever you want. If you're
45:12just getting back free-form text from
45:14LLM, like
45:15you know, how do you parse that? I don't
45:17know. It's It's just It's just arbitrary
45:18text, you know? But, um if you're
45:20getting back JSON, you know what to
45:22expect. And this is actually powered by
45:24an
45:25um a new Laravel component, the JSON
45:27schema component, which you can see
45:29there in like the type signature.
45:31Um so, we actually used this on the MCP
45:34package um to define JSON schemas for
45:36your MCP tools, and it's used here. So,
45:39um
45:40we actually wrote this component,
45:42because we needed it, you know, in the
45:43AI SDK, in the MCP package.
45:46Uh but, it's a really slick little
45:47component for defining JSON schemas.
45:51Yeah. I remember the first uh
45:53application, actually the first Laravel
45:55application I ever made was an AI
45:57application where I was They they we
45:59didn't have structured output at the
46:00time. This is about like 3 years ago. I
46:02think it was GPT-2 or something like
46:04that. Uh but I had to
46:06hope and pray that the agent responded
46:09in the way where I said, "Okay, here's
46:11use the response, put equals equals
46:12equals." And then then separate it, and
46:14then I had to parse it out from that. Uh
46:17structured output makes that so much
46:18easier. Yeah, it's like I yeah, I I mean
46:20even on Anthropic's models up until
46:23recently, like to get it to return
46:25structured output, you were literally
46:26telling it in the prompt, "Please
46:28respond with this JSON structure."
46:30>> [laughter]
46:31>> It's pretty pretty wild. Yeah, it's
46:33crazy times that we we we live in now
46:36where we could we could have it be
46:37exactly the way we want it. And so the
46:39the way the AI SDK does that makes it
46:41extremely extremely easy. Um a couple
46:43questions that I wanted to uh someone
46:45said for using SDK, we need to purchase
46:46the multiple model subscriptions like
46:47Gemini and Claude Code. Uh so Claude
46:49Code would probably be like your local
46:51one, and then Gemini, if you wanted to
46:53use that within the AI SDK, whether
46:55that's returning text responses or even
46:58um you know, image generation, uh
47:01multiple models would have to be added
47:03as API keys to the config locally and in
47:06production. Uh but I'm I'm correct,
47:09Taylor, where you can use like let's say
47:11you're building something locally and
47:12just want to test it, you can use local
47:14models through Ollama. Is that correct?
47:16>> that's correct. Yeah, we we released
47:17that pretty quickly after launch.
47:19Another thing you can do if you don't
47:21want to sign up for multiple model
47:22subscriptions is use something like Open
47:24Router where you sign up for Open Router
47:26and you have one key, and then you can
47:27actually access Anthropic models, uh
47:30OpenAI models, all all the models
47:32through one APIs or through one sort of
47:35like paid service.
47:37Uh but yeah, you can also just use local
47:38models. I actually did that myself the
47:40other day just testing stuff. You can um
47:42download Ollama and then pull in some
47:45kind of local model, whichever one you
47:46want, and you can um just run AI stuff
47:49locally.
47:50Yep, so that's Ollama, o l l a m a. Um I
47:55think actually Open Router, now that you
47:56mentioned it today, I think they also
47:58have like a uh uh of course they have
48:00like a paid one where you can pay them
48:02and they'll route it to different
48:03things, but I think they also have a a
48:05free where they'll route it to like
48:07providers as well.
48:08Which is really really interesting.
48:10Um
48:13I saw another one okay, you mentioned
48:15embeddings
48:17um and vector search. Can it replace
48:19LangChain as an example and can the
48:22tools be a rag replacement?
48:24Yeah, a lot of the tools kind of can be
48:26a rag replacement. As far as it can it
48:27replace LangChain, you know, it probably
48:29depends on kind of what you're doing
48:30with LangChain, but a lot of the ideas
48:33behind some of the built-in tools are
48:34around sort of like
48:36rag and similar concepts in the sense of
48:38like vectorizing files, searching them,
48:41doing similarity search across data
48:43using tools, um things like that. I
48:45think that's like just like a super
48:47common use case for tools in general.
48:49Um so yeah, there's opinions on that
48:52kind of built into the AI SDK because it
48:54is so common.
48:56Yeah,
48:57um and then uh
49:00there is
49:02uh we'll we'll get to that question in a
49:03little bit. Uh
49:05I will jump back in. What what is
49:07streaming? So
49:09Yeah, so streaming is just a really easy
49:11way. So instead of calling like the
49:12prompt method on the agent, if I call
49:14stream, super easy way to send back a
49:17stream or a stream of text to the front
49:20end so that, you know, like in the same
49:22way as when you interact with chats
49:24online, how the text sort of like
49:25streams in line by line as it's
49:27available. The stream method in the AI
49:29SDK makes that super easy. So this uses
49:32uh ACTP server sent events um called SSE
49:37for streaming text back as it's
49:39available. And then in this example, you
49:41can see we're using this kind of then
49:43method. We can actually do something
49:44once it's all been streamed. If there's
49:46some sort of like either follow-up
49:49action we want to do or something, I
49:51don't know, some arbitrary code we want
49:53to run after the stream is complete, we
49:55can do that here.
49:57But then on the front end in our
49:58JavaScript or um using Livewire or
50:01something, we can actually consume that
50:02stream to like show text as it as it is
50:05available so that you don't have to wait
50:07for
50:08uh the entire response to be generated
50:10before you see anything.
50:11Yeah, that is really nice part of of it
50:13being so succinctly tied in with the
50:16front end. I I like that. Uh for
50:18example, within View and React, we have
50:20the use stream hook. Uh but then
50:22Livewire has the the wire stream field
50:24to take all of this and and make it look
50:26like you would expect within Chat GPT
50:27where it's kind of streaming line by
50:29line.
50:30>> Yeah.
50:30Um what what about async? This is this
50:33is interesting to me cuz I've I've I've
50:35I've heard a lot of cool things about
50:37it, but I'm I'm not quite sure how it
50:38how it actually works behind the scenes.
50:40Yeah, so this is another example of how
50:42the S AISDK is sort of like uh leaning
50:45into a lot of Laravel's full stack, you
50:47know, batteries included opinions where
50:50um instead of calling prompt or stream,
50:52in this case we're calling queue or
50:53we're actually
50:55basically queuing a prompt. So we're
50:56saying, "Hey, we want to call the LLM
50:58with this prompt, but we want to do that
51:00in a queue job in the background because
51:03we don't necessarily need or want to
51:05wait for the response right now.
51:07Um we just need to queue this to happen
51:08in the background and then we'll do
51:10something with the response later."
51:12So when you call this queue method, it's
51:13actually going to create a background
51:15job on your Laravel queue to respond to
51:18this prompt and then the then and the
51:20catch callbacks will be invoked when the
51:22response is ready or if something goes
51:24wrong respectively.
51:25>> Yeah. Um so once we get the response, we
51:27can store it in a database or send an
51:29email or whatever we we want to do with
51:31that response. Or if something goes
51:32wrong, we can handle that as well in the
51:34catch callback. But basically, you know,
51:37this would be used where
51:39uh the user user submits something that
51:41we want to analyze with AI, but we don't
51:44really want the user to have to sit and
51:45wait for it to be done right then.
51:48So we we want to return a response to
51:50the user like, "Hey, we're processing
51:51this. We'll let you know when it's
51:53ready." or whatever. And then
51:56that happens really quickly or I
51:57basically instantaneously and in the
51:58background this job is running and then
52:01we'll do something. We'll notify the
52:02user maybe when it's ready or when it's
52:04done or whatever.
52:05This makes it super simple to do that.
52:08Yeah. So I would I would say like off
52:11the top of my head it seems like a good
52:12use case for this would be
52:14maybe transcribing a 100 megabyte 200
52:17megabyte audio upload where you don't
52:20want them to wait for a full request
52:22until they get the response for that.
52:23That's awesome. I love that. A question
52:25on the async stuff from Lynn. Is the
52:27async stuff non-blocking? Does it use
52:29the react PHP PSR-18 browser?
52:32Yeah, this async stuff is more like
52:35queue related.
52:36Um
52:37so it doesn't really it doesn't really
52:39have anything to do with like async in
52:40the sense of like concurrency or like
52:43you know
52:44go co-routines or anything like that.
52:47It's more about just like
52:49putting something on your background
52:51queue and then doing something with it
52:52later.
52:53Yeah.
52:55Um I think I saw another question. Yes,
52:58the SDK is compatible with the Ollama
53:01API. I would assume yes as well,
53:04correct? Yes, I think so. Yeah. Yeah,
53:06and you can even
53:08there's a new I think we just documented
53:10this the other day maybe two days ago.
53:12There's a in your config file you can
53:14actually override the base URL for any
53:17provider. So you can actually use any AI
53:19provider that is like OpenAI compatible
53:22or Ollama compatible
53:24just by overriding that base URL.
53:28Yeah.
53:29Question here that we we kind of glossed
53:31over here at the top I'm curious to hear
53:33what how how this was kind of built
53:35Taylor and then also what your thoughts
53:37are in terms of
53:39I know that there's also a use cheapest
53:41model trace uh and stuff like that. So,
53:43in terms of automatic failover, uh and
53:46then how does all that work behind the
53:48scenes?
53:49Yeah, so, um yeah yes, you can do this.
53:52And it pretty much works how um this
53:54person is asking, where you pass
53:56multiple providers and models to the
53:59prompt method or either uh or specify
54:01them in the provider attribute on the
54:03agent itself, where
54:05if one provider is either overloaded or
54:08you are rate limited on that provider,
54:11um it will actually just fall back to
54:13the next provider in the list that you
54:15give it. So, maybe you try Anthropic
54:17first. If Anthropic is overloaded or
54:19you're just rate limited on Anthropic,
54:21it will fall back to OpenAI, for
54:23example.
54:24Um we do that by checking like the
54:25response codes from the AI providers
54:28that we get back, so we can see like,
54:29oh, they're you know, overloaded or we
54:32get like a I think it's like a 429 HTTP
54:34code if you're rate limited or something
54:36like that. And then we can fall back to
54:38the next provider, and we actually raise
54:39an event that you can listen for to know
54:41that you got uh you had a failover. And
54:44we're working on bringing a lot of this
54:45event visibility into Nightwatch, so
54:47that you're going to be able to see, you
54:49know, like, how much tokens am I using
54:50on which providers, and how long are my
54:52tool calls taking, and things like that.
54:54Um anyway, so that that's kind of like
54:56model failover. And then uh you
54:59mentioned kind of like some of the
55:00attributes you can put on agents, like
55:02use cheapest model, use smartest model.
55:05I kind of wanted to have this nice way
55:07to just like annotate an agent with use
55:11cheapest model, for example. And
55:13sometimes you have tasks, like it's
55:14basic text summarization or something
55:16like that, where you don't need like
55:18Anthropic Opus 4.6 for that, right? It's
55:21it's like Yeah, it's like $100, yeah.
55:24Yeah, exactly. So, if you're just trying
55:25to summarize like a paragraph of text in
55:27the one sentence, for example,
55:29um you can just use like Haiku.
55:32So, the benefit of putting use cheapest
55:33model on the agent is, like let's say
55:35tomorrow a new cheapest model comes out.
55:38Mhm. You don't really have to go look,
55:40okay, like, what is that model? What is
55:42Let me copy the ID for it, like, paste
55:44it into my code. If you're just like,
55:45use cheapest model,
55:47and you just like, composer update to
55:48get the latest AI SDK, you're just like,
55:50always on Yeah. the latest cheapest or
55:52smartest model. So, it's just kind of
55:53nice. You don't really have to update
55:55your code if you know that this agent is
55:57doing something that's not very, like,
55:59doesn't require a lot of intelligence.
56:01Yeah. No, I I love that. Uh cuz it's one
56:03of those things similar to what you had
56:05mentioned at the beginning where you
56:08having the one package that makes it
56:10easier and more eloquent in the sense of
56:12you're not having to you you have those
56:14opinions already formed for you when it
56:16comes to cheapest, when it comes to
56:17smartest, uh even when it comes to
56:19automatic failover for
56:21uh those are things that you probably
56:22mostly want when you're building a
56:23full-stack application. You just don't
56:24want to do it yourself in a lot of ways.
56:26>> Yeah, totally.
56:28Uh lastly, what about attachments?
56:30This is interesting.
56:31>> Yes. Yeah, so attachments kind of come
56:33up in a few places throughout the AI
56:34SDK. So, in this example, we're using it
56:36when prompting and doing text
56:38generation. So,
56:40you know, maybe we're giving it I think
56:42in some of my demos I've given it like a
56:43CSV of sales leads or maybe you're
56:46giving it a PDF of a transcript and
56:48you're saying, "Analyze this this
56:50document or parse it or do something
56:52with it." So, we just make it super easy
56:54to give attachments in various places.
56:56So, you can do it here with text. We
56:58also actually let you do it when you're
56:59generating images. So, you could give it
57:01um you could prompt the model, "Hey,
57:03here's an image and I want to um make it
57:07like cartoon style or like a painting or
57:10I want to change something about it,
57:11like, basically remix an existing
57:13image." Um you can also use attachments
57:15for that as well.
57:17Um so, yeah, it's super easy to attach
57:20and upload files and and kind of
57:22interact with them. Yeah, here's a
57:23here's a uh image example. Yeah, or
57:25maybe you're asking, "What's in the
57:26image?" So, I can give it an image and I
57:28can combine that even with like
57:29structured output to like analyze images
57:33and generate um structured output that I
57:36can do something with.
57:39I love that. I love that. Uh quick
57:41question. Uh I'm going to close out this
57:44screen or or stop sharing my screen real
57:47quick just so we can uh answer this
57:48question. I'm going to pull up a demo
57:50that I built so we can walk through it
57:52in a practical sense before we and then
57:54we maybe if we still have some time we
57:55can build something from scratch cuz I'm
57:57curious to see uh how Claude code also
58:00works with the AI SDK and to get gaining
58:02to show everyone there. But question
58:04says now that we have the Laravel SDK,
58:05what's next? Laravel's focus on ASP is
58:07exciting and I think it's the right
58:08stuff for it. Are there any future
58:10plans? So I guess
58:11what You kind of mentioned the the the
58:13whole
58:15um
58:16uh remembering agent being one of your
58:18things that you're focusing on. What do
58:19you think is after that?
58:22Um I've got a actually got a list of
58:23things that I that I'm working on
58:26um sort of like post beta and even after
58:28the stable release. So yeah, I've got a
58:30list of features I want to keep working
58:32on for sort of the conversation storage,
58:34context retrieval, all of that stuff. I
58:37mean it's stuff like some basic stuff
58:39like pruning of the
58:41historical conversations or compaction
58:43of the history and sort of like
58:45summaries and snapshots and things like
58:46that. There's some stuff around tools
58:48and sort of agent loops that I want to
58:51look at. So like human uh tool approval
58:53is a good one. So like say you want the
58:56LLM to be able to invoke like a refund
58:58customer tool but it needs human
59:01approval if the refund amount is greater
59:03than $200.
59:05Um so like stuff like that. It's kind of
59:06these more advanced complicated tool use
59:09cases.
59:10Um and then there's other agent stuff
59:13like uh agent loops.
59:15Um
59:15you know, more customization of when the
59:17agent actually stops doing things. This
59:19is sort of gets into like some of the
59:22uh you know, you've seen people have
59:23these like so-called Ralph Wiggum loops
59:25of where you can
59:26have like infinite loops of agents doing
59:28things. So, a lot of like agent loop
59:30configuration and stuff like that. Also,
59:33want to build like a cool artisan
59:34command like um
59:36artisan agent chat. So, where like if
59:38you have an agent and you just want to
59:39like chat with it maybe to see how it's
59:41behaving or what it's doing. I can just
59:43come into the command line and do agent
59:45chat and then I can just like pick an
59:46agent from a list using Laravel prompts
59:49and then just chat with it and see like,
59:51"Okay, is it behaving like how I would
59:53expect, you know?" Uh
59:55and it'll almost like the the DD of you
59:58know, like agent debugging [laughter]
1:00:00in a way.
1:00:00>> The AI DD in a in a lot of ways. That'd
1:00:03be that'd be really interesting. I
1:00:04almost imagine then you could have its
1:00:07own agent MCP where your Claude code or
1:00:10LLM whatever is kind of can interact
1:00:12with the agent and talk to each other.
1:00:14Yeah. And then there's a there's a you
1:00:16know, there's already a bunch of PRs
1:00:18open from the community and one of the
1:00:20ones I think that is most requested or
1:00:23most on everyone's minds is uh so-called
1:00:25like sub agents. Um and one easy way to
1:00:29think about this is almost like the
1:00:31ability to expose another agent as a
1:00:33tool
1:00:35from
1:00:37another agent. So, like imagine I have a
1:00:38you know, one primary agent and in its
1:00:41tools method, maybe I don't only return
1:00:43tools I can also return other agents.
1:00:45So, it can actually call on, "Okay,
1:00:48you are sort of like an an orchestrator
1:00:50agent and you can call on maybe this
1:00:52planning agent or this research agent or
1:00:54this other type of agent that is more
1:00:55specialized to do things." There's
1:00:57actually a PR open for that right now.
1:00:59Um that I need to review. Hopefully, I
1:01:01get it reviewed today and but at least
1:01:04get kind of get out that sub agent
1:01:06behavior this week. But, a lot of it
1:01:08will be driven by the community, you
1:01:09know, I think there's like dozens of
1:01:10pull requests already out there and
1:01:12that's kind of how it always goes in the
1:01:14open source world, you know, like
1:01:16I I'm driving. I have one hand on the
1:01:18wheel but the community also has a hand
1:01:20on the wheel and you know, we're we're
1:01:22hopefully getting to a good destination.
1:01:24Yeah, co-piloting together. Yeah,
1:01:27exactly.
1:01:27>> great place. Uh that there kind of
1:01:30similar you kind of answered a couple
1:01:31questions in the chat I sense of can you
1:01:33call an agent from another agent doesn't
1:01:35sound like it's possible.
1:01:36>> Yeah, that's it. That is something
1:01:37that's coming
1:01:38in terms of agent orchestrate
1:01:39orchestration and everything like that.
1:01:42Again, thanks so much for everyone being
1:01:44in the chat. I wanted to talk I want to
1:01:45walk through a demo that I actually
1:01:46built and I'm curious to get your
1:01:48thoughts Taylor on what could be what
1:01:51could be added to this but then we can
1:01:53jump in and maybe build something from
1:01:55scratch as well.
1:01:57But thanks so much everyone for for
1:01:58joining. If you're just joining or maybe
1:02:00you've been here the last 10-15 minutes
1:02:02haven't heard me do this spiel yet. We
1:02:04are with Taylor talking about the AI SDK
1:02:08that was just released
1:02:10last Thursday, Friday, Thursday.
1:02:13Um and so it's only been out for four
1:02:17days.
1:02:18Um and so it's been crazy be able to see
1:02:20how many people are already building
1:02:21with it, how many like Taylor said how
1:02:23many new PR's have already been added.
1:02:26But just walking through the
1:02:28possibilities of it as well as just
1:02:30talking through why it was built in the
1:02:32first place. So if you're here, let us
1:02:33know you're here. You have any questions
1:02:35we'll try to get to them as best as
1:02:36possible.
1:02:37Yeah, I wanted to talk show this little
1:02:40demo that I built. It's actually live
1:02:44right now. I'm not going to tell
1:02:45everyone the the URL just because I
1:02:47don't I don't want to crash while we're
1:02:49on on on stream but I'll share it at the
1:02:51end
1:02:52so that you can have it as well as a
1:02:54GitHub repo. And I'll put it in the
1:02:56description if you're watching this in
1:02:59in review. But this is a little I wanted
1:03:01to try to find a way to build using
1:03:04almost everything or almost as much of
1:03:06the AI SDK as possible. Yeah.
1:03:09And the premise for this was I always
1:03:12save a lot of links that I always forget
1:03:16what those links are. Bookmarks have
1:03:17never worked for me.
1:03:19Um even most AI tools have never worked
1:03:22for me because it only saves like the
1:03:23link and maybe meta description. Doesn't
1:03:25necessarily save any of my thoughts on
1:03:28it or anything like that. And so I want
1:03:29[clears throat] to build a thing using
1:03:31the AI SDK that either
1:03:33uh takes a file that I upload, maybe a
1:03:35screenshot, a document, or a URL
1:03:39and then have it upload to this vector
1:03:42store. So we're actually using um the
1:03:45vector storage
1:03:46uh
1:03:48uh part of the AI SDK to auto chunk that
1:03:51embed. And this is one cool thing that I
1:03:52actually didn't know. I'm curious I'd
1:03:55get your thoughts on like what's the
1:03:56difference between storing something in
1:03:59vector using OpenAI's vector storage
1:04:01versus
1:04:02putting this in Postgres PG.
1:04:05>> Yeah. Yeah, this kind of came up in a
1:04:07Laravel idea. I mean, they are
1:04:09the end goal is really similar, I would
1:04:12say, in the sense of like we want to
1:04:13store embeddings about things and then
1:04:16search them or query them.
1:04:18What is cool
1:04:19I mean, I guess let's start with like um
1:04:22the PG vector stuff. So like
1:04:24in the AI SDK docs, we show you how to
1:04:26take a string of text and generate
1:04:28embeddings for it. And then you can put
1:04:29that in a vector column in a Postgres
1:04:32database. Now, as long as it has the PG
1:04:34vector extension, which um most local uh
1:04:37Postgres things do, like the Herd
1:04:38version and then Laravel Cloud's uh
1:04:40Postgres has it.
1:04:42Um and then you can run queries on that
1:04:44to find
1:04:45um things that are similar to your
1:04:47query. So like if I to go back to our
1:04:49Mexican food example, which we both
1:04:51like, like if I stored like um
1:04:55you know, restaurants that have great
1:04:57cheese dip and then I search for good
1:05:00appetizers, those things are
1:05:02semantically similar even though they
1:05:04don't actually have the same words
1:05:06contained within them. They're similar
1:05:07ideas. And so that that that's the
1:05:09purpose of semantic search is to be able
1:05:11to pull back things that are related.
1:05:13So, you can actually store those
1:05:15embeddings in your own Postgres
1:05:16database, or you can actually upload
1:05:20files or data to OpenAI or Gemini, and
1:05:23they will vectorize and store them for
1:05:26you. So,
1:05:27it's you know, it's just sort of like
1:05:29one of the convenient things about
1:05:30storing them on OpenAI or Gemini is you
1:05:32can just hand them a PDF, they will
1:05:34extract all of the text and vectorize it
1:05:36for you out of the PDF and all of that,
1:05:39you know, like doing that all locally,
1:05:41you're going to have to pull the text
1:05:42out of the PDF, you're going to have to
1:05:44vectorize it.
1:05:46Um, you're going to have to store it.
1:05:48Um, so it is kind of convenient just to
1:05:49be able to like throw files at OpenAI or
1:05:51Gemini and say, "Hey, like vectorize
1:05:52these for me. I don't want to even have
1:05:54to think about it." Yeah. It's just you
1:05:56know,
1:05:57it's two different ways of doing similar
1:05:58things. Yeah. Yeah. No, I I love that.
1:06:02Um, and so yeah, this this works in the
1:06:04sense of the there's a couple cool
1:06:06things that I really love is the how
1:06:07easy it is to take an attachment, let's
1:06:09say an image, and grab specific things
1:06:12out of this using an AI agent to say,
1:06:14"Okay, I have this image. Now I just
1:06:16want like I I save a lot of screenshots
1:06:19either on my phone or just like I'm I'm
1:06:20grabbing things and usually I have like
1:06:22a directory on my device. I wanted a
1:06:24place to be able to save all of those,
1:06:25but then also be able to query them in
1:06:27later things." So, again, the goal of
1:06:29this is one to kind of scratch my own
1:06:31itch and build with the AI SDK, and then
1:06:34two, be able to show as much as possible
1:06:36off of the AI SDK. So, I want to grab
1:06:38we'll go to we'll go to Laravel News. I
1:06:39want to find like a quick
1:06:41uh little link that I might be able to
1:06:43save. Um, so let's go ahead and maybe
1:06:46we'll see
1:06:49see here like OpenAI releases GPT-5.3
1:06:53Codex. I'm going to grab this link and
1:06:56go back to my URL here, and if I just
1:06:59paste this in, I can add a specific
1:07:02comment about this URL, or I can just
1:07:03capture this. And this is one of those
1:07:05uh async things that's happening then on
1:07:07in the background where um I can still
1:07:10you know go to different links
1:07:12uh but it's still being stored and
1:07:15retrieved in the background uh looks
1:07:17like because it's uh because it's a
1:07:20JavaScript front end I this this failed.
1:07:22So, here we're going to upload a
1:07:23screenshot instead. I'm going to go back
1:07:25to that link and I'm just going to
1:07:26upload we'll we'll zoom uh we'll zoom
1:07:29out here.
1:07:31And I'm just going to paste a quick
1:07:33screenshot.
1:07:36Just of this.
1:07:38There we go. We'll go back to here.
1:07:40We'll go paste this in and upload and
1:07:42analyze. So, what should be doing within
1:07:44the uh AI SDK
1:07:47is it's going to grab the process using
1:07:51the vision to say, "Hey, let's describe
1:07:53what is actually happening in this
1:07:55image." And not even just transcribing
1:07:57it, but the cool thing about that vision
1:07:59SDK is like you get additional aspects.
1:08:02You can send a specific prompt in with
1:08:04vision to say, "Okay, I actually want
1:08:06these particular things." If we're
1:08:08building a research library, I want to
1:08:10know what's pertinent to this. You can
1:08:12see here it says it's uh
1:08:14a detailed description of the image.
1:08:16Screenshots,
1:08:17uh top header, uh large rectangular card
1:08:20with rounded corners and a soft drop
1:08:22shadow. All all of the neat things that
1:08:24you might want from image. Uh it's a lot
1:08:25more than I actually thought it would be
1:08:27from just one one simple image.
1:08:29But the neat part about this is uh kind
1:08:32of everything that's happening in the
1:08:33background is a couple people mentioned
1:08:35uh
1:08:36"What would you recommend for front end
1:08:39things?" Some people said polling versus
1:08:41broadcasting. What are your thoughts,
1:08:42Taylor? And I'll tell you say what I did
1:08:44in this particular demo. Oh, man.
1:08:47Um I like there's something
1:08:50>> would be a good answer question, I
1:08:51guess. There is something just like
1:08:53really nice about for simple stuff just
1:08:56throwing like a wire pole on a div and
1:08:59it just like works. [laughter]
1:09:02So, if you don't have anything else
1:09:04Yeah, if you don't have tons of users
1:09:06like or maybe it's just a tool for
1:09:08yourself even, you know, like you're the
1:09:10only user,
1:09:11which I think in the age of AI we're
1:09:13seeing more sort of just like personal
1:09:15software, right? I'm just like tools
1:09:16people are creating for themselves that
1:09:18are really intended just for them. I
1:09:19think throwing like some polling or
1:09:21especially with Livewire I mean it's so
1:09:22easy with wire polling. I guess Inertia
1:09:24has polling as well now since Inertia 2.
1:09:27I think that's nice, but if if you're in
1:09:28like uh more of a production use case
1:09:30like let's say like Laravel Forge or
1:09:32Laravel Cloud's dashboard where there's
1:09:34hundreds of users in there at a time,
1:09:36wire polling can be a little bit more
1:09:38inefficient since you're kind of like
1:09:40continually hitting that back end
1:09:42whether there's data ready or not, you
1:09:44know. Um
1:09:45broadcasting kind of gives you more of
1:09:46like a push mechanism to where you're
1:09:48just pushing data as needed so it's a
1:09:50lot more efficient. Yep. Yeah, in this
1:09:53particular demo I did use broadcasting
1:09:55with with the use like with with Reverb
1:09:58and then like the use Echo
1:10:01hooks to watch for that so that way
1:10:03when we're
1:10:05you know, when we're grabbing something
1:10:07from the web or we're up uploading a
1:10:09screenshot for example, if I go into
1:10:11here and I just say something like uh I
1:10:14don't know. We'll just grab this from
1:10:16the from the AI SDK. And if I go into
1:10:19and and actually paste it, it's
1:10:21happening behind the scenes, yes, but
1:10:23once it finishes, you'll see that happen
1:10:26live. So, this is one of the things that
1:10:27of course like Taylor said, you could
1:10:28use this with wire pull where it's just
1:10:31constantly pinging the server
1:10:33every 5 seconds or whatever and it will
1:10:35update automatically, but this is using
1:10:38specifically for this demo, it's using
1:10:40the vector storage to store all of this.
1:10:42Once it does read it from the vision,
1:10:46and then we're getting a structure
1:10:47output so that when we chat with it and
1:10:50you can see that kind of popped up right
1:10:51there.
1:10:52Now we can chat and have all of those
1:10:56things and if I pop over to this chat,
1:10:58this is where we have a couple
1:10:59additional features that we kind of
1:11:01already walked through with the AI SDK.
1:11:03We have conversational to say, "Hey,
1:11:05here's like
1:11:06um
1:11:07remembering those particular agents.
1:11:10Uh but then we also have uh streaming
1:11:12set up as well as file search and web
1:11:14search. These are the two tools that me
1:11:16personally tell you I Those were like
1:11:18incredibly easy and I loved having them
1:11:21to not have to uh I I I've I've built a
1:11:24couple AI tools where I'm relying on
1:11:27uh maybe my maybe, you know, web search
1:11:31within PHP, for example, instead of
1:11:35giving AIs the tools to be able to do it
1:11:37themselves. Yeah.
1:11:39Uh I'm curious of like what were your We
1:11:41kind of talked about in tools, but what
1:11:43was your thoughts on those three tools,
1:11:44the file search, web search, and um the
1:11:47image search?
1:11:48Uh file search, web search, and web
1:11:51fetch are the
1:11:53three built-in Well, I guess there's
1:11:54also uh similarity search we haven't
1:11:56really got into, but Yeah, I mean, I
1:11:58wanted to give people um you know, I
1:12:00think these are just like a incredibly
1:12:02common use cases. Um
1:12:04web search and web fetch are of course
1:12:06used to sort of overcome the training
1:12:09knowledge cut off for the LLMs where we
1:12:11can actually go out and fetch relevant
1:12:13data. I actually use
1:12:16this type of functionality in LLMs a lot
1:12:19when I'm maintaining the Laravel docs.
1:12:21And
1:12:22um
1:12:23for or when I'm writing the docs for a
1:12:25new framework feature. So, one thing I
1:12:27will do is go into the docs, fire up
1:12:28Claude code, and say, "Hey, can you
1:12:30write docs for this PR?" And I will like
1:12:32paste in the PR URL on GitHub, and it
1:12:37works like incredibly well, actually.
1:12:39So, it's super convenient to be able to
1:12:41give the AI the ability to fetch web
1:12:43pages in a variety of scenarios. Yeah.
1:12:46No, I love that. I love that. I'm just
1:12:49going to ask you a question about
1:12:51um let's see.
1:12:53Uh maybe maybe we'll say
1:12:55what was that latest model
1:12:58from open AI?
1:12:59We should see it's search the specific
1:13:03things that it already has pertain to my
1:13:04local user and we should see those tool
1:13:07calling that should be made here. So
1:13:08it's searching the knowledge base
1:13:11and then it found you know the open AI
1:13:13model mentioned is GPT 5.3 codex in our
1:13:15saved notes. So this is a use case of
1:13:18particular conversational API that we
1:13:20kind of talked about.
1:13:22In the sense of it's remembering the
1:13:23things if I was to ask
1:13:25without having that context of something
1:13:27like
1:13:30what
1:13:32And you even had the tool call thing was
1:13:33that
1:13:34you even had the tool call thing like
1:13:35show up to us this really fancy. That
1:13:38was that was a fun I I I didn't know
1:13:40that that was possible
1:13:42because I've only ever worked with chat
1:13:44where it was just returning text so
1:13:46returning the
1:13:48when it returns and I could be wrong
1:13:51or or misremembering but I would I think
1:13:53when you return the structured output
1:13:57it open AI specifically which is what
1:14:00the provider I'm using with gives
1:14:03says when it's calling particular tools
1:14:05as a JSON so it's really easy to parse
1:14:07that within you stream is that correct?
1:14:09>> Yeah yeah yeah when you stream stuff you
1:14:11get sort of like tool call start tool
1:14:13call end you get like you know these
1:14:15various events for when tools are being
1:14:17used.
1:14:20Yeah I'm curious
1:14:22actually haven't tested if I can search
1:14:24on the web then
1:14:25of like what what link is that
1:14:28associated to
1:14:30on the web I think I marked this down so
1:14:32it doesn't actually go go on call but
1:14:34I'm just curious now
1:14:35>> [laughter]
1:14:35>> if that
1:14:37if that does it couple questions
1:14:40some people said I didn't
1:14:42I said there it is using a web search
1:14:43tools.
1:14:46Okay it'll blog post we'll call that.
1:14:48>> something yeah.
1:14:49>> Yeah, we'll we'll call that good. Open
1:14:51AI open AI blog post, which is probably
1:14:53what Laravel News is referencing. So,
1:14:55we're we're good there. Uh some people
1:14:58said I I made it easy to guess the URL.
1:14:59Yeah, I'm sorry. I mostly I did not
1:15:01scale I I think I'm using a very small
1:15:05Laravel Cloud server for this.
1:15:07Uh so, if it if it crashes, that's not
1:15:09Laravel Cloud's fault. That's my fault
1:15:11for not scaling it up. Um What did you
1:15:14use to build this, Josh? Is this a Is
1:15:15this a Claude code special or is this
1:15:17cursor or what? Is this Is this all
1:15:19handwritten? This is mostly Claude code
1:15:21uh with some handwritten stuff for the
1:15:23for the chat. So, I did figure out
1:15:25quickly uh what stuff uh
1:15:28Claude code doesn't seem to do a great
1:15:30job um with and the use stream stuff of
1:15:34uh it took me took me a while to figure
1:15:36out that uh it returns the JSON for
1:15:39structure for for streaming
1:15:41um and being able to
1:15:42kind of push that back and forth. I was
1:15:44trying to do some
1:15:46uh I whenever Claude code goes out and
1:15:49tries to build like five different files
1:15:51and I'm like, "No, this should probably
1:15:52just be like one option in the Laravel
1:15:54AI cuz I know it's a lot cleaner than
1:15:56that." That's when I start having uh
1:15:58some second guesses. So, most of this
1:15:59was built um within Claude code with
1:16:02with me referencing by hand each
1:16:04particular AI
1:16:06capability and being able to prompt to
1:16:08saying, "Hey, I know that the AI SDK can
1:16:10do this. Let's make it do that." Right.
1:16:15Um someone has said, "In an AI first
1:16:17future, do you see Laravel becoming an
1:16:19orchestrator of AI agents rather than
1:16:22just an HTTP framework?" I'm curious
1:16:24your thoughts on that.
1:16:25>> Yeah, I mean, I could definitely see
1:16:26that. If there's one thing I've learned
1:16:28in like the 15 years of maintaining
1:16:30Laravel is that people will use it for
1:16:32all kinds of things I didn't expect. So,
1:16:34[laughter]
1:16:34um
1:16:35yeah, you know, I mean, like I think
1:16:37people building AI agents and like AI
1:16:40features is going to be the thing for a
1:16:42long time for the foreseeable future.
1:16:44So,
1:16:45I think it's you know, it's good for
1:16:47Laravel to sort of have this good
1:16:49first-party
1:16:50option there.
1:16:53I love it.
1:16:54Um why don't I fire up uh
1:16:58um
1:17:00maybe a Claude code instance and we can
1:17:03create a Laravel application from
1:17:05scratch and maybe Taylor I'll have you
1:17:08talk through with me of of how you might
1:17:11maybe
1:17:13help start from scratch and and I'd be
1:17:15curious to see your your thoughts on
1:17:18what are the things you're kind of
1:17:19looking for to make sure everything is
1:17:21working the way what what kind of
1:17:22revisions you you look at. Um the idea
1:17:24that I have
1:17:26and we can maybe
1:17:27kickstart this a little bit more. The
1:17:29idea that I have mostly because I have a
1:17:30fallback to show what I built right
1:17:33before this just in case things don't
1:17:35work out. But the idea that I have is
1:17:37we maybe upload some markdown files
1:17:40about us or about the user as an
1:17:43individual and then we can use 11 Labs
1:17:46API as a voice
1:17:49to be able to talk
1:17:51to it, transcribe our voice, get an
1:17:53answer, then have it talk back to us.
1:17:56Okay. Yeah, that sounds good. Um so, let
1:17:59me go ahead and
1:18:01share my screen again real quick.
1:18:04Uh so, the for those still in the chat
1:18:07with us,
1:18:08what's uh
1:18:11uh what's your what's your favorite
1:18:13drink of choice maybe while you're
1:18:15coding? I know Taylor just took a swig
1:18:17of a Coke Zero.
1:18:19Uh so, who who else is Coke Zero fans
1:18:23or what what else is your your
1:18:25particular drink of choice? How many
1:18:27Coke Zeros have you consumed so far
1:18:29today, Taylor?
1:18:30Uh I think like three. I drink I'm I
1:18:33drink a lot of Coke Zeros. So, that's
1:18:36I I saw it research on it that like
1:18:38everyone talks about like oh you know
1:18:40you know
1:18:41diet Cokes are still bad for you and
1:18:43Coke Zeros are still bad for you and all
1:18:44this.
1:18:45Um but then uh
1:18:47some research said you had to you had to
1:18:49have I think somewhere around 40. Yeah,
1:18:52>> 40 or something Coke Zeros before it
1:18:55even before you even get to the required
1:18:58limit which is like a thousand percent
1:19:00lower than what the actual limit is or
1:19:02something like that.
1:19:02>> Yeah. Yeah, it's pretty wild.
1:19:06Um
1:19:08Taylor, what IDE do you use? I'll say
1:19:10that before you Oh, I'm still using a a
1:19:13lot of Sublime Text, you know. I'm kind
1:19:15of interested to see where like the IDE
1:19:18space goes like as a concept, you know.
1:19:21I think you even see this with Cursor,
1:19:22right? With them like kind of leaning
1:19:23more into their agent mode as like the
1:19:25default view even for booting up the
1:19:28IDE.
1:19:30And it's like, you know, um
1:19:32previously when we were like handwriting
1:19:34so much code, I remember when I was
1:19:36first starting in my like career using
1:19:38Visual Studio to write .NET and the IDE
1:19:41was just so incredibly important in
1:19:43terms of like the intelligence it gave
1:19:45you and the shortcuts and it was kind of
1:19:47built for like handwriting code
1:19:49productively, right? Yeah. And now as we
1:19:51move into a future where like
1:19:53handwriting code to some extent starts
1:19:55to feel painful just as a concept of
1:19:57like why would you handwrite all of the
1:19:59code? I'm curious to see like how the
1:20:01IDE arena
1:20:03sort of like evolves with like, you
1:20:05know, VS Code and and PHP Storm and you
1:20:07know, Sublime Text is more of almost
1:20:09more of an editor than an IDE, but
1:20:12um I'm still using Sublime Text paired
1:20:14with uh Claude Code and Open Code.
1:20:17I love it. I'm I'm similar not not with
1:20:19Sublime, but I'm using Zed editor with
1:20:22paired paired with Claude Code um
1:20:25and Open Code mostly because I I I kind
1:20:27of like the when I'm writing handwritten
1:20:30stuff, I don't want like even
1:20:31autocomplete maybe a little bit of
1:20:33auto-complete I'm okay with.
1:20:34>> Yeah.
1:20:35But most of the
1:20:35>> I'm kind of the same way. Yeah. I find
1:20:38Zed and Sublime like
1:20:40Zed is the closest thing to Sublime that
1:20:41like I've ever used that's not Sublime.
1:20:43You know, I find them so similar.
1:20:46I've not I've never used Sublime. I
1:20:47think I did when I was first writing
1:20:50HTML files but I didn't learn it enough
1:20:52to when it stuck cuz I was using VS Code
1:20:54for the longest time.
1:20:55Um
1:20:56People in the chat want water. We got
1:20:59Coke Zero. A lot of water.
1:21:01Boring.
1:21:02>> [laughter]
1:21:03>> Water for code. Pepsi Zero. I'm a
1:21:05villain. Yeah, that I don't know what he
1:21:08>> Really?
1:21:08>> [laughter]
1:21:09>> I
1:21:10Sometimes I'm at like a
1:21:11>> or Pepsi Zero?
1:21:13Uh I
1:21:15I don't know if I've even had Diet
1:21:16Pepsi. I've only I think I've only had
1:21:17Pepsi Zero. But sometimes you're just in
1:21:19a Pepsi establishment, right? And you
1:21:20just like don't have a choice because
1:21:22it's like a Pepsi household. Um so I've
1:21:24definitely had it.
1:21:25>> [laughter]
1:21:26>> That's true. I do like Dr. Pepper Zero.
1:21:28That is like Yeah, that is good. I agree
1:21:30with that. That's good. That is that is
1:21:32one of my specialties.
1:21:33We got Tim Horton's Steep Tea, some
1:21:35lemon iced tea.
1:21:37I'm I'm I'm thrilled at how many Coke
1:21:39Zeros there are in the chat. I love
1:21:41that.
1:21:42And I actually went to the store this
1:21:44morning after the kids went to school
1:21:46cuz I was out of Coke Zero. And so I
1:21:48pull into the parking spot and like as
1:21:50I'm getting out there's this lady
1:21:52loading her car next to me and she is
1:21:53loading that car with Coke Zero and I'm
1:21:55like
1:21:56this this is one of my people. Right?
1:21:59>> [laughter]
1:21:59>> Full size Cans too?
1:22:01Uh yeah, they were actually bottles.
1:22:02They were like big plastic bottles of
1:22:04Coke Zero. I almost like said something
1:22:06but I figured she'd just be weirded out.
1:22:09Why do you care about my Coke Zero? I'm
1:22:12curious if you've ever had the thought
1:22:13of of
1:22:15putting Coke Zero maybe like getting a
1:22:17whole bunch of 2 liters putting it on
1:22:18tap and and doing that or if you just
1:22:21like can all the way.
1:22:23Uh I mean I I like the can but I mean
1:22:26some places have like a really good tap,
1:22:28you know, like to me McDonald's they
1:22:30have like some kind of magic tap where
1:22:32it's just like the soda comes out
1:22:34so much better than it does other
1:22:35places, but Oh, yeah. Yeah.
1:22:38But to me to me actually the peak is
1:22:40like glass bottle Coke Zero, which is
1:22:42not easy to find to be honest. Um
1:22:46but they do have it and I've I've found
1:22:48it occasionally. More in Europe actually
1:22:49than in the United States, but a glass
1:22:52bottle Coke Zero that that really hits
1:22:53different.
1:22:55I I I maybe we'll get it have have this
1:22:58happen at this year's Laracon. By the
1:23:00way, Laracon US tickets are out live
1:23:03right now. We're going to Boston this
1:23:05year laracon.us.
1:23:06Uh but I I pictured this that last
1:23:08Laracon. Maybe we'll get it have it
1:23:10happen this one. I want to do Have you
1:23:12ever done like those blind tasting
1:23:14things? But we'll do Taylor
1:23:16blind tasting Coke Zero behind like
1:23:18glass, can,
1:23:20uh McDonald's Coke Zero or McDonald's
1:23:22Diet Coke and have you figure it out.
1:23:24Yeah, that'd be fun actually.
1:23:27So we'll we'll we'll put it now that
1:23:28it's in the atmosphere. We'll we'll put
1:23:30it out there.
1:23:31Um let's let's do a new let's do Laravel
1:23:33new.
1:23:35Okay. Um I'm going to call this uh
1:23:38the Larvis agent like Laravel
1:23:41Jarvis.
1:23:42Okay. Um
1:23:44and uh we'll we'll go ahead and we'll
1:23:46make this uh React just for simplicity.
1:23:49We'll go no authentication scaffolding
1:23:52uh past How do you like to do Laravel
1:23:54new, Taylor? Like what's what's your I
1:23:56was just thinking we need a Laravel new
1:23:58dash dash I was just thinking we need a
1:24:00Laravel new dash dash Taylor that just
1:24:02like it's all my own opinions. Ooh,
1:24:05that'd be awesome.
1:24:05>> usually
1:24:07Oh man, this is a good one. I mean it
1:24:08kind of depends on what I'm doing. I do
1:24:09like the blank starter kits like you
1:24:11picked here with like just no
1:24:12authentication scaffolding for stuff
1:24:13like this.
1:24:15Um
1:24:16and then uh you know, I usually just use
1:24:19the defaults like past or whatever. Mhm.
1:24:22Yeah. Yeah, I I I I know if you were
1:24:24dash dash react and then dash dash cuz
1:24:26you can put all that into one
1:24:28>> Yeah, I don't really do that. Yeah, I
1:24:29don't I never do that. Yeah, do you do
1:24:31you do the empty empty prompt then
1:24:34in terms of like the
1:24:36the Laravel new and not even any name
1:24:39after? I usually do specify a name. So,
1:24:41I'll do like Laravel space new space
1:24:43something. So, like I
1:24:45I actually forgot that you you can
1:24:46actually leave off the app name, I
1:24:48guess. It'll just ask you. Okay, I kind
1:24:50of forgot about that. But, I usually I
1:24:52usually provide the app name. Yeah, I I
1:24:54provide I provide the app name as well.
1:24:56But, yeah, one of the things I've
1:24:58learned with when I'm
1:24:59scaffolding something out with
1:25:02with LLMs like Claude code for example
1:25:04is
1:25:06the blank no
1:25:09dashboard and no authentication almost
1:25:11is easier because otherwise it tries to
1:25:14confine it into the dashboard sometimes
1:25:16and from a UI standpoint it looks a
1:25:18little then I just have it you know, I I
1:25:20usually do manually like cutting that
1:25:23out and clearing it up. So, I I like the
1:25:24I like the blank side.
1:25:26Um
1:25:27we will Um I'm not going to use the
1:25:30heard MCP. I'm going to use Claude code.
1:25:34Um
1:25:35and then I'm curious Taylor, what are
1:25:36your the ways if you like let's say
1:25:39you're scaffolding something out with
1:25:41with LLM, you're not jumping too much
1:25:42into the code to start until you get to
1:25:44a certain point, whatever that might
1:25:46look like. Um do you install packages
1:25:48first? Do you do any kind of like PHP
1:25:51artisan model creates or how do you
1:25:53What's your way of going about this? Uh
1:25:55yeah, that's a good question. Sometimes
1:25:57I will put a little bit of light
1:25:58scaffolding in place first and I did
1:26:00this as well as I was even just building
1:26:02the AI SDK where
1:26:04>> [clears throat]
1:26:04>> I would actually just scaffold out a few
1:26:06like sometimes even empty classes of
1:26:08like, "Hey, here's kind of like the
1:26:09structure of how I want things to look."
1:26:12Um so, maybe I'd define like an
1:26:13interface or a few classes just to kind
1:26:15of like
1:26:16put some kind of like rails in place,
1:26:18you know, as far as okay, here's kind of
1:26:20the shape of what we want this app to
1:26:22look like and then
1:26:23let the model kind of like, you know,
1:26:25paint between the lines from there.
1:26:27Yeah.
1:26:28Yeah, I kind of go I kind of go back and
1:26:29forth myself. Most of the time I I
1:26:31install all the packages that I want and
1:26:33set all the packages up first and then I
1:26:37get and usually then like any kind of
1:26:38like maybe tables, migrations that I
1:26:41know I specifically want and then I jump
1:26:43in. Uh yeah, I found that Yeah, that's
1:26:46the way I like to do it. It feels like
1:26:48I'm I am setting the the course, for
1:26:51example. Right. Yeah.
1:26:52>> Um
1:26:53Let's go ahead and install the Laravel
1:26:55AI SDK.
1:26:56>> [snorts]
1:26:56>> And then we uh
1:26:59we
1:27:00uh I believe we publish uh
1:27:03Yeah. Um
1:27:06I always forget the
1:27:07Yeah. We need to publish and then we can
1:27:08search for it.
1:27:10Yep.
1:27:12tags
1:27:13Yeah, I got it. Yeah, service provider.
1:27:16Oh. Okay, yeah, I think that works.
1:27:17Yeah. There you go. That's just the
1:27:19Yeah, that should be all we Oh, we
1:27:20actually need to publish the uh
1:27:23uh the
1:27:28Let's go into the docs. We actually need
1:27:29to publish the
1:27:32uh
1:27:32>> We need the migrations. Yeah. Yeah. So,
1:27:35we'll go ahead and do that as well.
1:27:39Perfect.
1:27:41I'm going to hide my screen and input um
1:27:44some environment variables cuz that's
1:27:45usually what I would do next. So, I'm
1:27:47going to
1:27:48do this mostly just
1:27:51I'm going to delete them anyways right
1:27:52after this, but just in case I don't I
1:27:55don't uh
1:27:56don't pull those up anywhere.
1:27:58Um
1:28:01So, we're going to use 11 Labs.
1:28:05And we're going to use uh probably
1:28:08OpenAI SDK cuz it makes it a little bit
1:28:11simpler.
1:28:14I'm going to copy this.
1:28:19Perfect.
1:28:28All right.
1:28:39And so, Terry, you said that you had
1:28:40used uh Claude code mostly in like open
1:28:43code. Do you Do you do uh like the
1:28:45dangerously
1:28:48Pretty much, yeah.
1:28:50>> all the time or do you Do you Do you
1:28:51watch it?
1:28:52>> much.
1:28:53Um
1:28:54No, I pretty much do this and to be
1:28:56honest. Yeah. Uh here's a Here's a quick
1:28:59tip that I actually didn't know
1:29:01until like the last couple of of weeks
1:29:03for those For those who do use Claude
1:29:05code, um if you go to and bypass or
1:29:08dangerously skip permissions, you can
1:29:09still do the shift tab to go into plan
1:29:12mode, for example. Um without going back
1:29:15and And you can go back into bypass
1:29:16permissions on. I I do that mostly cuz I
1:29:18always I always like to do plan mode um
1:29:21just to get things up and running. Yeah,
1:29:23I do too, especially for bigger
1:29:25features.
1:29:26So, I'm going to do uh PHP artisan
1:29:29um migrate and we're going to make sure
1:29:31I'm on the right one. Um
1:29:37Perfect. And so, we're going to in
1:29:41Claude code, we are going to build
1:29:44uh an application where we can upload or
1:29:49uh
1:29:49have some local markdown files.
1:29:53And using the Laravel AI SDK, we are
1:29:58going to
1:30:01take those markdown files and generate
1:30:05or and have uh an assistant be able to
1:30:09answer questions using the voice mode.
1:30:14So, uh when we
1:30:16talk to the assistant in an extremely
1:30:21simple
1:30:23UI
1:30:24uh they can transcribe
1:30:28our uh voice with voice-to-text from AI
1:30:32SDK,
1:30:34uh pass that to
1:30:36uh an agent to be able to generate an
1:30:40answer based on
1:30:43our markdown files.
1:30:45And then
1:30:47uh generates a response using the voice
1:30:53from
1:30:54AI SDK.
1:30:57So, I'm curious what was the difference
1:30:59between the Larvis directory and the
1:31:00Larvis agent directory? Um oh, are we in
1:31:03Larvis? Oh. Yeah, we're in Larvis.
1:31:05That's That's the This is This is the
1:31:07one that I already did.
1:31:09>> [laughter]
1:31:09>> Yeah, cancel that. Yeah, we'll cancel
1:31:11that one out.
1:31:12Uh we'll go into We'll go to Larvis
1:31:14agent.
1:31:15There we go. Thank you.
1:31:18Uh I already I already did this mostly
1:31:21because I wanted to something to show at
1:31:22the end if we either didn't have enough
1:31:23time or or if or if anything did. So,
1:31:26um
1:31:27but it's a it's a good uh good for those
1:31:31uh
1:31:31uh I'm curious then for those in the
1:31:33chat who are using LLMs. What is the
1:31:35thing maybe one or two things that you
1:31:37do
1:31:39or that you found that have been helpful
1:31:41maybe with building in Laravel or
1:31:43something that you always like to do
1:31:45even if you are having AI do most of the
1:31:48work. Like, what's something that you
1:31:49either look at, something that you make
1:31:51sure you review?
1:31:53Uh Taylor, when it comes to like
1:31:54reviewing the work or testing it out,
1:31:56what's your first thing maybe like after
1:31:58you run a prompt even plan mode, what's
1:32:00the first thing that you like go to
1:32:02double-check in the code?
1:32:04I kind of pull up
1:32:06um
1:32:07I pull up the whole like diff in the
1:32:09GitHub desktop
1:32:11actually.
1:32:11>> Okay. And that way it's just like it
1:32:13it's really easy visualization of every
1:32:15file that changed and I can really
1:32:16quickly just kind of scan through them
1:32:18and see what's going on.
1:32:20Um
1:32:22and then I will usually clean up a
1:32:24variety of things. It might be like
1:32:26really nitpicky things like method order
1:32:28or where I put a method in the class. I
1:32:30kind of like organize methods logically
1:32:32or in a certain way that I like. Um I I
1:32:35haven't really reliably been able to
1:32:37automate that with like a with an AI
1:32:39guidelines or anything. And then like
1:32:42um
1:32:43I'll just kind of continue the
1:32:44conversation with the AI, you know, if
1:32:46like maybe sometimes it um
1:32:49it duplicated something or I was like,
1:32:51"Hey, you could actually have a more
1:32:52general abstraction around this and like
1:32:54make this file and extract this here and
1:32:56then use it both places." I'll kind of
1:32:58coach it along
1:33:00um or try different things, but um
1:33:03yeah, that's kind of how I approach it.
1:33:04I don't use a lot of like
1:33:07really let's call them like fancy AI
1:33:09workflows in the sense of like I usually
1:33:11have like one or two Claude code tabs
1:33:13open. I'm kind of working on a couple
1:33:16maybe one or two things at most kind of
1:33:18specifically
1:33:19um at a time, you know, I have I haven't
1:33:21really dug too far into like I've got
1:33:23you know, 10 different Claude codes
1:33:26churning on in parallel-wise task on the
1:33:29same code base without them like running
1:33:31into each other and I haven't really
1:33:33experienced that or like found a need
1:33:34for that, but um
1:33:36you know, I'm pretty simple guy I guess
1:33:37when it comes to the
1:33:39AI usage and how I use it.
1:33:41Awesome.
1:33:43Um so then have you jumped into the the
1:33:45open claw stuff yet? So for those who
1:33:48don't know, you can run open claw on
1:33:49Forge right now. It's super simple to
1:33:51get it set up. Have you jumped into it,
1:33:52Taylor, or no? I I've only like
1:33:54scratched the surface of it, you know, I
1:33:56have got an open claw like up and
1:33:58running when I was playing with the Ford
1:33:59stuff. I I can't say that I'm like super
1:34:02deep into it like automating my entire
1:34:04life, but I do love the concept. I think
1:34:07like I actually wrote my own Telegram
1:34:10chatbot for AI
1:34:13along like year or two ago
1:34:15just because I find like Telegram or
1:34:17like WhatsApp like the super convenient
1:34:19sort of just like interface for talking
1:34:21to AI on the go. Uh mine was just a
1:34:24simple chatbot. It had no other like
1:34:26functionality that open claw has, but I
1:34:28think it's a really cool project. You
1:34:31know, I'm curious like how it evolves
1:34:32and where it goes from here, but pretty
1:34:34crazy how it's like you know, got
1:34:36everyone buying Mac minis all over the
1:34:38place.
1:34:41Awesome.
1:34:42We're doing some plan modes. Where do
1:34:44the
1:34:45markdown files live? Let's go resources.
1:34:48docs. I'm curious Taylor, you get your
1:34:49thoughts on like if you're doing
1:34:50anything locally.
1:34:52Like would you do storage app? Seems
1:34:54like AI loves storage app. I don't know
1:34:56what it is about that. So are these
1:34:57these are files that is this files that
1:34:59the user is going to upload? Uh maybe
1:35:02either user or if this was just meant to
1:35:03be deployed locally. Like let's say you
1:35:06>> Okay. If it's just going to be deployed
1:35:08locally, I think I would go resources.
1:35:09docs. I think that makes sense.
1:35:12Uh
1:35:13direct attachments. We'll probably do
1:35:15that instead of a instead of a
1:35:17file search.
1:35:19Let's do that. Um and then we can text
1:35:23to speech. Let's do 11 labs.
1:35:25And we'll do we'll do a single session.
1:35:27It's a super super simple super super
1:35:29generic. Um I forgot to publish the
1:35:33um
1:35:34the skills for the AI SDK. So before it
1:35:37jumps into actually implementing this,
1:35:38I'm going to
1:35:40cancel out and do that. We have some
1:35:41questions um in the chat. If you're
1:35:43still here, thanks so much for being
1:35:44here. Taylor's been awesome to be able
1:35:46to answer some questions. And now we're
1:35:47just having some fun
1:35:49building with it as well. But some great
1:35:51questions uh specifically for Taylor
1:35:53about AI. I had a feeling Taylor didn't
1:35:55like AI much. What made him change his
1:35:58mind? What was the turning point?
1:36:00I
1:36:01uh I never have actually been anti-AI or
1:36:04against AI. I think um you know,
1:36:07when coding agents and AI kind of first
1:36:10got started, there was this big um
1:36:14wave of like vibe coding as a concept,
1:36:16right? It sort of like dominated the
1:36:18discourse for, you know, a chunk of
1:36:202025. It was this concept of vibe coding
1:36:23where you're like not even reviewing the
1:36:25code.
1:36:27And I think um I don't relate a lot or
1:36:30maybe don't
1:36:32um for like production work, let's say,
1:36:34or for like real work that I'm going to
1:36:37like give to customers, I don't relate
1:36:38to that way of coding as much. I think
1:36:40that's a fun way of coding for like, you
1:36:42know, personal projects that are sort of
1:36:43low stakes and you actually don't care
1:36:45about the code quality.
1:36:48But, you know, what we've seen is just
1:36:49like over time as the models get better,
1:36:51I think like the quality of the code has
1:36:53gotten so much better. And I'm actually
1:36:55curious if you've seen like any
1:36:57I know Opus 4.6 just been out not even,
1:36:59I guess, a week yet.
1:37:01Are you seeing like any big improvements
1:37:04from like Opus 4.5 or has it not been
1:37:07long enough to really even judge?
1:37:09>> [laughter]
1:37:10>> Yeah, I don't think I have seen too many
1:37:13big improvements in the sense of like I
1:37:15haven't used the new Codex as much.
1:37:18I think the biggest improvements that
1:37:20I've seen from both of those, just in my
1:37:22gut reaction and feel, has been more in
1:37:24like the the thinking through something
1:37:27before doing it. I feel like I I'm I am
1:37:30telling it no, that was wrong. Let's
1:37:32think about this Let's do it this way
1:37:34instead much less.
1:37:36And I don't know if that's just because
1:37:37it's using skills better, it's using
1:37:39context better, or it's just smarter.
1:37:41It's always hard to tell in that aspect.
1:37:44You know what I'm sort of like
1:37:46>> [snorts]
1:37:46>> I curious to see where AI coding goes in
1:37:48the sense of like I I've seen Dax and
1:37:50open code kind of say similar things
1:37:52where like
1:37:53I almost feel like even if the models
1:37:55only got marginally better from where
1:37:57they are now at writing code
1:37:59if they got like let's let's say they
1:38:01got like 10% better but they got 10x
1:38:04faster and like actually doing it I
1:38:07actually think the speed for me is like
1:38:08what I crave even more than like more
1:38:12intelligence at this point. You know
1:38:13what I mean? Like I find it for most
1:38:15tasks that I'm asking them to do
1:38:17they're intelligent enough to do because
1:38:19I don't I don't I'm not doing rocket
1:38:21science, right? I'm like building web
1:38:23apps and pulling stuff from a database
1:38:24and it's sort of like crud interfaces.
1:38:27But if they could do it like 10x faster
1:38:30like for me that would be like such an
1:38:32even bigger unlock I feel like and then
1:38:34like you know like here we've been
1:38:35waiting for 6 minutes but what if it
1:38:37only took it like 15 seconds to do all
1:38:39of this? Like that would be incredible
1:38:41you know. And hopefully we get there
1:38:42over the next few years.
1:38:45I'm going to run boost install again so
1:38:47when we scaffold it out Laravel new we
1:38:49installed boost we ran it but if we run
1:38:50it again because we installed Laravel AI
1:38:52SDK it's going to actually ask for us
1:38:54right here to do these third-party AI
1:38:56guidelines which I'm going to enter. Um
1:38:59and make sure that our code here because
1:39:04we're clearing context and bypassing
1:39:05permissions it should use the skill now
1:39:08that has access to it as well.
1:39:10Uh but it might not at the same time.
1:39:12I'm always curious if I think we have to
1:39:14close out of it and then open it back up
1:39:15again for new skills.
1:39:17Uh we have um some people in the chat
1:39:20are probably know a little bit more
1:39:21about me in that particular aspect. Um
1:39:25uh question about when you were creating
1:39:26the AI SDK what did you want Did you
1:39:29want to try and create a competitor to
1:39:30LangChain and and that suite of code or
1:39:32I guess
1:39:34your thoughts
1:39:34>> I sort of um
1:39:36I sort of approach things pretty fresh.
1:39:39Um
1:39:40I'm not really like a heavy user of
1:39:42LangChain or LangChain you know like or
1:39:43anything like that Uh um
1:39:46and I've built a lot of things in
1:39:47Laravel this way for better or worse to
1:39:49be honest over the years where I just
1:39:51sort of like come at the problem with
1:39:54no really preset opinions or like uh
1:39:58baggage or like kind of historical
1:40:00experience around it even and just sort
1:40:02of look at it all fresh and based on my
1:40:04own like opinions and use cases trying
1:40:06to build something that I think is
1:40:07interesting. Um and I try to put like
1:40:12when I build a new package like the AI
1:40:14SDK or even other features in Laravel, I
1:40:16try to build just a good foundation that
1:40:19I can feel confident about and release
1:40:20into the community for like further
1:40:23iteration you know so like
1:40:25in in V1 of the AI SDK, I don't
1:40:27necessarily set out to solve every
1:40:29single um conceivable problem around
1:40:31developing with AI. I try to give like
1:40:33okay, here's a good foundation and like
1:40:36um some scaffolding that we can start
1:40:38from as a community and like have a
1:40:39conversation about and then iterate from
1:40:42there over a period of months and then
1:40:44and you know eventually years um to
1:40:46where things can evolve from there but I
1:40:48try to put like a foundation in place uh
1:40:51to start from.
1:40:53Yeah.
1:40:54No, that's awesome.
1:40:56Um you mentioned like the fast code. Uh
1:40:58have you tried uh someone in the chat
1:41:00said Kimmy 2.5. You tried Kimmy 2.5 with
1:41:02this stuff?
1:41:03>> It is quite fast, yeah.
1:41:05And I think it's a glimpse of like man
1:41:07how like life changing OBS things get
1:41:09even faster.
1:41:11Yeah, and I'm with you though in the
1:41:13sense of like most of the time unless
1:41:15I'm doing like some kind of MVP like the
1:41:18initial kickstart of a project, I might
1:41:20be using something like Cod Code
1:41:22um with with Opus 4.5 4.6 something like
1:41:25that or Codex
1:41:27um or a big feature but most time if I'm
1:41:29just like wanting to nail out certain
1:41:32things um either I'm
1:41:35hand coding it within Zed and I'm I'm
1:41:37just tweaking you know UI design or
1:41:39tweaking like how how I I like parts of
1:41:41the code or I'm just using something
1:41:44like Kimmy 2.5 or something very quick
1:41:46because I like the feel of it being Yes,
1:41:48it's good. I'm I'm asking questions
1:41:50about my code base. I want it now kind
1:41:52of thing. Yeah.
1:41:56Um so you use the Git desktop UI. Is
1:42:00that new or have you always been using
1:42:02that? I really only use it for a few
1:42:05things. So like I don't use it for just
1:42:07like committing or checking out branches
1:42:09or anything like that. I'll just use the
1:42:10CLI.
1:42:12I use it for like
1:42:14kind of reviewing history in the
1:42:16repository. So like I want to see a list
1:42:18of the previous commits and kind of what
1:42:19files they changed. I find it just like
1:42:21a nice UI for doing that or like if I
1:42:24want to quickly like look at the stage
1:42:26changes and then revert a few things.
1:42:28It's really nice just to be able to like
1:42:29right click a file, revert changes. Like
1:42:32I don't have to like remember kind of
1:42:33like how to do this on the command line.
1:42:36Um and then again just kind of for
1:42:38quickly reviewing things visually that
1:42:40the AI does. I kind of use it for that
1:42:42kind of thing. So it's more like
1:42:44I almost use it more of like a diff
1:42:46viewer like review tool than I don't
1:42:48really use it as like a
1:42:49you know kind of the branch switching or
1:42:51committing even or anything like that.
1:42:53Yeah. Um for me it's just a really quick
1:42:55kind of like review tool.
1:42:58Yeah.
1:42:59Uh and those of us just joining we're
1:43:02kind of using
1:43:04Claude code to help us generate a
1:43:07application using the AI SDK. So we'll
1:43:09jump into the code that it's generated
1:43:11to make sure one that's doing it the way
1:43:12that we would want it to be done and
1:43:15then two make sure that it works the way
1:43:16we want it to be done too. But the neat
1:43:18thing with the AI SDK is that since it
1:43:20is built you know by Taylor in an
1:43:22eloquent way it is designed in a way
1:43:25that's whether we're using it for our
1:43:27chat context like we're going to be
1:43:29using it with an open AI or the 11 Labs
1:43:32voice context. It's the same, you know,
1:43:34the same API, the same model kind of
1:43:37uh
1:43:38uh the same uh knowledge kind of model
1:43:41when we're when we're writing with it.
1:43:43Uh looks like it's almost done. Got his
1:43:45tests passed at least.
1:43:47I like how the AI I see this a lot when
1:43:49I use AI, too. It's like, "Let me run
1:43:50pint one more time to be safe." You
1:43:52know?
1:43:53>> [laughter]
1:43:53>> It's like
1:43:55Some of some of my uh some of my
1:43:56personal um looks like it just finished.
1:43:58Some of my personal uh projects that I
1:44:01use I I have it set up with Rector, uh
1:44:04PHP Stan, and pint. And so, it's it's
1:44:06it's fun to have it make sure that it
1:44:08does does all of that. Uh is there
1:44:11anything uh Taylor that you usually feel
1:44:12that like it still doesn't get quite,
1:44:15and so usually like you know every
1:44:16single thing that you build, you check
1:44:18this one or two things?
1:44:21Ooh, that's a good question. Uh I don't
1:44:22know.
1:44:23Um
1:44:26I didn't I hadn't really
1:44:27>> though, too. Yeah, it can change. I did
1:44:30nothing huge jumps out at me. Um
1:44:33I've tried to like kind of coach it
1:44:34along with my guidelines as much as I
1:44:36can.
1:44:37Yeah.
1:44:39So, let's go. We We have a knowledge
1:44:40assistant that created. Looks like it's
1:44:42using the
1:44:43>> I'd even use the attributes.
1:44:44Nice.
1:44:45>> Yeah, that's okay. I love that.
1:44:47Uh so, a friendly knowledge
1:44:48knowledgeable assistant. Um and uh the
1:44:53documents do not contain enough
1:44:54information to answer the question, say
1:44:56so honestly. Keep your answers concise
1:44:58to the conversation. So, just everyone
1:44:59in the chat that knows we're we're
1:45:00building um
1:45:01an app that if we go to resources and
1:45:03docs, we can create a
1:45:06a markdown file in here, and maybe even
1:45:08ask Claude code to generate some for us.
1:45:11Um I might I might even do that uh just
1:45:13for fun. I'll say,
1:45:15um open up a new Claude code, and I'll
1:45:17say we have this app that uses markdown
1:45:21files about the user in order to answer
1:45:25questions. Uh find some info on Taylor
1:45:31Otwell, and generate relevant
1:45:35markdown files in the resources
1:45:39docs
1:45:40directory.
1:45:43Kick start that off. But the the goal is
1:45:45to be able to then take all these
1:45:47markdown documents and with this
1:45:48knowledge assistant, and so we should go
1:45:51into I believe it's probably in a
1:45:52controller. Yeah, we have this ask
1:45:54controller here that's using looks like
1:45:56Laravel AI transcription files with
1:45:58documents audio.
1:46:01So I would assume hopefully it's using
1:46:03it in the proper sense where we're
1:46:05getting an upload of an audio. We're
1:46:07going to transcribe it.
1:46:09Um get all the
1:46:12docs in the
1:46:14markdown the resources docs directory,
1:46:17and then we're going to create a new
1:46:18knowledge assistant or new instance of
1:46:19the knowledge assistant, and with that
1:46:21prompt attach the attachments that we
1:46:23have in the markdown file, get it text.
1:46:25And then this is so clean. I love this
1:46:27Taylor like how easy it is to say, I
1:46:29just want an audio of this this answer.
1:46:32Um
1:46:34And so is is 11 correct? I don't
1:46:37Yeah, that's correct. Yeah. Okay. Yeah,
1:46:39that is correct.
1:46:41Sweet.
1:46:42Awesome. Uh the the one thing
1:46:45the one thing that I've noticed and I
1:46:46don't know if this is
1:46:48just
1:46:50me not prompting well, but the one thing
1:46:52I have noticed within um
1:46:54specifically within like the react
1:46:56context within Laravel, for some reason
1:46:58when it does stuff like
1:47:01within like a controller where it's
1:47:02hitting in hitting our API, it's not an
1:47:05inertia like use form request cuz you
1:47:07wouldn't use that for
1:47:09chat for example. For some reason it
1:47:11always likes to return it as an inertia
1:47:13page, and so you get like this JSON
1:47:16resource.
1:47:17So I always usually try to tell it to
1:47:19use
1:47:19Axios or something like that. Then it
1:47:21Yeah, yeah, yeah. Then it seems to do
1:47:23well. Are you still on Axios user?
1:47:25user
1:47:27I I do use Axios quite a bit, but this
1:47:30is actually something that's coming up
1:47:32in Inertia 3, which
1:47:35I think will be coming up, you know, in
1:47:36the next I I don't want to speak for
1:47:38what is coming out. I think it's coming
1:47:40out pretty soon, let's say, like in the
1:47:41next month or so.
1:47:43There's going to be a use HTTP thing
1:47:47that
1:47:49you won't have to really use Axios
1:47:52anymore with
1:47:54I don't even think we're including Axios
1:47:55by default anymore, which we do now in
1:47:58many different things.
1:47:59So, yeah, we're kind of giving people
1:48:02kind of dropping that and going a little
1:48:04bit lighter weight, but still giving
1:48:05people some of the like affordances of
1:48:07Axios.
1:48:09Yeah, I love that.
1:48:10>> I think Joe Tannenbaum's going to talk
1:48:11about that at his Laracon Europe talk,
1:48:13actually. He's going to kind of give
1:48:14like an Inertia 3 preview or kind of
1:48:17demo.
1:48:18Perfect.
1:48:20So, yeah, stay tuned.
1:48:22I I don't know off the top of my head if
1:48:23Laracon Europe talks are being streamed,
1:48:25but you they're probably going to be
1:48:27recorded at the very least, so be be
1:48:29sure to stay tuned for that. Question
1:48:32for Taylor, what's the recommended
1:48:34approach to implementing production
1:48:35grade guardrails? So, I guess within the
1:48:38AI SDK, let's say you're doing this and
1:48:40I I can maybe talk about even for my
1:48:41simple demo of what I made sure to do.
1:48:45What are some things that you can think
1:48:46of in terms of prompt injection misuse
1:48:49of elements, I mean API keys, that kind
1:48:50of thing. Yeah, this is something we
1:48:53haven't really built any opinions in
1:48:55thus far.
1:48:57I'm curious to see like what PRs come in
1:48:59or what sort of like community feedback
1:49:01is around this.
1:49:03Um
1:49:03you know, certain types of apps are sort
1:49:05of more vulnerable vulnerable to this
1:49:07than others depending on the sort of
1:49:08like what you're doing and what tools
1:49:10you're exposing and if you have tools
1:49:12that can like, for example, delete
1:49:14records out of your database, that's of
1:49:15course like much more potentially
1:49:18serious in terms of like prompt
1:49:19injection or things like that versus
1:49:20some other use cases. So, um yeah, this
1:49:23is something we haven't baked anything
1:49:24in on yet, but I'm curious to see sort
1:49:27of like what people want and what people
1:49:29have found, you know, most effective in
1:49:31other ecosystems that maybe we could
1:49:32learn from or borrow from.
1:49:35Yeah.
1:49:36Perfect. I love it. Um
1:49:38can Laravel AI SDK be extended or
1:49:40improved with plugins or add-ons? Um
1:49:44Yeah, so I think like one of the cool
1:49:46areas we'll see sort of like community
1:49:48extension and collaboration is um you
1:49:52could actually package up and distribute
1:49:54either tools or agents and tools as
1:49:57composer packages to give to other
1:49:59people. So, maybe you write a tool that
1:50:01is reusable and it does some some kind
1:50:04of functionality that maybe a lot of
1:50:05people would benefit from in different
1:50:06kinds of applications. You could
1:50:08actually package that up and distribute
1:50:10it, you know, on packages as a tool that
1:50:12other people can use.
1:50:14Um so, I think that's one of the primary
1:50:16ways you'll see people sort of like have
1:50:18plugins or so-called add-ons for the AI
1:50:20SDK is like distributing those types of
1:50:23things
1:50:24um
1:50:25throughout the community.
1:50:27That's awesome. I'm curious to know how
1:50:29much uh uh
1:50:31this got it right on you in terms of
1:50:33biography, philosophy, and projects is
1:50:36what
1:50:37what Cloud Code So, we have we have
1:50:38biography,
1:50:40um
1:50:42This all looks correct so far.
1:50:46And we just asked Cloud Code to generate
1:50:47all this for us. So, it's I'm assuming
1:50:49pulling
1:50:50uh just internet accessible things.
1:50:53Um
1:50:55Company grew from around eight employees
1:50:56to start 2024. Yeah. Yeah, this is all
1:50:59Yeah, that all looks correct.
1:51:02Simplicity and elegance.
1:51:06Warned against Cloud Code.
1:51:08Yeah, I think backwards compatibility
1:51:10has been a, you know, something that has
1:51:12been more important in more recent years
1:51:14of Laravel. I think people would
1:51:15question that in the early years of
1:51:16Laravel.
1:51:17>> [laughter]
1:51:18>> You know, probably as most projects go.
1:51:20In the early years of Laravel, you know,
1:51:21we were moving fast and breaking things.
1:51:23But lately I've tried I try to make it
1:51:25to where most people can upgrade, you
1:51:27know, without any changes. And it seems
1:51:29like the backwards compatibility uh is
1:51:31is even bigger now for AI because you
1:51:33almost have to have like uh oh, like
1:51:35things that happened a year ago, which
1:51:37models are getting smarter and more
1:51:38up-to-date, of course. But things happen
1:51:40a year ago, if it's using that, it still
1:51:42works.
1:51:43Correct, yeah. Yeah. Yeah, it is more
1:51:46important like not to break things in
1:51:48the AI world. I agree.
1:51:53Some some projects.
1:51:55Uh
1:51:58Yeah, cool. Okay, well, I'm curious to
1:52:01see if it's decent, yeah.
1:52:02>> Yeah, I'm curious to see how this works.
1:52:04Let's Let's go ahead and composer run
1:52:06dev. That is That is my preferred way of
1:52:09getting things up. Are you still the
1:52:10same way or PHP artisan server? I use uh
1:52:13either this or uh Herd. Okay.
1:52:17Open this up. Okay, we have Larvis. Uh I
1:52:20don't know
1:52:21>> UI.
1:52:22My sound Yeah, it's it's very minimal.
1:52:23Yeah, let's Let's go ahead. I don't know
1:52:25if my sound
1:52:26is going to work. Let's see.
1:52:29Can you tell me a little bit more about
1:52:32uh
1:52:32Taylor's way of thinking through things
1:52:35when he builds a project?
1:52:40Ooh, audio must be webm ogg web. I
1:52:43actually ran across this, I think, in my
1:52:44initial thing because I think it's it
1:52:47tries whatever AI did in the initial
1:52:49thing, I think it tried to save it as
1:52:51webm and
1:52:53by default, that's classified as video
1:52:55instead of audio.
1:52:56>> Oh, wow. Okay. Um
1:52:59Web assembly.
1:53:01Have this error. I think it's because
1:53:04when we initially try to uh
1:53:08save the vid
1:53:09audio file
1:53:11saves it a video format of it question
1:53:15mark
1:53:18>> [snorts]
1:53:18>> back to the chat if people were want to
1:53:22contribute to the level SDK on any
1:53:23specific task or areas you would like
1:53:25the community to help out with that is a
1:53:28good one I mean
1:53:30some of the recent PR so I've been
1:53:32getting around been around just like
1:53:33adding different providers so like
1:53:35someone contributed a deep seek someone
1:53:37contributed a mistral
1:53:40this is also like a tricky one because
1:53:41some of the more complicated features
1:53:43like um
1:53:44you know some of the conversation stuff
1:53:46I want to improve like
1:53:48I've got stuff in my head and sometimes
1:53:50it's hard and like if the PR is coming
1:53:52in and sort of like conflicting with
1:53:53that
1:53:55you know so
1:53:58I think I think that like agent chat
1:54:00artisan command thing is a is maybe a
1:54:01good one the community could riff on a
1:54:03little bit and then people have been
1:54:04adding providers
1:54:08human tool approval I don't I mean if
1:54:10someone wants to take a stab at that go
1:54:12for it I haven't started on it
1:54:14that might be a good one someone could
1:54:16work on but it's a little bit of a a
1:54:17meaty one you know it's not a not
1:54:19trivial probably
1:54:22awesome
1:54:23so yeah Taylor had mentioned a little
1:54:25bit earlier kind of like two things is
1:54:27chatting with agents that you
1:54:29automatically generate maybe like in a
1:54:30PHP artisan agent chat command
1:54:33and then
1:54:35a human tool approval correct me if I'm
1:54:37wrong Taylor but you mentioned it would
1:54:38be nice to be able to say okay there's
1:54:39specific tools that they happen but
1:54:42before the response is actually
1:54:44generated you have some form of human
1:54:46interaction to say hey this is good to
1:54:47go like let's run that kind of thing
1:54:49right yeah exactly
1:54:51those would those would be great
1:54:53um
1:54:54opportunities
1:54:57uh someone's mentioned token usage that
1:55:00would be interesting I don't know if
1:55:00there's anything currently built in,
1:55:02correct?
1:55:03You can get back to token uses from like
1:55:05a given prompt or whatever or given
1:55:07generation, but um we're going to be
1:55:09building a lot of token usage tracking
1:55:11stuff into Nightwatch. So Ooh,
1:55:13interesting.
1:55:13>> Yeah. People can kind of people can of
1:55:15course sort of roll their own cuz you do
1:55:16have access to that, but we'll be also
1:55:18providing that in Nightwatch.
1:55:20All right. Let's let's go ahead refresh
1:55:22see if it see if it does anything now.
1:55:24We'll say
1:55:25Uh can you tell me a little bit about
1:55:27how Taylor would think through creating
1:55:30a new package for Laravel?
1:55:37Uh
1:55:38I uploaded some markdown files. Please
1:55:40try again. PDF. Oh, you can't upload
1:55:42markdown files?
1:55:43>> Markdown files. Uh I I would assume in
1:55:45our our agents for the uh chat we're
1:55:48just uploading
1:55:51we're just uploading
1:55:54we're just collecting the documents as
1:55:55markdown.
1:55:56>> I was passing them. It should just like
1:55:57inject them into the uh Yeah. You know
1:56:00what I mean? Yeah. Instead of uploading
1:56:05cuz we get this error
1:56:08instead of uploading the markdown files
1:56:13let's inject them
1:56:15directly into the prompt.
1:56:18Yeah, like Should be like that. Yeah.
1:56:20See if that works.
1:56:21See if see if that works.
1:56:22>> I actually noticed that as well that
1:56:24OpenAI doesn't accept a whole lot of
1:56:27file formats on these attachments. Um I
1:56:29was surprised like you I don't think you
1:56:31can upload like a CSV. And it has to
1:56:34it's basically like PDF or image or
1:56:37I don't know why exactly or something
1:56:39before you do that.
1:56:39>> Yeah, yeah, yeah, yeah.
1:56:40>> I guess maybe are they trying to uh not
1:56:43to comment on OpenAI's business plan or
1:56:45anything like that, but are they trying
1:56:46to try to move everything into vectors?
1:56:48Like you do that first and then we'll do
1:56:50that.
1:56:51Uh
1:56:51that's very very interesting. Um
1:56:55It looks like it is making the correct
1:56:56change. So I saw where it was there.
1:56:58>> Yeah, I saw that in uh
1:57:01Yeah, so we're doing the
1:57:02docs with the question and prompt. So,
1:57:05bigger prompt, but we don't We probably
1:57:07be a little bit long longer of a wait
1:57:09before we get an audio response if we're
1:57:11generating the prompt, pushing it to 11
1:57:13labs, but still still should work. Uh
1:57:16uh I'm curious and that might be a good
1:57:18maybe if you're listening to this and
1:57:20you love 11 labs. I was actually going
1:57:21to take a stab at a PR. I don't know if
1:57:23you you even tried it, Taylor, like the
1:57:26ability to I think they have voice
1:57:28agents where you can like
1:57:30instead of having to transcribe and then
1:57:34get an answer and then put it back. I
1:57:36think they have the ability all wrapped
1:57:38up into one where you could upload a
1:57:40voice and it Yeah. you with the prompt
1:57:43or something like that? Or system
1:57:45prompt?
1:57:46Yeah.
1:57:47Which would make this a lot faster, at
1:57:48least for this this demo. One more time.
1:57:50Let's try try this out.
1:57:53Uh if I was Taylor, how would I think
1:57:56about building a new uh package for
1:58:00Laravel?
1:58:11That's great. When building a new
1:58:13package,
1:58:14>> [laughter]
1:58:14>> consider
1:58:15Taylor Otwell's development philosophy.
1:58:17Focus on simplicity and elegance.
1:58:20Ensuring that the package is intuitive
1:58:24Okay. It is working, but yeah, yeah.
1:58:28>> [laughter]
1:58:28>> Uh
1:58:29doesn't look like it was uh it was
1:58:31sharing my uh sharing my audio.
1:58:34Um but yeah, that worked. We have it. Uh
1:58:37I'm assuming we could uh speak. Uh looks
1:58:41like we have a little bug speaking
1:58:42thing. Yeah. Oh, there we go. Maybe I
1:58:45added Maybe clicking it a couple times
1:58:47added it. I assume I think because we're
1:58:48still saving it as
1:58:51uh memory in terms of like
1:58:53the
1:58:54um as controller do we have a
1:58:58Okay, it probably is just a one-off. It
1:58:59probably would reset then. I'm assuming.
1:59:02>> Yeah.
1:59:02Yeah, I think so. Sweet.
1:59:04Awesome. It's cool to be able to to get
1:59:06something like that up and running
1:59:08rather quickly. And honestly, there's
1:59:09not too much code involved with this.
1:59:12That's one of the things that I like
1:59:13about the AI SDK.
1:59:14Um is like we have an assistant being
1:59:18able to have and kind of as a good recap
1:59:21for those watching. You have assistants
1:59:23that you can create them PHP artisan
1:59:25make um agent. Sorry, not an assistant.
1:59:28Uh where we have a particular class that
1:59:30that all lives in. So, instructions
1:59:33um as well as tools. Is there anything
1:59:35else that can live in there, Taylor?
1:59:37Yeah, like the output schema.
1:59:39>> Output schema, okay. Things like that.
1:59:41Um and then with that agent, we can then
1:59:44uh create a new instance of it in things
1:59:46like uh controllers where we're creating
1:59:48a new instance of this knowledge
1:59:49assistant or passing it that prompts.
1:59:51So, we're using all the
1:59:53the configuration that we're doing in
1:59:55that assistant which also uses all the
1:59:57configuration of the AI config in terms
2:00:00of what's the default for audio,
2:00:01transcription,
2:00:03um images,
2:00:05uh and embeddings for example.
2:00:07And then it's as simple as using things
2:00:09like audio of the answer we're
2:00:11generating with and getting that
2:00:13particular response. And then it's
2:00:15converting that on the front end. But
2:00:16yeah, it's it's it's crazy how
2:00:19basically that whole demo was just
2:00:21really one controller
2:00:23if if in if in in the sense of that.
2:00:25Um and doing a couple things on the back
2:00:27end of how fast it can be. Uh and
2:00:29probably we could configure this quicker
2:00:31to use. I think in my
2:00:33uh my backup Larvis, I did a use
2:00:37cheapest model or use Is there a use
2:00:39fastest model? Am I correct? Uh it's use
2:00:42smartest and use cheapest.
2:00:43>> Use cheapest. I think I did a use
2:00:45cheapest for like the actual response.
2:00:47And then that way it just was a lot
2:00:49faster in a lot of ways.
2:00:50Um but yeah.
2:00:52Sweet. Let's go ahead and I'm going to
2:00:56Well, we're probably coming short up on
2:00:58time. So, is there any
2:01:00um
2:01:00I'll jump into maybe one more question
2:01:02with those of us who were those who were
2:01:05in the chat for this whole period of
2:01:07time, thank you so much for being here.
2:01:08Thank you so much for taking a journey
2:01:10along with us on how the AISDK was
2:01:12built, what it's good for, um and also
2:01:14getting a lot of questions for Taylor. I
2:01:16know uh Taylor, we're appreciative of
2:01:18you and your time of being here. Um For
2:01:21sure.
2:01:22Uh let's go ahead and see
2:01:26uh
2:01:28I don't know. There's not There's not
2:01:29too many more questions. Taylor, what
2:01:31what would be one thing that you would
2:01:33love to tell people about when it comes
2:01:35to like the AISDK, whether in terms of
2:01:37building with it or just
2:01:39you know, inspirational kind of kind of
2:01:41phrase to leave.
2:01:42Oh, well, it makes you know, integrating
2:01:44AI with your Laravel app super easy. So,
2:01:46I mean, it is out. Composer require
2:01:48Laravel/AI.
2:01:49There's a Boost skill available. So,
2:01:51install Boost and you will have the
2:01:53AISDK skill that kind of tells Boost how
2:01:55to use it and just start like, you know,
2:01:57kind of like we did here, just like
2:01:58building something fun, some cool ideas.
2:02:00You'll kind of learn how to use it,
2:02:01learn the ropes, and then uh hopefully
2:02:03can take it to um
2:02:05you know, whatever you're doing at work
2:02:06or whatever kind of like business you're
2:02:07building or whatever, and hopefully it
2:02:08makes uh your life a little bit easier.
2:02:11Awesome. I love it. And then for those
2:02:12who saw the uh the other kind of demo
2:02:15that I that I had kind of uh put
2:02:17together for
2:02:19um
2:02:19the kind of ragged instance and chatting
2:02:22with it. Um I'll put that link in the
2:02:24chat as well. Uh but I'll also be able
2:02:27to uh
2:02:28uh
2:02:29on that demo will also have like the
2:02:31GitHub link so that you can kind of take
2:02:32a look through what was what was built
2:02:35with that.
2:02:36Um uh I guess one last question uh
2:02:39to to kind of go through that as well as
2:02:42someone mentioned in terms of like prism
2:02:43PHP cuz I know you you mentioned a lot
2:02:45of some of the stuff in the background
2:02:48is using prism PHP. Someone mentioned um
2:02:50what does it look like to switch? What
2:02:52are some of the gaps at the moment in
2:02:53terms of switching?
2:02:55Um I think of yeah, we do use prism for
2:02:57some of the text stuff under the hood.
2:02:59Nice package uh built by TJ Miller.
2:03:02Um I think of prism and AI SDK a little
2:03:06bit like I think of query builder and
2:03:08eloquent. So like
2:03:10prism has kind of this fluid query
2:03:12builder interface for like querying and
2:03:14doing AI things. And I think the AI
2:03:16SDK's like
2:03:18maybe like a layer of abstraction on top
2:03:20of that you might think of where you
2:03:21have like agent classes and sort of like
2:03:24agent testing and things like that. So
2:03:26it's kind of like a little bit higher
2:03:27level
2:03:28uh of abstraction in terms of like code
2:03:30organization and like reuse.
2:03:33Uh which I think is kind of like the
2:03:34main difference between the two.
2:03:37Um and then it kind of like goes beyond
2:03:39in a little bit bit around like the
2:03:41similarity search, the file search, some
2:03:44of that stuff that is maybe not
2:03:45available in prism but is available in
2:03:47the um AI SDK as well as some extra
2:03:50providers like a 11 labs and things like
2:03:52that.
2:03:53Um so yeah, it's just a little bit
2:03:54higher level of abstraction um to get
2:03:56started and then of course um
2:03:58maintained here by us at Laravel but
2:04:00again using prism for a variety of
2:04:02things under the hood and uh
2:04:05yeah, great library.
2:04:07Awesome. Yeah, thank you so much
2:04:08everyone in the chat. Uh
2:04:11thank you for being here and for joining
2:04:13us on this journey. And for those who
2:04:15are watching kind of in on demand, uh
2:04:18thank you so much for watching this.
2:04:19Feel free to like and subscribe so that
2:04:21we can have additional content be pushed
2:04:23to you as quickly as possible. Uh but
2:04:25thank you so much Taylor for joining us
2:04:27and
2:04:27>> Yes, it's fun.
2:04:28for everyone getting started with
2:04:30building within Laravel and maybe AI,
2:04:32you can use composer require laravel/ai
2:04:37to get started um and use Laravel boost
2:04:40because it does have a scale to be able
2:04:41to generate some of those things even
2:04:42like that little demo that we just did.
2:04:45Simple, but it was easy to show how
2:04:48the LLM's can use things like Laravel
2:04:51boost and the AI SDK to create something
2:04:54pretty magical in such little code. But
2:04:57yeah, hopefully everyone has a fantastic
2:04:59day wherever you are or wherever you are
2:05:02joining and watching us from. Grab a
2:05:04Coke Zero for us and have some Taco Bell
2:05:08in favor of
2:05:09favor of Taylor and the Laravel team.
2:05:12But thank you all so much.
2:05:13Thanks. See you.

About this transcript

This page contains the full transcript of Building AI Applications with the Laravel AI SDK by Laravel, generated from the public captions YouTube serves with the video. The transcript has 24,247 words across 3,696 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to translate the transcript into 100+ languages.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, translation to 100+ languages, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.