YT YouTube2Text

YouTube2Text → Transcripts → Why Agentic Systems Need Ontologies — Frank Coyle, UC Berkeley

Why Agentic Systems Need Ontologies — Frank Coyle, UC Berkeley — Transcript

by AI Engineer · 3,099 words · 520 segments · language en · Watch on YouTube

Open in transcriber Summarize with AI Download SRT/VTT

Full transcript

0:01[music]
0:12>> Um, okay, we're going to launch here.
0:15So, my name is Frank Coyle.
0:17Um, I'm I'm an educator and teaching at
0:20Berkeley now. I've been doing this
0:22computer science stuff for
0:24oh, 30, 35 years.
0:26And um
0:28>> [snorts]
0:28>> I'm intro and right now it's kind of a
0:30critical time for uh
0:32poor computer science students. Used to
0:34be the used to be the only game in town.
0:36Degree was a guaranteed job, and now
0:39thanks to AI, it's not. But then again,
0:415,000 people are here. So, AI and and
0:45agents are um seem to be the way to go.
0:48So, the question is how do we leverage
0:50this new universe that we are moving
0:53quickly into. And so, I want to talk
0:55about how agents and ontologies will big
0:58word fit together. But before you do
1:01before I do that, I wanted to um
1:05wanted to give you my uh my educational
1:07philosophy.
1:09And this comes [clears throat] from uh
1:10someone called Sister Corita Kent.
1:13And it was made popular by John Cage,
1:16who is a uh an avant-garde musician.
1:19And
1:21you got to think about this little bit.
1:23Nothing is a mistake.
1:25There is no win.
1:26There's no fail. There's only make.
1:30And more and more today, that's what's
1:32important. Get down and make stuff, and
1:35that's how you're going to learn, not by
1:36necessarily reading.
1:38I'm also a big fan of writing.
1:41My early career was in neuroscience. I'm
1:43kind of coming back into it now that
1:45Agent AI is bringing uh
1:48kind of cognitive science back. But
1:51engage your senses. Get a notebook. Get
1:54a
1:55pen, a pencil. Draw pictures, write
1:58stuff down.
2:00Just don't type because when you type
2:02you when you're typing your brain is
2:04thinking about the letters on the
2:05keyboard. When you're writing in a book,
2:08your whole brain, your your whole all
2:10your all your sensory systems are
2:12engaged and you're going to learn
2:14faster that way.
2:16Okay.
2:18On to our talk.
2:21Agents and ontology. So, there are two
2:23lineages here and I want to talk about
2:25both, give you a little philosophical
2:26background. Um
2:29agents, when did we start talking about
2:31agents? Well, goes goes back to the
2:33early initial days of AI. People like
2:36John McCarthy,
2:38uh
2:39uh uh
2:40uh uh Selfridge, Marvin Minsky, Society
2:43of Mind. People started thinking about
2:45the fact that this new computing
2:47technology was going to lead us into
2:50some kind of artificial intelligence,
2:52which is a term that came
2:54in 1956 when all these characters got
2:56together and tried to figure out where
2:59the future was going. Okay? And the
3:01concept of an agent finally evolved,
3:04things that
3:05perceive and decide and then act and
3:07that's what we're seeing now.
3:09Now, what about ontologies? Well, it
3:10turns out ontologies are not that new.
3:13Okay? It was actually Aristotle who
3:16first came up with the concept of we
3:18need a philosophy of of being. Like,
3:22whoa, kind of heavy. Um but came up with
3:25categories of being and this kind of
3:27relates to what people are doing now
3:29with graph databases and knowledge
3:32representation. And there are a couple
3:34of other people who kind of formalized
3:35it.
3:36Uh
3:37Von Quine was a philosopher and then
3:39this guy Gruber, 1993. And I think this
3:42captures what
3:45knowledge and uh
3:48graph technology really represents. It
3:51is a a formal specification
3:54of a shared conceptualization. And
3:57that's what we want to give to our
3:58agents. We want to give them our concept
4:01our conceptualization of the universe,
4:04our universe, our domains. Okay? And
4:06now, what's happening
4:08is you're getting the convergence of
4:11something that is probabilistic,
4:14the agents, the LLMs, with the
4:18the more formal representations that you
4:20have with ontologies. And so, this term
4:24is now being used you
4:26hearing this a lot, neuro-symbolic
4:28AI. Sounds pretty fancy, but it's really
4:31neural networks tied into
4:35symbolic AI, which rule-based systems
4:39come under that category,
4:41um as do the knowledge graphs that we're
4:44that we're
4:45assembling. And so,
4:48what I'd like to argue is that
4:49neuro-symbolic AI
4:52sort of represents a way to keep the LLM
4:56on its guardrails, because LLMs are by
5:00nature probabilistic.
5:02People worry about hallucinations, but
5:05that's the feature. That's actually a
5:07feature of large language models. It's
5:09who we are. We hallucinate in a way. We
5:13imagine things that may not exist, and
5:15then we turn them into reality.
5:17And that's what large language models do
5:19in
5:19in a way.
5:20Okay? So,
5:23let's just quickly overview what
5:26ontologies are. It's not They're not
5:28complicated. They're basically a
5:29representation of entities and their
5:33relationships to other entities. And
5:35these entities have properties. And this
5:38whole concept of graph databases
5:42arose when
5:45people began to realize that relational
5:48databases
5:49sticking data into tables was too
5:52restrictive. You wanted to add something
5:55new to
5:56a relational database, so you have to
5:58add a new column. Man, I had then then
6:00you have to redo the whole structure.
6:02With a with a graph database, you can
6:04just attach another item. You can just
6:06attach a property. You can attach a
6:07relationship. Okay? So, the question
6:10often arises, okay, I I get it. I need
6:13to have an ontology
6:15to represent in a formal way what my
6:18organization is doing. How do I do it?
6:21Okay? There are a couple of ways you can
6:22approach it. You can have a top-down
6:24approach or a bottom-up approach.
6:26Top-down approach is
6:29you get the experts together and they
6:31sit down and analyze the domain, come up
6:34with the entities. What do we have? We
6:35have purchase orders, we have customers,
6:38we have customer representatives, and
6:41we're going to structure them. They have
6:42properties. These are the relationships.
6:44Okay, that's one way. And this models
6:47what we were doing back in the '80s when
6:49I was involved in expert systems.
6:51Everybody thought expert systems was the
6:53way to do AI. Symbolic AI was the way to
6:56go. Companies rose, millions of dollars
7:00were spent.
7:01Uh the
7:02the Japanese created this uh
7:06future world project in the late '80s.
7:09People in America were my my son was
7:12taking Japanese in school because of
7:14these expert systems. And
7:17but they couldn't scale. They couldn't
7:19scale, and then we went into a kind of
7:21AI winter.
7:23Where did neural networks came come
7:25from? Neural networks were put out there
7:27in the '60s, but they couldn't scale
7:30because we didn't happen to have Nvidia
7:34who was off making GPUs to make make
7:38reality of the of the video games
7:40fantastic, and then someone said, let's
7:43turn these things over to the neural
7:44networks, and of course, that's kind of
7:46why we're here now. So,
7:49the other way you can that that people
7:52are
7:53adding to or creating ontologies is is
7:55from the bottom up. For example,
7:58customer
7:59reactions.
8:01What are the things the customers are
8:02involved in? Wait.
8:04Do you these entities, these
8:05relationships, let's add this to our
8:07ontology. Let's Let's add this
8:09information to the graph. Now, as as a
8:12help,
8:14it's helpful to be aware that there are
8:16existing
8:18taxonomies that people have been working
8:20on for the last 15 to 20 years. Things
8:23like schema.org, which has a whole set
8:25of terms and relationships, so you don't
8:27have to reinvent the wheel. In fact, you
8:30it's to your advantage to use some of
8:33these ontologies. FOAF, Friend of a
8:36Friend, for modeling social networks.
8:40The Dublin Core, which was
8:42an early an early attempt to come up
8:45with terms for describing
8:47uh research papers and books and so
8:49forth. So, there's a whole series of
8:52things. In fact, Wikipedia is based on
8:54an ontology called DBpedia. So, when you
8:57do a search on Wikipedia, it's looking
8:59things up in its giant graph database.
9:02So, this stuff has been out there
9:04underlying a lot of what we already do.
9:07So, take advantage of these things that
9:09already exist.
9:12Okay. Now, what do you do when you build
9:15your ontology? Okay, so what? I know
9:17what these entities are, I know what
9:19their relationships are, they have
9:21properties. How can I do anything with
9:24them? Well, there are other augmenting
9:26technologies, auxiliary technologies.
9:29Things that we call the things like
9:31RDFS, which is a technology, and OWL,
9:33which I'll talk more about. So, these
9:35have
9:36these kind of sit over to the side of
9:39your graph. So, I'm not going to talk
9:41about on I mean ontology is a big word
9:44and it's often confusing and used in
9:45many ways, but think of it as a graph
9:47data structure. Okay?
9:50And you have the entities and
9:51relationships, but you want to apply
9:53some control over them. Or you want to
9:56be able to make inference over them. So,
9:59for example, there is uh
10:02some terms in this technology called
10:04RDFS.
10:05Domain and range. So, if I say
10:09teaches
10:11has a domain of teacher. That means if I
10:14say Bob teaches Scooter in my text, I
10:17can infer that Bob is a teacher.
10:21And if I say all teachers are persons,
10:23then this statement lets me know if I
10:26say Bob teaches Scooter, now I know Bob
10:28is a person, Bob is a teacher. What
10:29about Scooter? If I say teaches has a
10:32range of student, that means the the
10:35right side of the verb, then Scooter is
10:37a student. And now I have this extra
10:39information into my system.
10:43OWL also has a series of
10:47of of properties that allow you to make
10:50some inferences. So, a transitive
10:52property transitive property says, if
10:55Sue is an ancestor like ancestor is a
10:58transitive property. If Sue is an
11:00ancestor of Mary and Mary is an ancestor
11:02of Ann, then Sue is an ancestor of Ann.
11:07Okay? This was not initially into my
11:10graph system, but with applying these
11:13functional properties, I can then
11:16add and augment the system with this
11:18extra data. So, that's very useful.
11:21Then there's some
11:23properties called functional properties,
11:26which means only one.
11:28So, has father
11:31is a functional property. You can only
11:33have one father. You can only have one
11:35mother. That is a functional property.
11:38Okay? So, that's that can serve as a
11:41constraint. So, when
11:44if you say Bob is my Bob is Jim's
11:48father, BB is Jim's father,
11:51well,
11:53the inference here is that Bob and BB
11:55are two ways of representing the same
11:57individual because
11:59that is a functional property. Can only
12:01have one. So, these derivations and
12:04constraints that don't sit in the graph,
12:06they sit sort of on the side
12:09and they can help
12:11as we're going to see, I'm going to
12:12propose,
12:14when we deal with agents, how they can
12:16they can help us out. So,
12:18what about agents?
12:21Everybody's talking about agents now and
12:23everybody's talking about loops. Loops,
12:26loops, loops everywhere.
12:28Loops have been around for a long time.
12:31Back in the '60s, people were debating
12:34who has the best programming language?
12:35Fortran or COBOL? No, mine is better.
12:37No, mine is better. Oh, you don't know
12:39anything. You don't know what you're
12:40talking about.
12:42Bohm and Jacopini in 1966 came out and
12:45said, "Okay, there is no real difference
12:47in programming languages if they have
12:49three aspects.
12:51Sequence. I can put statement A,
12:53statement B, statement C. Fine. I have
12:56conditionals. I can have if then.
12:58And the last piece, I have a loop.
13:01If I have a loop, if I have iteration,
13:03if I take these three things,
13:05the the language is what's called Turing
13:08complete. Can do any can compute
13:10anything that a
13:13can be computed by computational devices
13:17from the work of Alan Turing.
13:19Okay?
13:20And now we're seeing this in agentic AI.
13:24Agents are now have loops. Loops give us
13:26the last piece in the equation of giving
13:29us
13:31a technology that is capable of
13:35doing anything that computational
13:37devices can do.
13:40The danger though of loops is that they
13:43can break.
13:45If you're If you're a programmer, you
13:46know, you've all go into infinite loop.
13:49Not good.
13:51Loops can drift as agents start talking
13:55to each other, things get all go off off
13:58the rails. And
14:00loops can cost you money.
14:02Token counts crank up as the loops
14:05continue. So, you don't you need to be
14:08careful, okay? But in a way we are
14:11revisiting some of the early stuff with
14:13symbolic AI. I would argue we're going
14:16back to the world of expert systems.
14:20Which is the symbolic part of the whole
14:21thing. So, I want to show you a little
14:23example using Claude
14:26agent. So, little code here. Don't get
14:29scared, but
14:31I know nobody does Python anymore, but
14:33you got to look at what the agent's
14:35giving you and you got to you got to
14:37move in and and manipulate it. So,
14:39here's a here's a loop while true,
14:41classic Python loop. Okay? And so, we
14:44have a client. So, we're actually So,
14:46the first little chunk here that you
14:48see, r e s p, the response, this is just
14:51some code where we have a model and we
14:55have uh
14:57we have a prompt, that's part of part of
14:59the messages,
15:00and we have a tool,
15:03and we're we're asking the LLM to
15:08solve this problem using a tool. Now,
15:11here's the here's the catch.
15:14LLMs can't do anything. All they can do
15:17is give us the next word with a high
15:19probability. Amazingly, we can now have
15:22these conversations it, but they can't
15:23do anything. But, we can give it a tool,
15:27and we can give it
15:29what we want, and say,
15:32"How do you think this tool can help us
15:34get what we want?" And then the LLM will
15:37set up the parameters,
15:41and come back to us, and say, "Okay,
15:44here's my response.
15:46I can't execute this tool, but I know
15:48what the input parameters are. I know
15:50what your context is. I know what your
15:53prompt is. So, here is the call that you
15:56need to make
15:58of the tool, because I can't do it. I'm
16:00the LLM. I'm just locked in this box.
16:03Okay? So, the second box the second
16:05chunk is
16:08stop reason. So, stop reason means
16:12the LLM has stopped for some reason.
16:15The The reason here is that it can't do
16:17anything, and if the reason is tool use,
16:20ah, now it's time. Let's go execute that
16:22tool. So, that second line, get tool. It
16:25takes the response, which is
16:28formulating the the parameters, and
16:30triggering the action.
16:32Okay. Now, I have this stuff in red
16:34here. This is where I think
16:36the LLMs and other uh
16:40I'm sorry, not LLMs.
16:41The ontologies and stuff can come in.
16:43So,
16:45if you look down there,
16:48after the the tool is called, it said
16:50tool runs. This is where
16:54ontologies could come in.
16:56The tool's going to give us information.
16:59We put the information in a form
17:02that our
17:04our our our validator can use, and think
17:07about the validator as operating with
17:10this these ontologies about our domain,
17:13then we can
17:16make some sense
17:17of whether the response of the LLM is
17:22reasonable. So, this is the loop. Call a
17:25tool, check the stop reason.
17:28If it's a reasonable result, then let's
17:31go with it. If it's not reasonable, go
17:33back to the LLM. Say, "Oh, this is this
17:35is not working." Or get a human in the
17:38loop. But the idea is
17:42to surround the input with checks. Now,
17:44I've got this
17:45something that you that you should be at
17:48least taking a look at if you're doing
17:49some of this coding is something called
17:51Pydantic. Pydantic is a way to specify
17:56the types of what you want the types of
17:58the parameters to be. Those of you who
18:01who do know Python, know Python is a
18:04unstructured type language. So, you can
18:06have a variable x = 20, x = hello, no
18:10problem. There's no typing.
18:12Pydantic adds typing to that. So, you
18:14want to
18:15check your types with Pydantic and then
18:18check your results with the ontology.
18:23So, Pydantic at the door, ontology at
18:25the ledger, and pure agents and by the
18:28way, your agents should try to have no
18:30side effects.
18:32That helps the whole logic. Meaning,
18:34they're not running off doing something
18:36that they're they're changing they're
18:38changing things in the database not yet.
18:40You want to run them through the
18:41ontology first and make sure that works.
18:44Okay.
18:45I only got an I've got I've got another
18:46I've just a short time. I'm going to try
18:48to show you some of the things that um
18:51that you can
18:53some logical constructs from from
18:55something called OWL, the uh
18:58the web object language for for objects.
19:01So, you have these functional
19:03properties, disjoint properties. I'll
19:05just put these you can look at the
19:06slides, but essentially the errors it
19:09can catch. Look over in the the
19:11right-hand column. A second refund on
19:14the same order
19:15is a is
19:17is a problem. But ontologies could catch
19:19it, whereas it's it's very tricky to do
19:21that in in English. A payout sent to the
19:24support desk instead of the buyer. Okay?
19:27You can catch that with an owl disjoint
19:29property where customer and support rep
19:31are two separate entities. Okay?
19:34Uh one of may a made-up value like
19:38probably shipped. You can specify
19:42you must have certain kinds of value.
19:44So, uh the status
19:47paid, shipped, or refunded, nothing
19:49else. And when you're in the pure text
19:51world, this can get this can get funky
19:54because the the LLMs are again
19:57probabilistic and
19:58um
20:00return some crazy stuff. Okay. Uh so,
20:03really what the point I want to make
20:05here is use these re- you can have a
20:08reasoner built on ontology to check keep
20:11the LLM on track, have guardrails to
20:16keep it honest. Okay?
20:18And for the guardrails, I'm referring to
20:21these concepts re- these support
20:24technologies with RDFS and owl.
20:28And
20:30my my bottom line is and nothing is a
20:32mistake, there's no win, no fail, only a
20:35make.
20:37Okay. Feel free to reach out to me
20:39coil@burkly.
20:40I've got a I've got a
20:41I've got a little website
20:42codesupreme.ai. I'm a big fan of if
20:45you're John Coltrane has a has a some
20:48jazz called uh
20:49called Love Supreme. So, I've named my
20:52site
20:53Code Supreme. And if you go there, I've
20:54got some music and it's all good. Okay.
20:57Thanks very much. 20 minutes.
20:59>> [applause]
21:15[music]
21:17>> Woo!

About this transcript

This page contains the full transcript of Why Agentic Systems Need Ontologies — Frank Coyle, UC Berkeley by AI Engineer, generated from the public captions YouTube serves with the video. The transcript has 3,099 words across 520 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.