AI Agent Tools

00:00 In this chapter, we're going to continue working on our conversation system by enhancing it with AI agent and tool execution capabilities. Before we dive into that, we're going to go back and implement what we left over from the previous chapter. We still have to implement message cancellation flow and we have to build the history dialog. After we do those two, we can go ahead and configure the AI agent with system prompts and create a complete tool execution system. So let's go ahead and make sure our app is running.

00:33 So make sure you have npm run dev. Make sure you have npx convex dev and make sure you also have npx ingest dash CLI dev. So you should have three running npm run dev, npx convex dev, and npx ingest CLI at latest dev. Great. Now let's go ahead and just test out test out our app a bit just to see what we can do and what we can't.

01:08 So if I go ahead and open my app right now I can definitely send a message hello world. But one thing I immediately notice is I cannot cancel it, right? So even though I have an indicator to cancel it, there's nothing really happening here. Also, I don't think there is a possibility to revisit older conversations. So I can definitely start a new one, new conversation.

01:35 That works too. But I cannot revisit the old one. We've just tested out. And besides that, obviously there is no AI processing. This is just mock.

01:48 So let's start by giving the ability to cancel a message. So this will be very useful to save some tokens, right? If you want to quickly cancel, you should be able to do that. Same goes if there is a very long running message here and the user in the meantime opens a new conversation and they send a new message we should also make sure to cancel the previous one because obviously they meant to discard that so let's go ahead and see how we're going to do that and I just want to give you one quick tip If you go inside of your graph here, source control, you probably see that I have these two commits that you never saw me do. I call them override because that's what they are.

02:29 I Just use them to update my README file. That's it. It's because I finished part 1 of this tutorial, so I had to update the README so people can actually see what the project is about. That's it. Your last commit should be this.

02:45 12 conversation system instead of a merged pull request. So all good. Don't worry about the fact that I have additional commits. It's just a read me change. So the first thing I want to do is I want to implement the route to cancel a message.

03:03 So I'm going to go inside of source app folder API. And inside of here, we already have messages folder. Perfect. So let's go ahead and simply create a new folder called cancel and inside a route dot DS. Now in here, let's go ahead and import the following.

03:22 We're going to import z from Zod, next response from next server and auth from clerk nextjs server. Let's import ingest client as well as the convex client. And then finally, let's import API and ID type from convex. Let's quickly define the request schema. So, what will this request accept?

03:46 Very simply, a project ID where the message is currently processing. So we're gonna make sure that at any point only one message can be processing for this project. Now the reason I'm doing this is simply so in this state of the application, we don't have too many ways to overspend our tokens. Obviously, this can very easily become a limitation if you ever wish to implement something like multiple agents, right? But it's a very easy change.

04:18 We're not really doing any architectural preventions of multiple running or processing messages. I'm simply implementing this so you don't have accidental running and spending requests. So this will be a post request so let's go ahead and simply export that and let's extract the user ID. Then we can immediately do an ALF check so if the user ID is missing, let's simply throw a 401. After that, we're going to go ahead and extract the body using await request.json.

04:57 And let's immediately parse that using our request schema. This will allow us to have a type safe param and also throw an error in case it's invalid. Now let's go ahead and let's get our internal key. So const internal key process.environment. And let me go ahead and remind myself of how we actually call this do we mention that here we do here it is Polaris convex internal key So I'm just going to use that here again.

05:33 The exclamation point at the end basically makes sure that this right now this is a string or undefined. If I had an exclamation point, it's just a string. So it's just for type safety. So I have a habit of putting an exclamation point at the end in case you were wondering. But we don't have to do that because we're going to do a check here.

05:53 If the internal key is missing, we're again going to throw a NextResponse 500 with an error InternalKeyNotConfigured. To give you a quick reminder, we need the internal key, because that's the only way we can query a convex client from Next.js API routes. So what we have to do now is we have to find all processing messages in this project. So yes, every time the user hits this cancel endpoint, we're gonna make sure that we cancel every single message that's currently processing. So we have at least one button, which says, okay, stop all token spend, right?

06:34 Just stop, abort everything. So we can very easily abort every single running background job which is very useful. So how do we get processing messages Well by awaiting a convex query. The problem is we haven't developed this query so let's go ahead and do that. So save this file and then we're going to revisit our convex system.ts.

07:02 I'm quickly going to check, do we have anything called processing messages? We do not. Great. It means we have to develop it. So I'm going to go all the way at the bottom here and I will do export const getProcessingMessages.

07:20 And this will be a query which will accept arguments of internal key and project ID and then as usual we're gonna have a handler which is an asynchronous function which allows us to access the context and the arguments. First things first, we're going to validate the internal key using a helper we have developed. I think this type error here is just something I have to restart. Yeah, that's it. Let me go down to the project, to the function I was developing.

07:55 So validateInternalKey is just a little helper so we don't have to write this whole thing every single time. It basically checks that the proper and matching internal key was passed. And then from here all we have to do is we have to query the database for messages. So let's go ahead and do that. I'm going to query for messages and then because if you take a look inside of our schema.ts in the messages, we have an index called by project status.

08:30 So basically this means not by project's status, it means both by a project ID and by messages status. So we can very easily and in an optimized way using an index find a message from a specific project which is currently processing, which is exactly what we need right now. So we're gonna use the withIndex helper like this, calling the byProjectStatus, which we defined right here. And then we're simply going to query, if, let me go ahead and display this like so if query project ID equals arguments project ID and if status is currently processing and make sure to execute the collect method at the end that's it This will be a system helper which will allow us to at any point in time show us all messages which are currently processing. This will almost exactly, well should always correlate to us having a running background job.

09:34 So now that we have that function we can go ahead and go back inside of our new cancel route here and we can call that specific query api.system getProcessingMessages. The arguments it accepts is our internal key and project ID, project ID which I'm going to typecast as ID project. There we go. Now that we have the processing messages, let's first check if we can easily return already if there are none. So, if there are no processing messages, Well, let's just break this API method.

10:17 No need to go any further because we can end here. But if that's not the case, we now have to cancel all the background jobs for each of these messages. How do we cancel a background job? Well, programmatically, we do that the following way. Let me just remind myself a little bit.

10:38 We have this functions I think it is inside of features, conversations ingest process message. Here it is. Instead of our process message ingest function, we have developed a cancel on property here. And the way we can programmatically cancel a background job is if we send an event message forward slash cancel and if we pass along a message ID property. And then this syntax right here is going to check if this events message ID matches process message event data ID.

11:20 So we have to explicitly know what message ID we want to cancel for. Luckily for us, we have all of that information. So, since this can be a lot of processing messages, even though it shouldn't, it should only be one, still, I want to develop it in a way that many of them can be cancelled at the same time, just in case, so you don't have to think of that. We're gonna do const cancelIds awaitPromise.all and let's do processingMessages.map let's run asynchronous method and very simply let's call await ingest.send message forward slash cancel and we also have to pass the necessary arguments. So data is going to be message ID, message, underscore ID, like that.

12:24 So now for each of these processing messages which we have successfully queried that they have a processing status in the convex database, we are going to attempt to cancel their respective background job because we have to assume that if a message has a processing status it must have a running background job. Because the only way a message can stop having a processing status is after process message function actually finishes because if you go at the end here, here it is, update assistant message and this system function calls update message content and in here we set the status to completed. So if there actually is something in the database which is still in processing status it must mean, most likely, that there is an equivalent background job running. And this way we cancel it. Great.

13:21 Now that we have that, we should also tell the user that a message has been cancelled. So for that, I'm going to call convex.mutation API.system. And similarly to update message content, we're actually going to implement a new method called update message status so we're not really interested in anything other than the status So let's go ahead and quickly implement update message status. I'm going to go back inside of my system.ts and let me see can I maybe copy yeah we can copy this entire thing? This will be called update message status.

14:06 It's going to be a mutation. It will accept an internal key. It will accept a message ID. But for the state for the next argument, it's not going to be content, it's going to be status, which is a union of processing, completed and canceled. Always double check that that matches your schema so you should be able to copy it from here and add it here and it should be exactly the same.

14:36 I don't think the order matters but there should be no typos, right? As always, we have to validate the internal key and after that we do the patch method but only on status and specifically we do it on arguments.status. There we go. That's our update message status. We can now go back inside of the route where we now have the equivalent update message status.

15:05 We can pass in the internal key, message ID to be message underscore ID, and the status will be canceled. And Finally, return message ID. Like that. And once we've done all of that, let's go ahead and do return. Next response.

15:29 Like so. Dot JSON success. True. Cancelled. True and message IDs canceled IDs.

15:42 What we return from this API endpoint is completely irrelevant. I just want to make it useful. So we successfully did it and we successfully cancelled. Do we really need both? I don't know.

15:55 I think we can make do with just success true and message IDs is useful just so you can at least see in your network history, all right, how many of these messages were actually canceled by calling this endpoint, right? Great, so that's the backend part finished. What we have to do now is we have to revisit the conversation sidebar. So the conversation sidebar is currently missing a proper handle cancel method. So conversation sidebar located in features, conversations components right here.

16:30 Let's go ahead and let me just find... Yeah, so nowhere in this code do we have a proper developed handle cancel method. So let's go ahead and do it. Const handleCancel() is going to be an asynchronous method. So let me fix the typo async.

16:49 And in here we're going to open a try and a catch method in the catch method. We can already throw an error unable to cancel request. And in the try, we're going to await ky.post api messages cancel So just make sure you don't misspell this part because it's not type safe What I mean by that is that there's nothing stopping you from writing this accidentally So always double check you know, is your API actually called that messages cancel like this. And in the post, we have to pass in the JSON. The only thing we actually expect is the project ID, which we can use a shorthand alias like this.

17:37 Perfect. Now that we have the handle cancel, let's first learn how to call it explicitly. And I think the simplest way is to add it inside of the handle submit and it's this case. So if we are currently processing and no new message has been submitted, this means that this is just a stop function. So now we can actually await handle cancel in here.

18:05 If is processing and if there is no message dot text. So let's see that in action. In fact, I want to make sure that I have my ingest running somewhere. Here it is, alright. And I have this and I also just for fun I'm gonna make sure to increase the sleep time of my function so we can actually see this effect.

18:30 So go inside of features, conversations, ingest, process message, and change this from 5 seconds to 50 seconds. Simply so we can actually see the cancel effect happening. So now I'm going to go ahead and do testing cancellation and I'm going to send it. And we should have a new background job running here and this should wait for 50 seconds. But if I click this, hopefully and looks like successfully we have cancelled it.

19:06 Here it is. You can see it says canceled. And I think there is somehow a way to observe this cancel event because technically we did just send a whole new event. So it should be documented somewhere. I'm just not sure where, right?

19:27 But what we just did is we used convex database to find, let me actually open convex, perhaps that's going to help us to. So if you go inside of your project here in the convex dashboard instead of data specifically inside of messages here you should find status column here and you can see that my last one has cancelled status. So just to make things easier I'm going to delete all of my messages and I'm going to also delete all of the conversations. Then I'm just going to go ahead and create a new brand new project here. So it's easier for me to understand what's going on.

20:10 And I'm going to send again testing cancellation. So right now what's happening here is that we have a new message. So go inside of your messages table and you can see that right now there is a response from the assistant which is currently in its processing status. So this is now spending tokens. Well, not right now, but it will in the future.

20:35 This means AI is thinking of something. So if user changes their mind, and if they hit this, what we do is we query all of those messages which are currently processing, and we fire a cancel event on their ingest background job. And then we also change their status to cancelled. Right? So one thing I wish we could do as well instead of the ingest, perhaps we can somehow, I wish I could change the actual cancel status in here, in the cancel on.

21:14 Maybe There kind of is a way to do that. I'm not sure. I have to check documentation. But right now what we're doing is we're doing it directly in the API route of the cancel right here. So we simply fetch all the processing messages from the database, which basically means all the assistant messages that are currently processing.

21:34 We early return if there are none. But if there are, we send ingest message forward slash cancel, because that's exactly what we defined here. And we send message ID. So what's important is that you didn't misspell message ID anywhere in here, right? And also when you invoke process message, let me see process message, where do I define the event?

22:03 Here it is, message forward slash sent. So when you invoke this in source app API messages, it's also important that you didn't misspell the variable here. All of those variables are important for this clause right here which will allow us to successfully cancel a message. Perfect. So explicitly cancelling a message seems to work just fine.

22:28 The problem is User has no idea what just happened. Well, they technically do, right? But it would be nice if there was an explicit way to show to the user, hey, this message has no content, and that's because it was canceled. So for that, we should go ahead and find, I think it's also inside of the conversation sidebar. Let me find it right here, where we iterate over our conversation messages.

22:58 We should check if a message has been canceled. Right now, instead of the message content here, we have if message status is processing show thinking, otherwise show the content. So we should do kind of a double otherwise. I'm sure there was better ways to say that but yeah I think you know what I mean and in here let's just go ahead and check otherwise if message dot status is equal to cancel then go ahead and render a span request cancelled and let's simply give it a class name text muted foreground and italic and then in here let's go ahead and make sure we return a message response. I hope this is proper syntax.

24:01 It is. There we go. You can see it says request canceled, right? So if I go ahead and send another message here, you can see that right now it's thinking and when I hit cancel, it will change to request canceled and allow us to send a new one. So we don't allow the user to spam and have a bunch of background jobs running.

24:23 Great. So now that we have that finished, there is still another way a user can have both. So if I do this, for example, if I go in this convo I am processing so I will send this message. This is now processing right? You can see that a message is processing, a background job is running.

24:49 And if I attempt to send anything else here, I'm going to cancel it. But if I open a new one and do hello, now we don't actually cancel anything. We just have two processing messages. And two running background jobs. Chances are, user probably doesn't care about what is in the other one.

25:10 Maybe in the future, your users will and you will change this behavior, but I feel like most of the time, this is just user deciding to start a new conversation. So it would be a good idea that every single time we send a new message, we cancel all the currently running requests for this project. So let's go ahead and do that. We're going to revisit our source app API messages route.ts. And basically what we have to do is, We have this to do called invoke ingest the process the message, but I think we are already doing that.

25:52 So I think we can remove this to do. We're just missing some data, which we're going to have to update. But this is the to do I care about. Check for processing messages. So inside of this async post function here let's go ahead and find the place where we define the project ID before we call convex mutation create message.

26:18 Before we do any of that, we're gonna go ahead and check for processing messages. Lucky for us, we can now do this quite easily. So, in fact, you can open your cancel right here. And you can go ahead and do this right so instead of this to do copy this paste it find all processing messages in this project so we are calling the same function get processing messages and I think we have to cast it this time. Yes, we don't have to, we can just use this.

26:53 What's important is that you pass the internal key along and the project ID. And now we're going to check if processing messages.length is larger than zero, meaning we have some processing messages in this project while the user is trying to send a new one, let's cancel all of those processing messages. And we can actually copy this entire thing. So this await promise all can be copied entirely. Like this and just paste it here.

27:30 I'm just going to fix the indentation. There we go. So await promise all, processing messages.map, we get the individual message. We first trigger the cancel event from ingest, so we shut down the background job using the equivalent message ID. And after that, we update the message status to cancel.

27:52 Why? Well because remember we can't do that inside of the cancel event so we have to do it here. And we actually don't have to return the message ID. Previously we do that because we want to return the network request, but in here we don't have to do anything like that. And I think that's all we really need to do.

28:12 Everything else I think works just fine. So here we create the user message. In here we create the assistant message which is set to processing. Let me just go ahead and add a comment so we understand what we're doing here. In here we add a trigger to process the message and then we just return like hey we're doing something.

28:35 If you want to you can also add a little boolean like did cancel any messages and then you can return did cancel any messages true. Simply saying your network history you are aware that what happened here was some messages were canceled before a new one was created. So let's take a look at our situation now. I'm going to do the same thing. I'm going to start a completely new conversation and I will say in this conversation, I am running, right?

29:06 So now I wanna take a look at here, one processing message, one running background job. And now I'm gonna open a new conversation. Previously, the problem was if I sent something I would have two of them running. This should cancel the old one. So right now nothing should be different here but you can see this one was automatically canceled.

29:30 Only the newest one is processing and this background job was canceled. Only the newest one is processing and this background job was canceled. Only the newest one is processing and now I can explicitly cancel this one too and there we go. We now have a very safe way of sending messages without polluting our background jobs, without spending too many tokens. Because remember, all of these are going to be very complex AI requests.

29:54 You can't really afford to have a leak in your processing somewhere. That's why we are focusing on this so much. Excellent! So with just a few system functions and just a few routes we created a very thorough system for keeping track that only one message for a project is currently processing. What we ought to do next is implement the history dialog because at the moment there is no way to revisit previous conversations.

30:25 Let's get started by creating the past conversations dialog component. I'm going to go inside of source, features, conversations, components. And in here I'm going to create a new file, past-conversations-dialog.tsx. I'm going to go ahead and import, start with a directive, use clients and then I'm going to import format distance to now. I'm going to add the following elements from the command component, dialogue, empty, group, input, item and list.

31:03 I'm going to add if we have it let me quickly check use conversations hook from hooks use conversations and I'm going to import ID from convex generated data model. Let's go ahead and define the interface past conversations dialog props which will accept the project ID for which we are going to load the conversations open on open change and on select which will allow us to select a conversation. Now that we have those props we can define our component like so. Make sure to extract Project ID open, onOpenChange and onSelect. Now let's load all conversations for a project ID that the user has passed.

31:54 Let's implement a very simple handle select method which will accept a conversation ID called the on select, which we can, we might as well make onSelect required and then we don't have to do this weird optional chain. And make sure to close this dialog after the user selects an older conversation. Now let's go ahead and return. What we're going to return is the command dialog component. The command dialog component will have the following props.

32:28 Open and on open change, as well as a title and description. The title will say past conversations and the description will say search and select a past conversation. Now we have to define the command input with a placeholder which will be search for conversations. Then we have to add a command list and we have to add a command empty. This will serve as a placeholder if something the user wrote does not exist in our database so no conversations found.

33:03 Finally, let's go ahead and add the command group component with a heading conversations. Now we can go ahead and iterate over our conversations using conversations.map. Make sure to add a question mark since conversations can be undefined. Now in here we're going to return a command item. The command item will have a key of conversation underscore ID.

33:33 It will have a value which will combine the conversations title with the ID. This is to prevent multiple conversations from being selected at the same time in case they have the same title. And let's pass in our handle select right here. Now in here I see I have an error. So let me quickly see what that's about.

33:56 I think I'm missing a parenthesis. There we go. So I was missing one parenthesis here instead of command item. I'm going to open a div. I'm going to render span inside with a conversation title and I'm going to show the created at timestamp with another span.

34:17 And instead of using created at we will simply use underscore creation time because that's built in with convex. So the div has flex flex column and the gap point five. The span for the title has no styles, whereas the span for the timestamp has an extra small text and kind of a muted color for the text. Perfect, so That is our command dialog finished. And now we have to find a way to render it.

34:55 So we're gonna go ahead back inside of the conversations sidebar, which I believe is, yeah, it's in the exact same folder. I didn't even have to close it. And the first thing we're going to do is we're going to define the state which is going to control whether this is open or not. So Let's add it right here underneath the selected conversation ID. So we're going to call this state past conversations open and set past conversations open.

35:30 And now let's find a button which is going to toggle that. In our case, that's going to be this, the history icon button. So simply give it an on click, set past conversations open to true. Now, after this button right here, actually, well it doesn't really matter where we do this, but what I like to do with dialogues is I like to wrap my entire component instead of a fragment. And then I like to render it outside.

36:04 So past conversations dialogue like this. You don't have to do it this way. I believe all of these dialogues from Shazian use react portal, which moves them to the outer DOM. But I still like to semantically render my components how they are going to appear in the DOM. And this will be kind of above all this content.

36:27 Because when I see something rendered inside, I kind of expect it to be shown inside, but that's not the scenario for the dialogue But just to be clear you can render this wherever you want. It doesn't matter So what do we need to pass here? We need to pass the project ID so we know exactly what conversations to load We need to pass the open status. We need to pass onOpenChange and we also need to pass onSelect. And the onSelect will simply call our setSelectedConversation ID which we have already utilized in this project.

37:06 So, let's go ahead and try. If I go ahead and click on this button, you can now see that I have a bunch of these previous conversations. And you can see that we can see exactly this testCancellations which we were trying just a moment ago. So we have successfully implemented that too. At the moment we can't really do any useful search here.

37:29 You might see some results like this. This is because we are matching the ID, right? Because we combine the title and the ID of the conversation. And this is how it looks like if there is no results found. This makes, this isn't too useful right now because they are all named exactly the same.

37:48 And that brings us to, well, our next step, which is basically implementing the agent. I want to start by revisiting our ingest process message function as well as our API endpoint to create a message. So instead of API messages route.ts, right now when we trigger this message.send event, We only pass in the message ID because that's the only thing we need. But now let's extend it by passing the conversation ID, project ID, and the actual message. Now we have to strictly type those inside of the actual feature.

38:34 So features, conversations, ingest, process, message. The message event only accepts this two. So now let's enhance that. Besides the message ID, we will also accept conversation ID. We're also going to accept project ID and finally the actual message.

38:56 Now that we have a proper message event from here, we will be able to extract everything else we need. But let's go ahead and stop here and let's install a package which we are going to use to well set up the Ingest agent kit. So I'm going to go ahead and just do npm install at ingest forward slash agent kit like so. I didn't shut down my app. I still have all three running.

39:32 There is no need to shut down your app at the moment. And once this function has installed, I'm going to go ahead and show you exactly inside of package.json, which version I'm using. So for me, It is 0.13.2. And it's also a good idea to use the link on the screen to visit Ingest Agent Kit. In here you can find the actual documentation and you can find the exact quick start that we just did.

40:06 We already have Ingest installed and now we also added Ingest AgentKit. And you can see that starting with agentkit 9.0, ingest is a required peer dependency. You must install both packages together to ensure proper runtime. This is actually a good tip, perhaps we should run the whole thing again and if we're going to do that we should shut down the ingest tab. So yes, I'm going to run this again, simply because it's a peer dependency.

40:36 I'm not sure if it's any different if you install them in the same turn. Let me see. Is my package. Jason any different? This is my ingest agent kit version.

40:46 And this is my ingest version. So yeah, I mean, I would suggest just running the install at the same time, like this. After that, go ahead and run npx ingest cli at latest dev again. I don't think there should be any issues. If there are, it's probably version related.

41:07 You didn't write anything incorrectly. Alright, so we have that ready and now before we explore Ingest AgentKit any further, let's go ahead inside of source. Let's go inside of features, conversations, and in here, in the ingest folder, I'm going to create a file constants.ts. In here, we're going to define our system prompts. Obviously you can change and tune your system prompts to whatever you prefer but these are the ones I'm going to use.

41:44 You can find this using my source code which you can find a link for on the screen. It's free. And you can also find it in the assets file, which I usually show you. So go ahead and add the coding agent system prompt. If you're using the source code, simply navigate to the exact file I am in right now.

42:07 And let's go ahead and also add one more, which is going to be the title generator system prompt. Again, using the link on the screen, you can find these in my source code, or you can find them in the public assets repository. Now in order to prepare for tool calling, we're going to have to add a bunch of system queries and mutations in here in the convex system file. So let's go ahead and have that ready so later we can just simply query them. We're going to start with a very simple get recent messages.

42:50 We're going to use this for conversation context. Perhaps you can add little comments here. Use the for agent conversation context. This can help you understand why you need these system functions. The arguments it accepts are very familiar, internal key, conversation ID, and then the limit.

43:12 The limit is basically to decide, you know, how much context that we want to give to the AI. The last five messages, last 10 messages, or 100, right? Depending on how AI is advanced, you might be able to pass along all of them. After you validate the internal key, you're going to query the messages using the by conversation index and simply passing along arguments dot conversation ID. Make sure to order them by ascending and make sure to execute the collect.

43:43 Perfect. And then let's go ahead and simply limit them like this and then slice them. There we go. So that is the first one we have to do. The second one we have to do is a mutation called Update Conversation Title.

44:05 This will be used to get rid of the annoying new conversation for every single conversation that we have. Basically using the context of the messages, AI will be able to update the conversation title. The arguments in here are similar, so internal key, conversation ID, and the title. And in the handler here, we're going to First, as always, validate the internal key and then we're going to simply call patch on the conversation ID with the new title and we're going to refresh the updated at key simply so we know which one was the latest one we worked on. So you can go ahead and if you want to add a comment what this is used for.

44:54 So used for agent to update conversation title. Great. Next one we need is get project files. So I'm gonna go ahead and start preparing this. Get project files, accepts internal key and project ID.

45:19 The handler will validate the internal key and it will simply call all files by a project ID. As simple as that. So This will be basically used for list files tools, used for agent list files tool. We are going to have to create tools ourselves and we're going to have to define what those tools do. So when the user asks, how many files do I have in this project?

45:52 The agent will call list files tool. And then we're going to make the agent call this query right here, which will send back to the agent all the files in this project. So that's how that's going to work. That is get project files. Now we need get file by id.

46:19 So this right here which I've just pasted. GetFileById is a query which accepts internal key and file ID and very simply just returns the file ID. As simple as that. The next one will be update file. So this one is what it says.

46:37 It's used to update the file. If we instruct the agent to change something in the file, we need to create a tool which will do that. So let's go ahead and start by defining the handler which is always is going to validate the internal key. We're first going to check if the file even exists so we can return early if it doesn't. And then finally we're going to call a patch method like this.

47:07 So the only thing we're going to patch are the content and the updated at. And finally let's return arguments.fileid. So let's slow down a bit and simply write some comments. So get file by ID will be used for read files to The update file will be used for update file tool. And now let's go ahead and go on to the next one, which is create file.

47:42 And you can already guess what tool that's going to be using. So again, let's create the same mutation, create file, accepts internal key, project ID, name, content and a parent ID which is optional, right? So if this is inside of a folder, it's gonna have a parent ID. Otherwise, it's in the root of the project. And make sure to do the validate internal key.

48:13 So the first thing we're going to do is we're going to get we have to make sure that the agent doesn't accidentally create the same name, the file in the folder, right? So because of that, we have to get all files which are currently in this files project, in this files parent, or if there is no parent in the root folder. Because remember, when you create new files, I think we can actually try this out. If I call this hello.jsx, and if I try to do another one hello.jsx I get an error. We shouldn't be able to do that and neither should the agent.

48:54 So that's why we have to in this create file method in the system prevent the agent from doing that. So if we can find an existing file with the same name and file type file we're gonna go ahead and throw an error. Hey, this file already exists. You cannot create this. Otherwise let's simply insert a new file so inside of context database insert files add project id, name, content, type of file, parent id and updated at and lastly return file id so you've already guessed it The create file will be used for create file tool.

49:45 And now we're going to create a very, very similar one. It's just going to be used for bulk creation and it's going to be very useful in fact. So create files, not create file. This one is create files. So let me define the handler.

50:07 Let me go ahead and fix the indentation. And like this, there we go. So what are the arguments? Internal key, project ID, parent ID, and then an array of files, which is an object. I mean, each of the item in the array is an object which has a name and content inside.

50:29 So basically we can give AI 50 files to be created, but it can only be for a specific folder. So that's kind of the limitation. We can already go ahead and add a little comment here. The create files will be used for agents and let's add bulk create files tool because that's what it is. It's used to create files in bulk.

50:53 We also have to be careful here, right? So let's make sure that there are no existing files with the same name. Now the way we can do that is a little bit complicated. We're going to define an empty array and we're going to give it a very specific type. So the results is by default an empty array and inside of here we expect objects with name property, file ID property and an optional error property because any of these files in the array can be problematic.

51:30 So let's start by going over them. For file of arguments.file, first thing we're gonna do is we're gonna check if there is an existing file with the same name and the same type. If there is an existing file, we're simply going to push to the results array the name of the file, the ID of the existing file and the error file already exists and let's continue because we have more files to create and we can very easily just create a file otherwise so insert into files project ID name, content, type, parent ID and updated at and finally let's do results.push name, file.name and file ID and make sure to return results so that tool is used to create files in bulk now Let's go ahead and develop the create folder method. So the create folder method is almost identical to the create file one. Let's go ahead and copy this.

52:41 Let's go to the bottom and let's change this used for agent create folder tool. So this will be create folder and it's just not gonna have content. So it will have the internal key, project ID, name and parent ID. Then we're going to validate the internal key as always and we're going to query files so by project parent, all of this is Google but we're just going to change the existing query to check by file type folder and it's going to throw folder already exists. The content in here will be empty and the type will be folder like that.

53:30 Now if you want to you can create the equivalent create folders, but more often than not it's not that useful. I've seen agents call create files way more than create folders. If you want to, you can create it. In fact, it will be a nice challenge for you to create a tool that I don't write simply so you learn how to create different tools. All right.

53:59 Now we have only two left. So next one is rename file, which is used to allow the agent to rename a file if the user requests so. Internal key, file ID and the new name as the arguments. Then a check if that file actually exists and then again we have to check we have to check if a file with the new name the agent wants to rename it to exists in the same parent folder same with creating a file So let's get all the siblings using query files by project parent index, querying by project id and by file.parentid and collect. And now we can go ahead and check if it exists.

54:51 And we have to check if the sibling name matches the new name. We have to check if the sibling type matches the file type. And we have to of course check that we are not comparing with the file we are intending to rename, right? Because there is always one file with the same name. We should allow the user to rename to the same file.

55:16 I mean the agent in this case. So this will take care of both folders and files. Great. If existing ends up being true, we have to throw an error. So a file or folder named whatever it's named already exists.

55:35 As simple as that. And then let's go ahead and patch this arguments.fileid, name, arguments.name, updatedAt date.now. And finally return arguments.fileid. To be consistent I'm going to add a little comment here simply so I know that all of these ones are used for agent tools So this one would be for rename file tool. There is only last one we have to do which is the delete file mutation.

56:11 So I'm gonna go ahead and define it here. Can I quickly copy the comment here and just change this to delete file tool? There we go. It will accept internal key and file ID and after we validate the internal key, Let's check if the file even exists in the first place and then depending if this is a folder we have to recursively delete file folder and all of its descendants. So let's go ahead and define a constant delete recursive.

56:48 It's going to be an asynchronous function which accepts file ID to be a type of arguments.fileId. Then Inside let's first fetch the item. Item await context database get file id. If we failed to fetch it let's break this function. Then let's check if it's a folder delete all the children first.

57:17 So if item.type is equal to a folder, first let's query all the children by using by project parent query using the file ID that's currently passed here. There we go. Once we have the children, we're going to delete them using the very same method. So for child of children call itself. It's a recursive method And this way if we encounter another folder, it's going to do the same until all files are deleted.

57:53 Right, so that's why we are developing a recursive method. Now, we also have to delete the storage file if it exists. We currently don't really use this at all but it's important to not forget that we will have a way to store binary files. We will see this in action once we implement github imports. So if the file was a binary file, we should delete its equivalent storage key.

58:23 Otherwise, it's just going to be taking up space. And then finally, let's delete the file or folder itself. And then we actually have to call that function for the first time so await delete recursive with arguments file id that the agent provided and let's return arguments file id. This way we are completely ready to develop our execution tools. We don't have to worry about revisiting this file again for agent purposes.

58:53 At least I'm pretty sure this is all that we need. Most of these are pretty similar. You can always visit the source code if you're unsure or if you think you've made a mistake or if something doesn't work. Great. Now that we have this developed, let's actually go ahead and develop the agent.

59:12 So the work we have developed now is inside of process message. So inside of conversations, ingest process message dot ts. So far what we have in here is a cancel event. We have a proper on failure if something goes wrong, but we don't actually have a real message.send event. You can see we are just pretending to do some AI processing here.

59:38 So let's go ahead and change that. Let's actually start by doing everything that we need. Let's start by destructuring the proper items. So it's no longer going to be just message ID. It's now conversation ID, project ID, message ID, and the actual message.

59:57 The internal key check can stay the same. That's good. Now we don't have a delete sleep. What I would do is wait for database sync and give it maybe one or maximum five seconds. And to do, I'm going to add check if this is needed.

01:00:15 I'm doing this because during my development I've encountered some kind of out of sync state where an ingest agent can run faster than convex database updates and that kind of puts it in a weird position. I'm 99% sure this is not needed, but just to stay true to my original source code I will show you that I had this and we're gonna try and remove it later and then we're gonna see what happens All right. So the first thing we have to do is we have to change the conversations title so get conversation for a title generation check. And now let's go ahead and do the following. So we're defining a new step basically.

01:00:58 We're going to get the conversation using a step. So a way step dot run which we're going to call get conversation and in here we're very simply going to return a weight convex dot query API dot system get conversation by ID pass in the internal key and the conversation ID. This way we can use the conversation to see, Well first of all, does it exist? If it doesn't, let's throw a non-retriable error. It's not found, there is no need to retry any steps, right?

01:01:39 Now that we have the conversation ID, we can also fetch recent messages for conversation context. So again, another step we are defining here. Get recent messages. And inside of here, we are going to call our API system get recent messages query with the internal key conversation ID and we're going to limit it to 10 messages. Of course, depending on how good AI models get or more precisely how cheap AI models get, you might increase this, but the more context you give it, the worse results it actually gives you right now.

01:02:18 And it's just more expensive altogether. So that's why 10 is kind of a sweet spot right now. Great, so now we have the entire conversation object. We have the 10 most recent messages inside of this conversation for context. And what we can do now is we can build the system prompt.

01:02:36 So I'm going to add a little comment here. We're not going to build system prompt with conversation history. We're going to exclude the current processing message though. So let's start by defining the system prompt in a changeable let and we're going to give it coding agent system prompt which we can import from constants. Remember we developed this I told you you can find it in the source code or in the public assets, along with the title generator system prompt.

01:03:06 This is also optimized for Anthropic models. It should work just fine with Gemini models too, but Anthropic models work very well when it comes to XML. I am not sure what is the structure for other ones. But I think this should work just fine for all generally. Great.

01:03:24 So by default, the system prompt is just the default coding agent system prompt. But what we're going to do now is we're going to attempt to inject the context of the recent messages into the prompt. So first we're going to filter out the current processing message. So no need for that. We are just interested in the past, right?

01:03:44 So context messages, recent messages dot filter, and we are simply looking at the message, the current message ID and we remove it from the last 10 recent messages because we are not interested to adding that into the context. That message will be processed either way. And now we have to check are there any messages at all before this message? Because maybe it's the first one, right? Now if it is, we have to create that history text using context messages and then we have to map each message inside of the context messages with the filtered out current message so each message will return a template literal string message dot role to uppercase which it's basically gonna look like this it's gonna look like assistant, how can I help you?

01:04:39 And then it's going to be user, do this and this. So that's going to be the history basically. That's why we're doing this. Message.role colon message.content That's the structure we're developing right now. And let's also join with backward slash, backward slash, and.

01:05:00 This basically means empty space and I think AI likes it this way. And what we're going to do now is we're going to append to the system prompt all of this history text. So system prompt plus equals and I would highly recommend just using the link on the screen to copy this part or using you can again you can use the source code by going directly instead of the process message to just copy this of course you can also pause the screen if you want to write it out, or you can use the public assets folder. Great, now we have the updated system prompt. Now let's see if we should generate a new title or not.

01:05:45 So how do we define that? Well, we should have constants.ts in the convex folder. And in here, you should find default conversation title, new conversation. So what we're going to do now here is we're very simply going to check if the current conversation.title matches the default conversation title. Now, you can make sure to just import this from convex constants.

01:06:16 Is this the most nice way to do this? There are probably better ways, you know, inside of your schema, we could have nicely added something like is title updated, something like that. But I'm just using a very quick and easy way to do that. Also, I'm not sure how this works, right? I can import from convex constants.

01:06:42 So I assume everything is fine. I mean, convex is just a folder. So yes, this should be working just fine. This isn't a protected folder or anything. But in case this ever causes problems, you can always try moving constants somewhere outside and then just having duplicate default conversation title one in convex for the convex functions.

01:07:05 Let's see if I go find in folder and if I search for default conversation title, it looks like we're only exporting from here. So perhaps we don't even need this here. Let me go ahead and actually improve this if I can at the moment. So I want to find all the places where I'm using default conversation title. I can see I'm using it in the conversation sidebar and I'm using it in the process message But I'm not actually using it in convex.

01:07:37 So it makes no sense that this constants file is there Obviously I planned on using it there. Maybe I will in the future for now what I want to do is I want to move it and I'm going to move it So I'm going to copy the constants file and I'm going to set of features conversations right here So we already have constants instead of the ingest folder and want to keep that there. These are system prompts, right? So no need to pollute that so make sure that you have new constants dot TS inside of your Now, your app will break. So you have to revisit conversation sidebar.tsx and let's go ahead and simply change this to a much more...

01:08:32 There we go. This looks better right inside of the conversation sidebar Features conversations components conversation sidebar. You can now import that from the near constants, right? We've just we've just added here. Nothing else needs changing in the conversation sidebar.

01:08:51 The other problematic spot is the process message which we are developing right now, which we can again simplify. Let me see maybe another one back or maybe this one. There we go. So we have two constants. One inside of the ingest folder right here.

01:09:09 The other one outside of the ingest folder but still inside of features conversations. So that's a better place to have that in my opinion. Great. So if we should generate a new title, let's go ahead and create a title agent. So if you're wondering about this create agent and how it works, I highly recommend having agent kit documentation open because I mean this is how I learned how to use it.

01:09:43 We're now going to import create agent and entropic from ingest agent kit. You can of course import Google if you're using Gemini. We've already went over how you can define different models. It works very similarly to AI SDK. So let's actually do this.

01:09:58 Let's import create agent and anthropic from ingest agent kit. I'm going to go ahead at the top. I'm importing create agent and anthropic from ingest agent kit. Now my title agent here will have a name title generator It will have a system prompt of title generator system prompt Which I can import from dot slash constants alongside my coding agent system prompt It is this one right here the short one Once I've added the title generator system prompt, I have to define the model that is going to execute this agent. So in my case, that's gonna be Anthropic.

01:10:48 Again, in your case, this can be whatever you want. It can be OpenAI. It can be Grok. Let me see. Okay it's Gemini.

01:10:58 It's not Google. Keep in mind that I mean we're not executing any tools yet but in the past Gemini had very bad performance with executing tools. So that might be something you should be aware of simply so you understand that you didn't do anything wrong sometimes it might just be the model if possible in any way using anthropic models would be amazing But I understand if it's not possible, go ahead and try with Gemini either way. So use a cheap model for title generation. That's why I'm choosing Haiku because it's fast and it's cheap.

01:11:35 For Gemini, I'm not sure what's the equivalent but you can just use the same model. And you have to add the default parameters. Now, you don't have to do this for every model. For Anthropic, you have to. Otherwise, you have an error.

01:11:49 If you switch to Gemini, let's see, does it need it? No, not from Ingest. So if you try Gemini, you can see there's no error. But if I use Anthropic, it requires default parameters. So be aware of that too.

01:12:12 Temperature is gonna be 0 maximum tokens is gonna be 50 So we don't need too much power here, we're just generating a title. Now let's go ahead and actually run that agent here. So await title agent dot run, pass in the message and pass in the step. And in here we have the output. From that output, we have to find a text message from the assistant.

01:12:39 So the text message can be found using output.find and in that message search if the type is text and if the role is assistant. So we successfully found a response from the assistant in a form of text because the assistant can return tools, executions, a bunch of things. That's why we have to check if the type is text. And now, let's do if textMessage.type actually is text, simply because this can yield undefined, right? So if it is text, let's go ahead and extract the new title which we have to update the conversation for.

01:13:17 So first things first, if typeof textMessage.content is equal to string, in that case let's make sure to do textMessage.content.trim. Otherwise, let's do textMessage.content.map. For each content we have, simply return the text part of that content, join all of it, and then trim it. What we're doing here is we're handling various scenarios in ways AI assistants can respond. You can try and be strict with AI assistants to always to tell them to basically do this, but you can never rely on them fully.

01:14:02 That's why we're doing the trimming and the joining. And then finally, if we manage to obtain title, let's go ahead and do await step.run updateTitle. Let's do updateConversationTitle. It's going to be an asynchronous method. Let's do await convex.mutation API dot system update conversation title which we've developed at the start, pass in the internal key, conversation ID, and the new title.

01:14:37 So you remember, we developed this update conversation title used for agent to update the conversation title, accepts internal key, conversation ID, and very simply the new title. And we are using it right here to let the agent do the update it needs to do. Perfect. So perhaps We could already try this out. I think...

01:15:04 Let's see. Let's see if we can try this out. I'm gonna refresh my Polaris app here. I'm gonna start a brand new conversation and I'm gonna try and ask it something specific. Set up a React plus Vite project.

01:15:21 I'm gonna try something like that and let's see if it will run this or not. Because I'm not sure we, It's not doing anything and I think it's because, let's see, should generate title. I think all of this passes. I just think that the title agent, I'm not a hundred percent sure if it's up to us or if it's up to something else. What I'm interested in is the actual Ingest server here.

01:16:04 Oh, it looks like there is error here. Oh, my balance is too low. Okay, so that's the problem. I'm gonna go ahead and just update my Anthropic balance. And just as I updated my balance, you can see that the next one succeeded.

01:16:21 So let's go ahead and look at it again in action. So create a React plus Vite project. Right now It is named new conversation, but after it processes this, it should change this to reactivate the project setup guide. There we go. So our first agent actually works and it works quite well and it's quite cheap and fast.

01:16:48 And basically this is a super primitive version of what we're going to build next, which is another agent, but it's going to be a coding agent. And it's going to be Way more advanced in a sense that well it will be able to call and execute tools. So let's go ahead and start doing that. Create the coding agent with file tools. We're going to start by defining the coding agent.

01:17:17 And let's go ahead and give it the name I'm going to call this Polaris. I'm going to give it a description of an expert AI coding assistant. The system will be a system prompt. Remember we have extended the system prompt with the history text right with all the previous messages so now the system prompt is much better. And now we have to define the model again.

01:17:42 This is your own choice. Again Anthropic is really really impressive when it comes to tool execution. So I'm going to go ahead and use Cloud Opus simply because I found it to be the most impressive. It's also very expensive, so be careful. You can also do it with Haiku, but the better the model is, the better the results are gonna be, right?

01:18:10 And you need to pass default parameters. So temperature 0.3, max tokens, 16, 000. So these are random limits I found work well. You can of course tweak them if you know what you're doing. And in here we're going to add tools.

01:18:27 Right now we have No tools at all. So how about we Create our first tool. I'm trying to think of the simplest one of these we can do Well, let's just start with a read files tool. That's gonna be the one we're gonna need So I will simply close everything. That's not process message and instead of the ingest folder, let's go ahead and create a new folder called tools and inside read files dot TS.

01:19:06 Now I'm going to import Zod and create tool from ingest agent kit. Again, I would highly recommend that while you're doing this you're also reading the documentation on tools. So you are not, you know, just blindly following, but understanding where I found this information and how I know how to write it, right? Alright, So once you've imported Zod and create tool, let's also import the convex util, API and ID from convex generated folder. Create an interface, read files tool options to be internal key and a string.

01:19:48 And let's also define the params schema here. So we're going to accept file IDs which is going to be an array of IDs and we're going to throw errors if it's empty or if the array is empty. So now let's go ahead and let's create Create Read Files tool which accepts the internal key and it simply uses the interface we defined above. Now let's go ahead and immediately return create tool. Let's give it the name of read files.

01:20:26 So that's the name of the tool. Let's give it a description. Descriptions are actually quite important. The same as the name of the tool. So read the content of files from the project, returns file contents.

01:20:39 Now let's go ahead and define the parameters. So the parameters are a z.object, file IDs, and then inside we're going to define an array of string with describe array of file IDs to read like that. Then we have a handler in which we can get the params And we can get step Give it an alias of tool step for easier understanding Let's go ahead and do parsed params schema.safeparse params. This way it will throw an error if the tool is attempting to fire with weird parameters if they are not what we expect, right? So if not parsed.success, let's go ahead and return error, parsed error, issues, first in the array, message.

01:21:53 And this will be sent to the agent. So the agent can then retry even if it fails So it won't make the same mistake twice Let's destructure the file IDs from parsed.data. So now we have completely valid file IDs at this point or at least we should have. Let's go ahead and open a try and catch block here and I'm going to do return await toolstep question mark dot run read files. It's going to be an asynchronous method and in here let's go ahead and define the results.

01:22:33 Results by default are going to be an empty array but what we expect here is an array with objects. Each object should have an ID, a name and the content inside. So now that we have that defined let's run them through a for loop. For const file id of file ids let's go ahead and fetch the file using await convex query api system Get file by ID. Pass in the internal key.

01:23:05 Pass in file ID. File ID as ID. Files. So we don't have any type errors. If we successfully fetch the file and the file has content.

01:23:18 Let's do results.push id file id name file.name content file.content. There we go. Now that we have that defined let's go ahead and check if results.length is equal to 0, let's return an error, No files found with provided IDs. Use list files to get valid file IDs. And writing this error message I realized before we can actually test this out, we will have to write one more function.

01:24:07 Sorry about that. But yes, I completely forgot that the agent right now doesn't really know where to pass the file IDs from, right? How is it supposed to know what file IDs to read? Right, so this function will actually be used once the agent is instructed to read a specific file ID. So still, we're gonna need this function either way we're just not gonna be able to test it out right away.

01:24:37 So make sure to return stringified results and in the catch method here grab the error and return error reading files error instance of error display the error message, otherwise, unknown error. Like so. Let me go ahead and check what the... Okay, I should not do something here. I should end that here.

01:25:13 There we go. Alright, so that's our first function implemented. Now that we have read files tool, let's go back instead of process message here and let's go ahead and pass create read files tool. You can import it from tool read files and in here simply pass the internal key. Now, technically we could have just as easily used the internal key right here, right?

01:25:43 We could have defined const internal key process.environment blah, blah, blah. But if we pass it as a prop here, it's kind of already validated at this point, you know, simply because we have used it in all of these previous steps and we've checked if it's valid here, right? So I kind of feel like we can just pass it. I don't know. If you feel like this is not a good practice, you can of course define it here directly.

01:26:13 Alright so that's the create read files tool. Now let's go ahead and implement create list files tool. I'm going to copy read files and I will rename it to list files dot TS. So let's check. First things first I will modify the interface which is now called list files tool options It will accept the project ID and the internal key It's not gonna have any params since it's just gonna read all the files that we have So let's go ahead and change this to, it's no longer called create read files tool, it is create list files tool, which uses list file tool options and extracts project ID and the internal key.

01:27:03 The create tool itself will have a name list files. And the description is also as always important and you might want to copy this one from the source code as well simply because it's long. I mean or you can just pause the screen and you know write it out. List all files and folders in the project. Return names, IDs, types and parent ID for each of them.

01:27:28 Items with parent ID null are at root level. Use the parent ID to understand the folder structure. Items with the same parent ID are in the same folder. So I'm just giving it a little bit of context of the of what it will receive, right? Now the parameters are just going to be an empty object so write them as such.

01:27:50 To skip the params you can just write an underscore here. Now we're not going to have anything to parse so we can remove all of that. Now inside of the try method here Let's go ahead and do a similar thing. So return await tool dot step. This will be called list files.

01:28:07 And it's going to be a little bit simpler. So I'm going to remove everything inside of here. Can I do that? I think I went a bit too much. Okay, and we still okay, let me just bring this back.

01:28:25 I think I just needed to remove this. There we go. All right, so tool.step list files. The first thing we're going to do is call convex.query API system get project files. So we developed this specifically for list files tool And all it does is list all the files in the project using the internal key and the project ID.

01:28:52 So we make sure that we pass the internal key and the project ID here. Perfect. Now let's go ahead and sort folders first, then files alphabetically. So we've already done this before for our file explorer because it displays it in the exact same file. We're just giving the agent the exact same view now.

01:29:15 So we're sorting the files by type first and then using a simple locale compare for the name alphabetical. And then let's go ahead and structure the file list. So using those sorted files let's go ahead and create an array of objects with an ID of the file, name of the file, type, and a parent ID or null. Just make sure it's not undefined. So, parent ID or null, like this.

01:29:45 And then, let's go ahead and let's return. JSON stringify file list. The error will be error listing files or unknown error. There we go. Another tool finished.

01:30:00 Let me remove the extra space here. Let's go back instead of processing message here and let's do create list files tool. So from list files and let's go ahead and let's pass in the project ID and the internal key. Now I will order this one first simply because it's going to be used more often than the read files tool. And of course I've messed up my imports, so let me just go ahead and quickly fix that.

01:30:31 So let's see. Is it still... Ok. So, in here, internal key and project ID and I change the order. That's it.

01:30:46 So I think that at this point, we might already have something. So actually, I thought we can already try it, but we can't try it simply because the coding agent is unused. So we defined the tools, great, but there's nowhere to try it out, right? So, okay, I'm going to stop at these two tools simply because I think it's better for us to start seeing some results before we write all the tools because we just won't be able to see the results until we finish the entire thing. So, how do we call this coding agent?

01:31:23 Well, what we have to do is we have to create a network. Again, this is a term from Agent Kit. So, I highly recommend to read about networks to understand how they work and why we need them. Specifically, we need them because we need to create loops. We need the agent to iterate with itself, with its state and call various tools until it determines that what there is no more work to be done.

01:31:53 That's why you need networks to create loops, right, because agents, smart agents work in loops until they're finished. So in order to do that we're using the create network. So let's import create network from ingest agent kit. In the create network give it a name, Polaris Network. Let's give it agents which is coding agent just a single one and then we have something called max iterations.

01:32:26 So I keep this at 20 simply because I find it to be a sweet spot. Basically, iterations are how many loops you will allow it to create before you abruptly stop it. This depends on your budget most of the time. So if you have an infinite budget, you might run this infinitely, but you most likely want to keep it at something like 20. For example each tool execution is an iteration right so if you have a thousand tools well I mean it's probably not going to use all tools all the time But depending on how smart your agent is you might need more or less iterations.

01:33:06 Again this is something you can read more and understand by reading the documentation here. So routing and maximum iterations. You can see why and how you can define what it is. So specifying a max iteration option is useful when using a default routing engine or a hybrid router to avoid infinite loops. Yes, I mean it's not like it's going to run always infinitely but there is a chance it can get stuck and it can cause you a very big bill.

01:33:34 Great. Now let's go ahead and define the actual router here. So we extract network from here and what we have to do is we have to get the last result. The last result is network.state.results at minus one. Now let's go ahead and check if we have a text response from that last result.

01:33:59 So last result, question mark dot output, sum, message has a type of text and it's coming from an assistant. Similarly to how we checked if the title is finished, we checked if the assistant message is text. So we are now doing the same here. We are checking if it's finished. So just as we developed hasTextResponse, let's do hasToolCalls.

01:34:24 It's the same thing. Last result output sum message.type is tool call. This should be type safe, I believe. Yes, tool call. So, now it depends which model you're using again.

01:34:41 For example, Anthropic can output both text and tool calls together. JMI, for example, doesn't do that or OpenAI. So, what I'm gonna do here is only stop if there is text without tool calls, final response. So, if we detect that an assistant just sent us a text response and assistant has no more tool calls, we return undefined which basically breaks the router or it signifies that it's finished. Otherwise we return coding agent which symbolizes run another loop basically.

01:35:22 So basically this is the code which decides should we do another iteration, or should we break it, we're finished. All right. Now, let's actually run the agent. Like this. Result, await network.run with a message.

01:35:40 Now let's go ahead and let's extract the assistant's last text response from the last agent result. So again, last result result.state results at minus one. Then let's find the text message. If message.type is text and message.role is assistant. Let's go ahead and give it a default assistant response.

01:36:05 I processed your request. Let me know if you need anything else. So we're going to use this if we are just unable to find the last message of the assistant, right? Depending on which AI model you use, this code might work or maybe it expects something completely different. These AI models change very frequently.

01:36:25 But you will be able to debug this yourself if you've come this far into the tutorial. I will show you how you can see the output and then you will be able to tweak things, maybe even ask AI to help you with it, right? So we now have to check if we can modify the default assistant response. So if the type of text message content is actually a string, we're going to use textMessage.content. Now on an off chance, it's not a string, but instead it's an array we're going to map over the content, return the text part of the content and then join it, like so.

01:37:01 There we go. And now that we have the assistant response, we can actually update the assistant message. So I'm going to add a little comment here. Update the assistant message with the response. This also sets the status to completed.

01:37:15 So, update assistant message and this time the content will be assistant response. There we go. And what I like to do at the very end here is simply return success, true, message ID and conversation ID. So it's easier to debug if it goes wrong. All right.

01:37:37 I think that now we might be able to try this out. So I'm just going to copy this entire file here. I'm going to create a new file functions.ts and I will paste it inside. So I believe this is now saved. It is, perfect.

01:37:56 So I will start a new conversation and I will focus on the ingest server too. And let's see, so what files do I have in this project? Did we do this correctly? Did we forget to plug something in? Let's just see.

01:38:12 So right now it's generating the title, list project files. And now you can see it is creating the network. It is listing files successfully. So that's great. You can see it managed to call the tool list files.

01:38:28 You can see it didn't even need the read files tool. It just needed a list of files, which gave it back an array with functions.ts, a type of file, parent ID null. And I think the response should be, there we go. You have two files in this project, both at the root level. Functions.ts, a TypeScript file, hello.jsx, a React.jsx component file.

01:38:55 There are no folders in the project currently, both files are located at the root directory. Amazing! So we successfully made an agent use a tool called create list files and give us a response. Now I've succeeded with this with Anthropic. I'm not sure what the results are going to be with Gemini.

01:39:21 Technically it should work just as fine with Gemini, but you saw me have a bunch of problems with Gemini before. I keep running into timeouts and limited requests. I'm not sure if I'm doing something wrong, but because of that I just have to, you know, continue developing with reliable models. Otherwise, it's just very, very difficult to get a proper, reliable result for the tutorial. Still, if you are using Gemini, let me know in the comments if it works, if it doesn't work and how you debug it if you encountered any problems.

01:39:54 The code itself should work just fine for any model. This isn't tailored to Anthropic. The only thing I have tailored to Anthropic is making sure right here that we make sure there are no tool calls because Anthropic also responds with tool calls. For other models you can just check if there's a text response it's most likely the last response. With Entropiq you can't really be as reliable.

01:40:21 All right all that's left to do now is just create more tools. So we only have two tools now and we have to add update file, create file, create folder, rename file, delete files and also scrape URLs for firecrawl. I'm gonna start with creating the update file tool so we finally see some changes in action. I will copy read files, I will paste it and I will rename it to update file.ts. Then I'm going to update the interface here to be update file pull options with internal key.

01:41:00 I'm going to modify the params schema to accept an individual file ID as well as the new content we attempt to update I'm going to update the entire function here To be called create update file tool accept this new params and then return create tool. I'm going to change the name here to be update file. I will change the description to be update the content of an existing file. And I'm going to modify the parameters to match the schema. File ID with a describe of the ID of the file to update.

01:41:39 Content, the new content for the file. We're going to start by parsing so this can stay the same in fact and the only thing we're gonna extract from the parsed data will be an individual file ID and the content. So previously it was IDs, but now it's file ID and content. So now what I'm going to do is I'm just going to validate if the file exists before I even run this step using convex.query API system getFileById. I've mentioned it will be used by read files tool but it can be used by various tools basically by agent tool right.

01:42:25 Get file by ID accepts the internal key and the file ID And in case the file doesn't exist, I'm just going to go ahead and return an early error and instruct the agent to use list files to get valid file IDs. And I will also check if the file type exists but its type is folder and I will give the agent similar instructions file is a folder not a file you can only update file contents. Great So after we do this early checks, which kind of improves the user experience and they don't have to, because if you don't do this early errors, it will get stuck in trying to repeat the API call because it thinks that maybe the network timed out or something. Basically, the agent doesn't know what it did wrong. So that's why you kind of have to do early returns with descriptive errors to let it know, hey, this is not a file ID, or this is the wrong file ID.

01:43:30 You're probably not looking for that. So the tool step is called update file. And in here, well, it's actually quite simple. Let me go ahead and remove everything in here. The update file will simply call a convex mutation API system update file with internal key file ID and the new content.

01:43:56 And let's also make sure the agent knows it was successfully updated. In the error I'm going to change this to Error Updating File. Unknown error. There we go. That's it.

01:44:10 Now I'm going to go ahead back inside of processing message and I'm going to pass in create update file tool with the internal key. Make sure to invert it like so. So let's see if this will work now. Update functions. Ts to be a simple hello world console log.

01:44:38 Nothing else. We are going to see. Does it have enough tools? Does it need any more tools? Also keep in mind, just because it succeeds today, it might fail tomorrow.

01:44:50 AI models are non-deterministic. They are unpredictable. Sometimes it might work a thousand times and then fail a thousandth first time for no reason at all. It just got confused. Sometimes it's context, sometimes it's something else.

01:45:05 All right, let's see what did it do here. I've updated functions.ts. It did, perfect. So The reason it didn't immediately update here is because of the way we defined this component right here. It cannot immediately receive updates because it will reset your cursor position and then it's very annoying to type code.

01:45:30 So just change files or change code something or restart or refresh so you can see the update here. There we go. It works. There are some bugs in styles here. We're also going to take care of that.

01:45:42 Don't worry. But the more important thing, it works. We can now ask an agent to update a specific file for us. It can find that file and it can update it. And now we're just going to continue, you know, creating tools for other things.

01:45:58 Now I'm going to copy update file and I will call this one create files so a bulk update and again if you don't want to write all of this one by one you can just visit the source code of course but I will try my best to at least kind of explain what's going on here, even though I think at this point you understand what's going on, right? So create files tool options accepts project ID and the internal key. Params are parent ID, an array of files, and each of those files has a name and a content. Content can be empty, right? We don't require that but at least one file needs to be created if we're using this tool.

01:46:42 Then we're going to change the name and the props of this tool to create create files tool. This error I believe is just a TypeScript server error. If I restart my VS code it goes away. So we use project ID and internal key here. I'm going to go ahead and change the name of the tool to be create files.

01:47:09 I'm going to go ahead and give it a bit of a longer description simply so it works better. Create multiple files at once in the same folder. Use this to batch create files that share the same parent folder. More efficient than creating files one by one. So the reason I developed this tool in the first place is because it's very easy to hit maximum iterations if you use just you know create file one by one so this bulk create tool actually helps a lot.

01:47:38 Now the parameters are going to be a bit different more of them whoops Let me just go ahead and indent this properly. There we go. So parameters are going to be an object. The first item in the object is going to be the parent ID, which is a type of string. And it's the ID of the parent folder.

01:47:59 Use empty string for root level. Must be a valid folder ID from list files. So we indicate to the agent again if you forget how to get the ID just list all files. The second argument in the parameters is an array of files and inside of that array we have an object. The object needs to have a name.

01:48:21 The file name, which is going to be created, including the extension. And the content, the actual file content. And we describe this as an array of files to create. Right, so exactly what we defined here, just with describe methods. So in the handler again we do the parsing which is fine but we don't really expect file ID or content.

01:48:48 Instead, we expect parent ID and files. Now, what we're going to do is I'm going to clear up this code here simply because let me see. I'm going to clear this up like so and I'm going to clear up everything in the try method. We're going to start by calling the tool. So return, Okay, like this.

01:49:17 Return await toolstep.run create files, asynchronous function. Let's go ahead and first resolve the parent ID. Now let's check if there is a parent ID and if the parent ID is not an empty string. We're going to open another try method here. So make sure it has an equivalent catch method down here.

01:49:45 Let's go ahead and do resolved parent ID to be parent ID as ID files. Then we're going to use convex query and API system get file by ID to get the parent folder. So we pass in the internal key and file ID, resolved parent ID. And we're basically doing this again to throw early errors. So if there is no parent folder, we're going to do parent folder with ID, Parent ID is not found.

01:50:17 Use list files to get valid folder IDs. Again, we're just doing this for early return methods. So it will be much faster if we throw an early error than if it has to figure out what it did wrong. So only if the agent decided to pass parent ID because the agent is allowed to not pass a parent ID if it wants to create files in the root folder. So if it passes a parent ID and if we detect that the parent folder cannot be loaded using that ID, we simply tell it right away, hey, something's wrong, you cannot do this.

01:50:57 So if parent folder doesn't exist, we throw an error. If parent folder type is not a folder, we throw an error. The parent id is a file, not a folder. Use a folder id as a parent id. We're just making sure it can understand what it's doing wrong.

01:51:15 And in here we just throw like an overall error invalid parent ID use list files to get valid folder IDs or use empty string for root level great and now let's go ahead and simply do convex mutation API system create files passing the internal key, project ID, parent ID and files now in here let's see which files we have successfully created. Because remember, create files will keep track of all the results which have failed, such as file already exists, and all the files which have succeeded. So we are filtering out all the files which have been successfully created and those which have failed. So be mindful of the exclamation point here. And let's start with a response.

01:52:19 You need to give the agent a response so it knows if it did good or not so created for example five files And then let's go ahead and do if created.length is larger than 0. Instead of the response, let's go ahead and actually display which files were just created. And then we're going to simply do the same if some files have failed. Like so. So let me just quickly zoom out so you can see how this looks in one line because when it collapses, it's kind of hard to understand.

01:52:56 So you can pause the screen and copy it. All right, So we're just doing this so we have a nice response for the AI model. There we go. And in here, let's do error creating files like this. There we go.

01:53:16 Another tool finished. We're going to try this one later. I just wanna work on knocking out all the tools that we need. So this is the create, create files tool. So import it.

01:53:33 There we go. The next one we're gonna do is the create folder tool. Which will accept project ID and the internal key. So we don't yet have it. So we're gonna go ahead and develop it right now so I am going to copy create files paste it here we name it to create folder I'm gonna go ahead and change the interface there we go it accepts the exact same props the params are gonna be different though It will accept the name of the folder and the parent ID in case it's a subfolder.

01:54:16 I'm going to change the props here and the name of the function to create create folder tool and the same with the props here. Then I'm gonna go ahead and change the name of the tool to be create folder. I'm gonna go ahead and change the description to be create a new folder in the project. I'm going to change the parameters to be an object. First item in the object will be the name, the name of the folder to create and the second is going to be the parent id.

01:54:51 So the id not the name of the parent folder from list files or empty string if we are creating the folder at the root level. Great! So parsing works exactly the same. Except we are not expecting parent ID or files. So let's just do name and parent ID.

01:55:14 Now let's go ahead and again call await tool step within try and catch, except this one will be here in the create folder. So let's start by validating the parent ID if it is provided. So let's check. If we have parent ID, let's try and get the parent folder using API system get file by ID and parent ID here. Let's just do as ID files.

01:55:51 If there is no parent folder, let's go ahead and throw the exact same error. So parent folder with ID, parent ID is not found. Use list files to get the correct ID. You can of course, I mean, you can use the same errors. And we do the same thing if the parent folder is not a type of folder, basically as much as possible to let the agent know what it did wrong.

01:56:16 Alright, So now that we have the parent ID, let's go ahead and simply call the mutation. So new folder ID await convex mutation API system create folder. There we go. Pass in the internal key project ID name. And then if we have a parent ID pass it as parent ID and cast it as ID files or pass along undefined.

01:56:45 You can remove the created and the response. And we're very simply going to return a static response. Folder created with ID, folder ID. Error creating folder, like this. Let's go back inside of process message and let's go ahead and import create create folder tool like so alright at this point let's just try it out a bit so we have two that we have to check.

01:57:16 Can it create multiple files and can it create a folder? Create two files named foo.tsx and bar.tsx. Put simple content inside. So let's see if it is able to use the create files tool. That's the first thing we're going to do.

01:57:39 So right now it's generating the agent and it's generating the network and it uses the create files tool. And you can see the output. This is what we were doing. Created two files, foo and bar. There we go.

01:57:53 Foo and bar and very simple props inside. Now I'm gonna say create a simple folder called source. Nothing more. So I'm trying to give it very simple, explicit instructions. Right now, it should use the create folder function and it is folder created with ID successfully and I have the source folder inside.

01:58:18 Amazing. Both of our tools are working just fine. So now let's go ahead and add Rename File Tool. The Rename File Tool is quite similar to update file. So close others.

01:58:35 OK. I'm going to copy update file and I'm going to rename it to rename file tool. Now I'm going to go ahead and change the interface. The prop is exactly the same. The params are quite similar except instead of content is going to be the new name.

01:58:54 Now I'm going to update the props and the name of the tool itself to create the rename file tool and rename file tool options. I'm going to update the name of the tool and the description of the tool as well. Then I'm going to update the parameters to match new name. Perfect. Now in here, instead of extracting content, we will extract a new name.

01:59:20 And this can actually be the same. So we are validating if the file exists before running the step. If it doesn't, we throw an error. If the type is folder, well in this case, no. We should allow read file, I mean rename file to rename folders that's perfectly fine.

01:59:41 So remove that check and here let's go ahead and make sure to call this tool step run to be rename file. And we are simply calling rename file. And we are passing along new name. That's it. And this will be let's go ahead and give the agent as much information as possible so this will be renamed file name to new name like so And in the error it will be error renaming file.

02:00:20 Perfect. Let's go inside of process message and let's add create a rename file tool and pass in the internal key. Great. So what else do we have to do? We have to implement delete files tool.

02:00:36 So I think out of all of these, the most similar one might be create files. So I'm going to copy it and rename it to delete files So let me go ahead and start with this Interface delete files tool options Let's change the params to be file IDs It's gonna be an array of strings like so. As always I'm going to update the tool name and the props. I'm going to go ahead and change the name of the tool to be delete files. I'm going to go ahead and change the description.

02:01:21 Delete files or folders from the project. If deleting a folder, all contents will be deleted recursively. And then I'm going to repeat the parameters once more with the proper describe handlers. There we go. Now, instead of handler, we parse as we usually do, and we extract file IDs from here.

02:01:46 So now, what I want to do is I want to validate if all files actually exist before we run the step. So files to delete are going to be an arrays and in each of that object inside of the array we expect an ID, name and a type. So let's go ahead and very simply run a for loop. So for each file id inside of the parsed file ids I'm going to go ahead and attempt to fetch the file using convex query api system get file by id passing the internal key and passing the file id itself then I'm going to go ahead and check if there is no file let's return an error file with ID file ID not found use list files to get valid file IDs so the agent doesn't attempt to delete files which don't even exist. Otherwise, let's go ahead and push them to the array.

02:02:50 Files to delete. Push the id of the file, the name and the type of the file. There we go. Now we are ready to open try-catch and call a tool. So the tool is gonna be called delete files.

02:03:02 So let's go ahead and change this. Let's go ahead and let's modify everything inside. I think it's easier this way. So instead of delete files, let's start with results, which is an array of strings. We're going to open another for loop iterating over our array files to delete which we have confirmed exist we're then going to go ahead and call convex mutation api system delete file for each of those files along with the internal key and then in the results we're simply going to go ahead and push deleted file or folder and then the name of that file or folder successfully and then finally what we're going to do is return results.join.

02:03:52 So to the agent we're just going to return a plain string saying these are the files we have deleted. Error deleting files So the agent doesn't get confused. All right. Now let's go inside of process message. Let's add create delete files tool and the internal key inside.

02:04:15 Great. So I believe there is one more tool to create and that is the ability to scrape URLs using Firecrawl. So, I'm going to go ahead inside of tools here and this time I'm just going to create a new one from scratch. So scrape URLs.ts. Let's go ahead and import Zod and create tool.

02:04:39 Let's go ahead and import params schema which will basically just accept an array of URLs. So array of URLs like so. Let's go ahead and export create scrape URL tool. Let's return to actual create tool, util. I'm going to indent this back.

02:05:00 Let's go ahead and give this a name of the tool to be scrape URLs. And then I'm going to go ahead and write a description. Scrape content from URLs to get documentation or reference material. Use this when the user provides URLs or references external documentation returns marked down content from the script pages perfect now let's go ahead and actually add the parameters there we go And I believe the last thing we need is a handler. So we get rid of those errors.

02:05:39 There we go. So parameters are just an array of strings. There we go. OK. So inside of the handler itself, as usual, we get the params and we destructure the step and we rename it or alias it to tool step.

02:05:55 We do the usual parsing here. Parse, parse schema, save parse. We break if there is any error with parsing and now from the parse the data we can extract the URLs. Let's go ahead and open try and catch like so. In the try method we're going to go ahead and open the actual tool called scrape URLs.

02:06:29 Let's go ahead and prepare the results. So it's going to be an array of URLs and the content behind the URLs let's go ahead and do for const URL of URLs open another try and catch in this inner try let's go ahead and use result from await firecrawl.scrape passing the URL and formats into markdown now that we have the result let's simply check if it's valid and then push it to the array. So if result.markdown is received, simply push it and mark its content as result.markdown. In the catch, we are going to push that we failed to do something. So if whatever error happens, just fail to scrape that URL.

02:07:25 Perfect. Now let's go ahead and do if results.length is 0, no content could be scraped from different-rated URLs. Otherwise return JSON stringify results. And then in the catch method here, let's simply return error scraping URLs If error is instance of error, display the error message. Otherwise, fall back to unknown error string.

02:07:55 That's it. Let's go inside of process message and let's add the last tool, which is create scrape URLs tool. Perfect! So I believe our agent is quite capable at the moment. So if I go ahead for example and start a completely new project now.

02:08:15 Let me go ahead and do that. And if I tell it to, for example, create a React plus Vite app and a simple to-do app inside, I think it should be able to do that. Again, AIs are non-deterministic. You can get as many successful results as you can get bad ones, right? But the more tools you give it, The more descriptive you are, the better that these tools get, and the more tokens and the higher budget you have, obviously the better results you are going to have.

02:08:56 So what we've developed right now is a good harness, right? And you can always improve this harness. I'm kind of limiting myself so we don't this tutorial can go on forever. Right. I could be doing this for months but I have to call it quit somewhere.

02:09:13 So that's what I'm doing right now. If you want to create more tools, if you see any gaps for optimization, go ahead, do it. And you can see how this is happening right now, right? Our agent is just calling a bunch of tools, create files, create folder. Here it is.

02:09:28 You can see how it's creating all these files in real time. I suggest like opening these folders so you can actually see when the new files are created. So what I'm going to do now is I'm just going to pause and you know maybe it's going to be successful maybe it's not Sometimes it will be successful the next time you do it. And these things you see, like 1, 2, 3, 4, 5, those are iterations. So if you see it's getting close to 20 or it often gets interrupted near 20, you might have to increase your amount.

02:10:02 And here we go. In my example, it was successful. It's telling me how to install it, how to run it, and where to see it. Of course, we can't really do that right now simply because we don't have the preview ready, but that's what the preview is going to be for. It's going to be able to actually install, run, and open the browser.

02:10:21 But looking at the code, I see no reason why this shouldn't be working just fine. So yeah, very, very impressive clone of Cursor already. I mean, we are basically finished. At this point we have finished majority of things that a very basic cursor clone would have. Cursor for example doesn't even have preview.

02:10:41 Doesn't even have export. So those are the things we're going to do next. We're going to do export, we're going to do preview and I'm going to make sure that this markdown looks a bit better because right now the contrast behind this message is kind of invisible and we've saw some markdowns where it just looks bad. So that's going to be a super simple styling fix but I'm going to do that later. Amazing.

02:11:02 So that marks the end of this chapter. We implemented proper message cancellation which you can see now is very useful right. If I go ahead and start a new project now and if I tell you no create a React plus V app and a simple weather app and my agent starts going wild. You know, he starts calling all of these tokens, blah, blah, blah. Previously, we'd have no way of stopping that.

02:11:27 Now we can just press stop. That's it. Request canceled. You can see that it's canceled. It's no longer spending any tokens.

02:11:34 So that's why that cancellation was important for us to develop initially. Perfect. So let's go ahead and merge all of this. So I'm going to shut down a bunch of my apps right here. And let's see chapter 13.

02:11:48 So this will be Git. Well, we can check out first, yeah. Git checkout new branch 13 AI agent and tools. Git add dot, Git commit. It's going to be 13 AI agent and tools and then git push uorigin 13 AI agent tools like so.

02:12:14 Once this has been pushed I'm going to go ahead and go inside of my repository here I'm going to open a new pull request and then I'm gonna go ahead and review it. And here we have this summary. New features. We added ability to cancel in-progress message processing. We added past conversations history dialog with searchable list.

02:12:42 We introduced comprehensive file management tools. Create, update, delete, rename files and folders. We added URL content scraping capability using firecrawl and our scrape URLs tool. Implemented dynamic conversation title generation. We enhanced message status tracking with processing, completion, and cancellation states.

02:13:07 And now we have 8 comments but they're nothing scary, don't worry. So the first thing here is that we are returning an invalid request on the body here it says we are so okay request schema parse throws on validation failure and will surface as 500 oh so I should be using safe parse instead of parse All right I see yes because technically this failing shouldn't be an internal server error 500 it should be just a user facing error 400 user sends something incorrectly. All right yes And now for the rest of the comments, it is simply telling me that I need to handle toolstep to avoid returning undefined. So I'm not exactly sure what it means here, but yeah, I think it's just the fact that when I invoke toolstep.run, I use a question mark here. I'm yet to see if this behavior is needed.

02:14:10 I think from the agent kit, it's not. I will make sure to research the documentation a bit more, but I do not think this is needed. I think it's fine like this. And all the other comments are referring to the same tool step just so it doesn't cause a failure, but all of those are labeled as minor. So we did a very, very good job.

02:14:37 Almost 3, 000 lines changed and 19 files changed. Some of them are, of course, generated files and package locks and package JSON, but overall, amazing, amazing job. Let's go ahead and merge this pull request and then let's go ahead and let's go back here git checkout main and git pull origin main. I almost forgot how to write the command. There we go.

02:15:09 Now we are officially up to date on our main branch here. So If I go ahead now in here in graph, you can see that I have detached for 13 AI agent and tools and then I have merged that pull request back inside. So yes, as I said, I have these two commits here, which I just used to update the readme because I finished part one of this tutorial. You don't have these two commits. That's perfectly fine.

02:15:39 Your last commit should be 12. Conversation system and we now finished 13. AI agent and tools. So that marks the end of this chapter. We implemented message cancellation flow.

02:15:53 We built conversation history dialogue. We configured AI agent with system prompt and we created a complete tool execution system. Amazing, amazing job and see you in the next chapter.