In the previous chapter, we've created an infrastructure which allows us to run long-running tasks within our project. In this chapter, we're going to put that to test by adding AI functions. In order to do that, we first have to choose an AI provider. Using the link on the screen you can visit AISDK. Go ahead and head into the providers section and in here you can find a list of all compatible providers.
You can see that it ranges from popular ones like OpenAI, Anthropic, Google, which is Gemini, DeepSeek, all the way to lesser known ones. And now I've extracted average pricing for those, simply so you know if you have the budget or not. Gemini, which is what I will be teaching you how to use in this chapter, is completely free, at least at the moment of me recording this video. OpenAI and Anthropic, which are by far the best and most popular models require a minimum of $5, which will be more than enough to complete this project five times. But in order to make this easy for everyone to follow, I will be using Gemini.
So let's go ahead and start with setting up AI SDK. So I'm gonna go ahead here and let me try and go back here and first things first, let's do npm install AI. So you can see that I'm using AI SDK version 5. So I will simply do npm install AI and then I'm going to head inside of my package Jason here just to show you the exact version. Here it is.
AI 5.0.60. Once we have that, let's go ahead and let's add a provider. So in order to add a provider, you have to go inside of providers here and you have to find the one you use. If you're using OpenAI, you would simply select OpenAI provider, you would select your package manager and you would run the command. So in this example we're going to be using, I'm never sure which one it is, okay, We're going to be using Google Generative AI, which is Gemini.
So let's go ahead and select NPM. Let's go ahead and copy this and let's paste it inside. NPM install at AI-SDK forward slash Google. And then let me show you that as well so you can see the exact version. So that's up here.
Here it is. AISDK forward slash Google 2.0.17. So that's the version that's compatible with my 5.0.60 version of AI. Now that we have that ready, we have to go ahead and get an API key. So for whatever provider you choose, you can very quickly just click on the hyperlink here and you will be redirected to their website and from there you can find the API keys.
So for Google Generated AI, let's go ahead and click here which will take us to ai.google.dev. I will have the link on the screen so it's easier to find here. One quick tip for you. The first time I tried accessing this website, I went into Explore Models in Google AI Studio and I was actually kicked out from here. And the reason was I didn't verify my age.
You need to be at least 18 years old to access Google AI Studio. So in case you are wondering why are you getting kicked out from Google AI Studio, it's probably because you have to verify your age. So just go to your Google account, go into settings and find a way to verify it. But if you can access this screen, most likely everything is completely fine. And right here at the bottom you can actually see get API key.
So I'm going to go ahead and click create API key button and I'm going to call this node base. And now you have to either create a project or choose a project. So I don't have any so I will click create project and I will call this node base project. Let's go ahead and wait a second for this to be created and then hopefully we will be able to select it right here. Here it is node base project node base create key and once the key has been created let's go ahead and click copy API key and you can see I'm on the free tier I don't have the credit card added hopefully that's the same for you and let's go inside of .environment here and let's go ahead and do Google Generative AI and let's do Google or maybe Gemini secret key and paste it inside.
And now to find the actual proper name we should be using for this. I mean, it doesn't matter. You can of course use whatever name you want, but I just want to show you that if you actually go to AI SDK here, you will be able to find the exact value that they are expecting and here it is. So API key which is a prop so you can always pass your custom name but this is the one they are expecting. Google Generative AI API key.
So I will use that instead. So Google Generative AI API key. Perfect. Now that we have that, let's go ahead and try and use it. So I'm going to go ahead inside of first just a normal TRPC router to keep it simple.
No need to mix it with Ingest for now. And I will simply add a new route here and let me go ahead and expand this a little bit. Let's call this test AI. It will be a protected procedure. It will be a mutation.
And inside of here, Let's go ahead and let's see what's the best way to do that. I want to do this. I want to do the generate text one. So I will mark this as an asynchronous method here and I am going to import generates text and Google from above. So let me just add Google from AI SDK Google and generate text from AI.
And then inside of here, I'm very simply going to get that text here. So let me just copy this and let me paste it here. Here it is text. And let me just return text back. So write a vegetarian lasagna recipe for four people.
You can change this, of course, to whatever you want. I will stick with the original query here. So we are using a model Gemini 2.5 Flash. Depending on when you're watching this video, perhaps they updated the documentation, but I think that it's all type safe. Yes, you can see exactly which ones they offer.
So let me just move it back to 2.5 flash. I'm not really familiar with Google models, I guess this is okay. Now that I have that in my router, I'm going to go inside of my source app folder, page.tsx, And I'm going to do const test AI use mutation drpc test AI dot mutation options. And let's go ahead and use this test AI and let's just do a new button here with test AI text here on click it will simply call test AI dot mutate and just to be aware of how long it takes let's also disable if test AI is pending like this. Now let's go ahead and make sure our app is running.
So you can do that using npm run dev but just a quick reminder in the previous chapter we have also added dev all But since we don't need ingest at the moment, npm run dev is fine. So let's go ahead to localhost 3000 here. And in order to test this, I think the best way is to actually open your network tab because I don't see how else we can see the results really. So let me go ahead and go inside of my network tab here. Get workflows is failing.
Maybe because I'm logged out. Let me go ahead and log out. And let me go. Yeah, since we removed, we removed the redirect. So make sure that you go to log in manually.
If you have your errors here, it's probably because you are logged out. So just make sure you log in again, maybe the cookie or the session timed out or something. Right. And let's click test AI. And here we have test AI.
Let me go ahead and you can see it takes a while, right? So something is happening now. And we're actually gonna get a much prettier view of what's happening once we connect this to ingest, to a background job. And you can see how useful background jobs are for AI queries. You can see how long this took and we had no idea if it went successfully or not until we actually look here and here it is.
Here's a delicious and hearty vegetarian lasagna recipe designed for four people focusing on fresh vegetables blah blah blah blah so it works we successfully connected Gemini to our project using a free API key but you can see the problem right when I click test AI it really does take a while right so this is a real world example of a long running task and this can very easily time out in production If you deploy this on Vercel or Netlify and you don't use their streaming option, this kind of request will timeout and your users will never actually get the result back. And what's worse, they will actually use up your tokens. So the perfect solution for this is to put them in a background job instead. So now we're going to do that. Let's go ahead.
And in case you're struggling with this like logged in logged out thing, you can also just make your life easier by going inside of routers app. Instead of using a protected procedure, just use a base procedure and import base procedure. And this way, even if you're logged out, so even if you go in incognito mode, you will be able to test the AI function. So now instead of running this here, we need to abstract it instead of something like this. So let's do that.
I'm gonna go ahead inside of source and instead of ingest and here are the functions. And now I'm going to delete everything in here like that. And I'm going to change this a little bit. So let's call this execute. And let's call this execute as well.
Or maybe let's just be a little more specific. So like execute dot AI and ID execute dash. I don't know. I just want to make sure that we can do this. I think this might be the proper one.
Maybe like that. Okay. And let's rename the Hello World here to execute. And once we have that, let's go inside of our TRPC API route. My apologies, not here, not TRPC, Ingest right here.
And let's replace hello world with execute function. Here it is, so execute, execute. You can close the DRPC route and functions here, perfect. And now, Instead of calling all of those weight elements, let's simply call the AI function. And now here we can import createGoogleGenerativeAI from AISDK.google and we can import generateText from AI.
Now let's go ahead and let's define Google, right? So const Google is going to be create Google generative AI. And in here, you don't actually have to pass anything simply because it will work out of the box like this because we are using Google Generative AI API key name. So if you named it something else in that case you would have to pass that here and then process.environment custom Gemini key, right? But since we are using this specific name and since it already worked through just a TRPC router I'm fairly confident that it will work like this as well.
And then in here there's a cool thing that Ingest offers us instead of running the actual AI dot generate text what we can do here is we can extract the steps from a weight step dot AI dot wrap And let's go ahead and call this Gemini generate text. As the second argument let's pass generate text and then as the third argument let's open an object. And then in here in this object, let's define the system to be you are a helpful assistant. And let's define the prompt from the user to be what is 2 plus 2. Like this.
And let's also add the actual model that we will be using, which is using Google. And inside of here, let's pass in the Gemini 2.5 flash. So let me collapse this so it's easier to look at and understand. So inside of our ai.wrap we have three props passed. First one is the name of our step.
Second one is the function that we are wrapping, which is generate text from our AI library. And third one are the properties of this generate text from AI. So essentially we're doing the same thing that we just did inside of this route.ts routers instead of underscore app here. You can see it's kind of the same thing, but it is optimized for ingest. So it kind of becomes a step.
It's abstracted in this way. And now that we do that, what we can actually do is we can just return the step here. And I think already we should see some results. Steps. Whoops.
And now let's just make sure that we actually fire this event. Execute forward slash AI. So again, inside of our underscore app folder let's go inside of test AI. And now here we're going to do the same thing. I'll wait ingest dot send.
And we are just going to send Execute forward slash AI. Let me just confirm so it's name of the event. Here it is. Execute AI and no need to pass any data really. And yeah, we can just copy this.
So again, we just queue the job. We're not going to make this long running task happen and let the user wait here with a disabled button not knowing how long it's going to take. We're going to give them an instant response. Now in order to run this, we also have to run our npx ingest-cli-latest-dev or You can use what we developed last time if it worked for you. Npm run dev all, which will run both Ingest and Next.js simultaneously.
So now I'm going to go to localhost 8288. In here I have my runs and I'm going to refresh this one more time and I will click test AI. Instantly I'm unblocked and in here you can see that we had a Gemini step and it was called, let me see the full name, Gemini generate text And the output here is, so the content is what's 2 plus 2. And if we go all the way down here somewhere, we should find the response. I'm going to try and find it here.
If we can't find it, I will very simply just change the code or maybe in the finalization step. Oh, here it is. 2 plus 2 equals 4. So it instantly responded. I thought it was going to write, here's your answer, blah, blah, blah, but it instantly responded that 2 plus 2 is 4 perfect so that's how we're going to run our AI jobs through background jobs So this way when someone just tries to execute an AI job they are going to get an instant message like hey all right that's in progress do whatever you want we're going to handle this for you right.
So let's go ahead and try this once again. And now I'm going to go inside of my functions here and let me just add a simple await sleep here. I'll wait stop that step that sleep. And let me see I'm going to call this. I don't know.
Pretend and then let's sleep for let me just see. Oh, five seconds. I think I can just do this. And let's go ahead and change our page.tsx here so that when we click on the test AI in here, in the mutation options let's just copy the unsuccess and let's just say AI job queued So you can now finally start to see how handy these background jobs are when we have a real world example. So let's refresh.
Let's click test AI. Instantly AI job is queued. And in here you can see we have a pretend step and then we are starting the AI job. So imagine in the pretend step, we kind of fetched something or we communicated to some third service, that's what's happening here, right? And you can now see how useful this is going to be when this test AI button becomes execute workflow button and then we are going to execute topologically all the nodes that are connected.
That node can for example be fetch a website or fetch an API and then use the result of that fetch and put it inside of an OpenAI request or a Gemini request and after that send it to Slack or Discord. You can see how long that would take within a normal network request. That would certainly time out on all providers. But if we offset that to a background job like Ingest, It is much more reliable, much easier to work with, and a much better experience for the user because look at this, instantly they get a response. All right, we got it.
You can do whatever you want. We are going to handle this entire workflow for you. Don't worry. That's what we are doing here. And we can do a bunch of side effects, we can do analytics, we can do whatever we want during these background jobs.
And not to mention, all of this can retry. If Gemini step fails, We're just gonna retry it. Right? That's how useful all of these things are. And now to finish the chapter.
Let's see. We selected Gemini. But I'm also gonna show you how to add OpenAI and Anthropic regardless if you have the API key or not. I would suggest that you follow along and simply let them error, right? You don't have to have them working.
We also need to demonstrate errors somehow. So just integrate them together with me regardless if you have the API key or not and you are just gonna have an error right for example I don't have an anthropic API key I just have open AI so my anthropic is going to fail and my open AI is going to work but it's useful for you to have the proper integration of all three of them because that will make it quite powerful to whoever you want to show this to. Right. And because these are very popular and very good models. So it would be a shame because it's super simple to create them.
Right. So the way we do this is we go back to AISDK. Right. And we click on OpenAI. We select NPM and we install.
So let me go ahead here. Let me install AISDK OpenAI. You scroll a bit down and in here you can find open AI API key environment variable which tells you the default that it's expecting so let's go ahead and do open AI here it is and let's do anthropic already And we're gonna do the same thing. So Anthropic, NPM, install. And let's find the variable, Anthropic API key.
Here it is. So now I'm going to show you how you can obtain OpenAI API Key simply because I have it here. So you have to go to platform.openai.com. You can see the link on the screen. Once you sign in here, in your settings here, make sure that in your billing here, you have some balance.
So as I told you, the minimum to add is $5. I've been using this balance for the past three or four projects that I've done. So you can see it's very cheap for what we are doing, except if you don't overuse it. And in here you have API keys. So you can see I already created one when I was developing this project.
So just click create new secret key, node base. If you don't have a project, just select the default project and click create secret key, copy it here and then just paste it here. And the reason I'm telling you that you should add OpenAI and the thropping regardless if you have API keys or not, after you finish this project, you're not going to use your API keys. Your users will add their own API keys, right? So we are just doing this for demonstration purposes.
For example, I'm going to demonstrate leaving Anthropic empty because I don't have that API key. But later when the project is finished, for each of these products, for each of these nodes, your users will have to create new credentials. And in those credentials, they are going to enter these fields right here. So they will have to take care of the costs of their AI providers. So don't worry about that.
And now let's go ahead and we don't have to create any new actions. We can just focus on this one AI job that we have here. So let me go ahead and do const open AI create open AI. I'm not sure which one it is. Let me just quickly check.
So It is create open AI from AI SDK, open AI. And oh, it's open AI like this. And let's do Anthropic, create Anthropic. And I'm 90% sure it's just Anthropic create Anthropic as simple as that. And now that we have all three here regardless if you have the API key or not.
Let's go ahead and call this alias this to be Gemini steps like that and then copy this. Paste it here, change this to be OpenAI steps and use OpenAI. Change this to be, I don't know, There are so many options. GPT-4, I guess. I don't know.
And let's copy it one more time. Change this to be Anthropic steps and change this to Anthropic. And oh, this, I don't know. Try to use some cheap ones. If that means anything to you, I have, I think Opus is very expensive, so I'm going to use Sonnet, maybe, I don't know.
And let me just change this to be anthropic generate text and this will be open AI generate text perfect and then here we will just return Gemini steps, OpenAI steps, and Anthropic steps. And did I... Like this. There we go. So now we should have no errors at all and if we try to go back instead of our app here refresh and let me close this and click test AI you can now see we have this running and let's see what happens.
So what I expect to happen is the first two to succeed. So Gemini succeeded. Now we are doing the second one. OpenAI succeeded and Anthropic has failed because as you can see the API key header is required and you can see what's actually happening here. It's retrying, Right?
So that's what I was trying to explain to you. Imagine this wasn't an API key issue. Imagine this was a rate limiting issue or a connectivity issue. If this was a normal network request, the user would have to do it all over again regardless if they already spent the tokens for these two. But thanks to ingest we can still save this by attempting to run Anthropic again.
But since this is an obvious issue you can see it will just continue to fail in the second attempt. And the third attempt will take even longer to try. You can see they purposely added delay before trying again. They don't want to get rate limited, right? So yes, as expected, OpenAI is, I mean, Anthropic is failing because we didn't add the API key here.
So let's just click cancel for now so it doesn't run. And yes, this one is failed. For you, it might stop after OpenAI, right? But as I said, implement OpenAI Anthropic with me simply because when you finish this project, it will be your users who will add API keys, not you. But we do need our API keys to actually test if this is working, of course.
Or to just test if it will fail. But the cool thing about using generate text from AI is you can see that it's the same everywhere. The only thing we modified is the model that we are using. So you can be pretty confident that if your free one is working, the Gemini one, all the other ones will work as well because this is the same API, right? I would expect this to work.
I wouldn't even test it. I'm 99% certain that the Anthropic one works without even trying the API key. So that's what I tried to show you. That's what I was telling you about a background jobs. And I hope that this kind of explained it even further, why they are so useful, because they can really be long running tasks.
And it's such a good experience for your users to instantly get a success message like hey, okay, we are taking care of that, go ahead and relax, do whatever you want. And that will be extremely useful, if not fundamental to this project, which is workflows, right? We are based on these background jobs. So I think that now we have a very good understanding of background jobs and we even tested them with another fundamental of this project which is AI, right? So we set up AI SDK and we use the SDK within just now let's open a pull request here so I'm going to click here create new branch 0 7 AI providers I'm going to go instead of source control here I will expand my screen a bit so you can see with me.
I will close the graph here. Not too many unsaved files. I mean unstaged files. So I will stage all of them. And I will add a commit 07 AI providers.
Let's click commit and let's publish the branch and once the branch has been published we can go ahead and simply start a pull request and let's quickly review our changes and go to the next chapter. And here we have the summary by CodeRabbit. We added a test AI button to trigger an AI job and it shows the AI job queued toast on success. We introduced a new public API mutation to queue an AI execution because we added base procedure there. And the back end now processes AI tasks using multiple providers and returns aggregated results.
This is referring to the ingest function. And of course, we added AI SDK dependencies to support this new AI job execution. As always, a file by file and a cohort walk through here. But what I'm interested in is of course the sequence diagram. So let's just quickly take a look even though it's fairly simple and we've already established this a few times.
The user clicks on test AI. We call drpctestai.mutate. We send the event called execute ai to ingest and we immediately return back to the user success job has been queued. This is exactly what we were talking about I believe in the previous chapter right we achieved this again this time with a real world example and not just some sleep steps and then what happens in the execute function is just calling various SDKs three different external services all of them could fail all of them could take a while, and that's why it's perfect for a background job. So all of them can then run and then return back the results.
But we're going to improve that even further by adding real time to it. So then users will see exactly which node is executing, which one succeeded and which one has failed. And as per the code comments, It here, it told us to use a fully qualified Anthropic clause on it. I'm not familiar with the models here, so I don't know. It says Gemini Flash is okay and GPT-4 is correct, but for this one that we should use versioned ID.
I don't know. Sure, we can change it to that. I'm not too familiar with the models. And in here, of course, it noticed that we changed the base procedure and that we should add authentication there. So you can see how familiar it is with our protected procedure, right?
It's really like a partner to check all of your code. Obviously we did this on purpose because we didn't really have a proper outflow set up, but yeah, we are going to remove the test AI route anyway. Amazing job, very good pull request here as always, very useful review by CodeRabbit and once you merge that make sure that you go back here, select main and simply synchronize your changes. This way you are up to date. And then instead of your source control here, just make sure to look at the graph and just confirm that you have number seven here and that you've merged it in here.
This way you know that you're up to date. Perfect. I believe that marks the end of this chapter. Let me just double check. Yes.
Amazing, amazing job and see you in the next one.