Now let's go ahead and let's add a third way to add images to our canvas. So besides uploading and picking an image from the Unsplash API, we're going to enable the AI generation. So I want to start with the UI part first. Let's go ahead and let's go inside of our features editor and let's go ahead and slightly copy one of the existing side bars. For example, we can copy the fill color sidebar.
Copy and paste this one and rename this to AI sidebar. Go ahead inside of the new component, close everything else and rename all instances of fill color sidebar to AI sidebar. Now go ahead and remove the value, we are not going to need this. And you can also remove the onChange here. Go ahead and change the active tool to look for AI value.
You can now go ahead and remove the color picker from here. Go ahead and remove the input for the color picker as well and you can remove the constant fill underscore color from the first import that we have. Now let's change the tool sidebar header. The title inside of here will be AI and the description for this can be generate an image using AI. There we go.
And now let's go ahead and let's add this to our editor components editor. So I'm gonna go ahead and do the thing that they usually do which is simply copy the last one so this will be AI sidebar from AI dash sidebar. There we go. I'm going to copy this one and I will just duplicate what I've written last and change the name of this one to AI sidebar. And now go inside of your sidebar where you have all the sidebar items and just confirm that your sidebar item which has the sparkles icon so this one right here with the label AI is actually working with the AI tool and you can always confirm inside of your types where we have defined the tools right here that you have the AI option.
There we go. So my sidebar is already opened. So I'm just going to refresh everything to confirm. When I click on AI, there we go. The AI sidebar opens.
Now let's go ahead inside of our terminal and let's add a package we are going to need. It's actually not a package it's a component from chat-cn. So bunx-chat-cn-ui-latest-add-textarea. For now that's the only thing we are going to need. Let's go ahead and do bun run dev and refresh your localhost.
Then go inside of the AI sidebar and let's go ahead and let's import our new component. Text area from components UI text area and I also want to prepare button from components UI button. Great! Now inside of the scroll area div right here let's go ahead and let's add a form element. Inside of the form element we're gonna add a self-closing text area and below that we're going to add a button with the text generate.
There we go. And we can actually rename this div into a form instead and then remove the inner form. There we go. So this can serve both as a form and also as our style. So now if I go ahead and if I click on AI, there we go.
I have a text box and I have a generate button. Now let's go ahead and style this a bit further. I'm going to go ahead and give the text area some properties starting with the placeholder. Let's go ahead and let's write something that for now we can just write literally something, right? It doesn't matter.
And let's go ahead and write cols 30 and rows 10. Let's also make this required with a minimum length of 3. So we don't want to waste our resources if literally nothing descriptive can be written in this area. And for the button itself I want to give it a type of submit explicitly so we know that this button will be used to submit this entire form. I want to give it a type of submit explicitly so we know that this button will be used to submit this entire form.
I want to give it a class name of width-full. There we go. So this is our text area which will be used to input something and then once we click generate it will submit the form. So before we can continue working on this we actually have to set up our API endpoint, which we are going to call, which will be a mutation to the React query. But before we can do that, we have to create a replicate SDK, right?
So let's go ahead and do the following. Inside of your terminal, go ahead and run bun add replicate or npm install replicate. Go ahead and run bun run dev again and let's just refresh the localhost. I'm gonna go ahead and close everything I have here and I'm gonna go inside of source lib and I'm gonna create a new file replicate.ts. We can now import replicate from replicate itself and let's export const replicate new replicate and inside of here for auth property define process.environment.replicate underscore API underscore token.
We don't have this yet so we're gonna add it in a moment. Now let's go ahead and let's create our replicate account. So Google replicate and click on the first link or simply go to replicate.com. After you have created your account you're probably not gonna see these options. I already have my account so I already have some predictions and stuff like that.
What I want you to do is I want you to go into the upper left corner and click on API tokens here. Go ahead and create a new token name. So I'm going to call this image AI. Go ahead and click create token and there we go. You can click copy token from here.
Now I'm not sure whether Replicate offers a free tier but I just want to give you a note that I've set up my billing here, right? So make sure that you check your account settings and check the billing. So in case you're having any errors in the terminal, it might be because you need to activate your billing. But here is a breakdown of, you know, my spending. So you can see that a year ago I spent $45 on I don't know what honestly, but here it is.
This is my current period for this tutorial and I barely spent 12 cents. So I do think you can actually go through this with a free tier, but I just want to give you a quick note in case you're having any errors it might be because you did not activate the billing. So all I added was add my credit card. Now let's go ahead and create the replicate underscore API token variable. So inside of dot environment of local I'm gonna add replicate underscore API underscore token and I'm gonna paste what I've copied.
Obviously do not share this with anyone. As always I'm gonna remove this token the moment I finish uploading this video. Great, so we now have the util to create AI images and everything else AI related that we want. So Now what I want to do is the following. I want to go ahead and create the API endpoint which we are going to call to generate AI images.
So go ahead inside of the catchall route and create a new file ai.ds. Go ahead and import hono from hono. Define your app to be new HONO and export default app. Then let's go ahead inside of the route catch all route file. Go ahead and import AI from .AI and go ahead and chain a new route with the prefix AI to our new AI group.
Now let's go back inside of AI here and let's actually chain our first, not get, but our post method here. So the first argument will be the name of our route. This will be generate-image or you can just use generate, however you want. And before we move on further, I want to go back to the route here and let's not forget that we have to enable our POST request. So POST will do handleApp.
And since we are all already here, I want to go ahead and enable it for everything else we are going to need. So export const patch handle app and export const delete handle app. Those are the HTTP requests which I'm going to use throughout this app. So just make sure you have all of these enabled. Great.
Now we can head back inside of our AI post request. Now let's go ahead and let's add a little comment here, add verification. Then let's go ahead and let's add our z-validator here and we have to import z-validator from HonoZodValidator. There we go. We're going to validate specific fields here for JSON and now we have to import Z from Zod so just make sure you've added this.
So we're going to validate by defining what the JSON can look like. So the JSON will be an object which simply accepts the prompt. The prompt will be what the user will write in the text box and it's going to be a type of string. And let's actually remove this comment, I just think it looks confusing. But yes, we are going to add the verification middleware here later when we have it.
Great and after we've added the validator let's add our controller here so it's going to be an asynchronous method which holds the context and then inside of here let's go ahead and let's extract the prompt by using c.request.valid.json so we know this prompt here will be a string because we are using z-validator to validate our JSON body. Now let's go ahead and let's find a way to actually run a replicate model. So I want to go back to the replicate website here. Let's go ahead and let's search for stable diffusion. If we can do that from here.
There we go stability AI stable diffusion. You can of course find any AI model that you prefer. So for example, let's copy this prompt right here. An astronaut riding a horse on Mars. HD, dramatic and lightning.
And let's go ahead and instead put that inside of our AI sidebar as the placeholder for our text area so the user knows that they have to write something descriptive inside And now let's go ahead and let's find a way to actually run this. So you can go ahead and I believe click on Node.js here. And oh, looks like they have an NPX tool now. I didn't notice, but it's okay. We did this ourselves, right?
We added the replicate and then we added the replicate API token and we have this util here and this is what we have to run it now so that's where you can copy from here or any other AI model that you want so go inside of the playground node.js right here and skip to the last part right here where you actually have the hash of the current model. And now let's go ahead inside of our AIRoute group here and now that we have the prompt we can go ahead and add this constant here. We now have to import replicate from add slash lib slash replicate. And now instead of the prompt being hard-coded this, let's go ahead and actually enter our prompt constant. There we go.
Of course if you want to you can go ahead and specify some other width or height, number of outputs, guide and scale, all of these things. You can play around until you find the one that you like. So now the problem is I found replicate to not have very precise types right but the thing is we know what this will look like but in order to type it properly we first have to mark the output as unknown, like this. And then what we have to do once we have the output is the following, const response to be output as array of strings, like this. And then we can return c.json data and simply the first item in the array.
There we go. So now that we have the API endpoint to generate an image using a prompt and a stable diffusion replicate model. Let's go ahead and let's actually create the mutation for this. So we can go ahead and go inside of source features and let's go ahead and create a new folder called AI inside of here. And let's create another API folder and then let's create useGenerateImage.ts.
I'm going to go ahead and import useMutation from tanstack react query. And I'm going to import inferRequestType from HONO and inferResponseType from Hono. I will also import the client RPC from add-slash-lib-hono. First let's define the response type. The response type will be infer response type from typeof client API AI generate image and all of this is in one line right so ignore that this will go into the lower line right here It's supposed to go all in one line like this.
Then we're going to target the post request like this and then you can copy and paste this, change the lower one to be the request type and instead of using infer response type this time use the infer request type and what the request type will be is from this post specifically JSON. There we go. So we now know exactly what the mutation will return and we also know what the mutation expects, the prompt string. Now let's go ahead and actually add this to the useMutation. So export const useGenerateImage will be a mutation.
Oops. From useMutation. Before we run this, we have to define the types inside of here. So let's go ahead and do this first. Make sure you are writing this inside of these two pointy brackets here.
The first one will be the response type, the second one whoops, the second one is the native error and then request type. There we go and make sure you don't have any trailing commas here. Go ahead and open the options like this and write the mutation function. This will be an asynchronous method which accepts, which passes the JSON, right? So this will be prompt string.
And then we're going to generate a response by simply awaiting our RPC client API AI generate image dot post and simply go ahead and pass in JSON inside of this object and then return await response JSON. There we go. Let's not forget to also return this mutation here. Perfect. And now let's go ahead and let's go back inside of our AI sidebar right here and let's actually add the mutation.
So here at the top above the enclose I'm going to define the mutation constant to be use generate image. Make sure you've added the import from features AI API use generate image. Once you have that, let's go ahead and let's also create const on submit here to be an event which is a type of react.formEvent HTML form element. First of all, we're going to prevent default, so the page doesn't refresh when we submit the form. And then I'm going to add to do block with paywall.
We currently don't have paywall implemented so we cannot do this feature. Otherwise, let's go ahead and call the mutation.mutateAsynchronous and pass in the prompt to be the value. We currently don't have the value so we can just go ahead and pass an empty string here. Let's go ahead and chain.then data and we can simply call the editor ?addimage and pass in the data. Data will be the direct URL to the newly generated image from Replicate.
There are two ways you can call what happens after the mutation. You can use mutate asynchronous and you can also do mutation mutate, pass in the prompt, and then you can add options like this for example on success oops and then in here you can get the data as well is it data Let me just go ahead and check. So on success, yes, I think it should be data like this. There we go. And you can do editor, add image, data.
I prefer it this way, actually. There we go. Now let's go ahead and let's simply create a set state for the value. So const value, set value, use state from React with the empty string as the default and make sure you've added the import for useState. And let's go ahead and let's write the onChange method here.
So for the text area, what we're gonna do is we're gonna add onChange to get the event and simply pass set value event target value. Now let's go ahead and give this form an on submit of on submit like this. Let's give this text area a disabled of mutation is pending and let's also give it a value of value so this is a controlled component and let's also disable this button if mutation is pending so we don't spam the request. Let's go ahead and see whether this is now working or not. If we've done everything correctly, it should be ready.
So I'm gonna go ahead and refresh everything, make sure you do that as well. And I'm gonna keep my network tab open because I'm interested in my AI endpoint. So that's what I'm going to search here, AI. I'm going to go ahead inside of here and how about I copy the prompt from here. So I'm just going to copy the ostrich not riding a horse on Mars so I can be specific.
I'm going to paste that here and click generate. There we go. It's now generating an image and the first time can definitely take some time, right? And as you can see, there we go. You can see how it generated the image.
Now you might be getting an error here instead of an image like I did and let's go ahead and debug why this is happening. Also this is definitely not an Austronaut riding a horse on Mars so we're gonna debug why this happened as well. First of all, let's go ahead and do the following. I wanna go inside of my next config here and I want to add the following to fix any errors with the images. So you have to add replicate.delivery so that your image will not break when you add images from replicate.
There we go. And now what we can do to try and tweak our AI model to be slightly better is you can go ahead and remove everything besides the prompt for example and scheduler. There we go. And I think You can also play around with different models until you find the one that works the best. Let's go ahead and try and generate this again and see if we're gonna get any better result.
So I'm gonna refresh everything here. Let's go ahead and just wait a couple of seconds. Looks like it's getting slow for some reason. Perhaps something that can help here is just shutting this down and doing a BUN Rundev or NPM Rundev again. And let's refresh this one more time.
So let's go ahead and let's try some other example from the Stable diffusion here. And perhaps I copied the wrong one, right? Let's go ahead and try and search from the homepage here. Let's click on explore here. Perhaps I copied the wrong version of stable diffusion.
There we go. So we have the stable diffusion three here. Maybe that will be better. Let's go ahead and try again. So I'm going to go ahead and click on Node.js here and I'm going to go ahead and copy the input here.
We can actually copy the entire thing again. So basically I just found new stable diffusion because I feel like the current one is just not working as intended. So I'm gonna remove the const output from here. Let's paste it again and I think that now there we go so this is also much clearer name So perhaps you can also copy it from here. And we're gonna go ahead and directly add the prompt here.
And let's also do one more thing. Let's also console.log the prompt. I want to make sure that we are actually passing the correct prompt to our backend. Perhaps that's the issue. And I'm going to copy the exact thing that they have here, so this graffiti.
And see if we are able to reproduce this. So I'm gonna open my terminal here so I can see the console log. Let's go ahead and click on AI here and I'm going to click generate. Oh, looks like my prompt is empty. So that's what the issue was.
So now it generated a completely random image. Okay, that's what the issue is. Let's go ahead and see why this is happening here. So I'm gonna go ahead and set up the use generate image here. We have the JSON here, we have the prompt.
Let's go inside of the AI sidebar. So we have the set value here and we are not sending the value. Let's send the value. All right, one more try. So it looks like we could have used the older version as well, but you know, why not?
Let's go ahead and let's try with the newer version here. So I'm gonna go ahead and copy this or perhaps we have some more examples. There we go. We have a wizard here. So I'm gonna go ahead and copy a 3D render of a wizard.
Let's see if this will improve it in any way. So I'm gonna click generate and let's hope for the best. There we go, a 3D render of a wizard. Perfect, so I actually want to stay on this table, Diffusion version 3, I think it looks great. Amazing, You've implemented AI features inside of your editor.
What we're going to do next is we're going to enable another AI feature which will be used to remove the background from any image. Great, great job!