In this chapter, we're going to go ahead and wrap up our form section by implementing AI thumbnails. Basically, this little button left in our thumbnail drop-down. So the goal is to create a thumbnail prompt model so that we can go ahead and enter what we want for AI to generate because it doesn't really make sense to use the transcription or image generation as that really doesn't work well in my experience. We also have to create the proper generation workflow and wrap up form section by adding the skeleton to its loading state. And just some update about my last run.
Looks like it ran for five minutes. I'm not sure why it took that long, but the good news is it works. It definitely worked. It took its time, but it absolutely did what it was supposed to do. And that's one cool thing about background jobs is that they are very reliable, right?
So even if they don't seem to be working right away, you can know for sure that they will at least retry if they fail and you will of course have all the logs and analytics you need to discover why some things happened or didn't happen and here's the new description. So let's go ahead and create the thumbnail upload, the thumbnail generate model. We're going to go ahead inside of our modules, studio, UI components, and let's copy the thumbnail upload model. Let's rename it to thumbnail generate model. Let's go ahead and change this to generate.
We are gonna have the video ID open and on the open change all of that is fine and we're also gonna have form schema here using Zod so import Zod and we're just gonna add prompt And let's go ahead and make it have a minimum of 10 characters so that it's at least something descriptive to tell the AI to do, right? I'm just gonna move this to the top and what I'm gonna do now is I'm gonna go ahead and change this to not use an upload drop zone. Instead, we're gonna go ahead and open our form from components UI form. So while we are here, let's go ahead and everything else we need from our form here. So that's form control, form field, form item, form label and form message.
We can remove the upload drop zone from here. And let's also add text area. That's going to be our component. And we're also going to need use form and Zod resolver. And let's wrap up our imports by also adding host from Sonar.
One more component is the button so we can submit the things. There we go. So we have the form schema and we have prepared the form here. It looks like it says, cannot find form. Oh, I forgot that I removed the form.
Okay. Now let's go ahead and let's create the form constant here. So use form, z.infer type of form schema, resolver, Zod resolver, form schema, and default values, prompt, empty string. Like this. There we go.
So now instead of here, we can go ahead and spread the form. And inside of here, we can open up a native HTML form element and we can pass in on submit to be form handle submit, pass in on submit. And we can turn this into an onSubmit method here. So we can leave it like this for now. We are going to modify what's inside in a moment.
And let's also give this native html form a class name of flex flex column and gap four now let's add the form field which is a single self-closing tag here let's give it a form control Let's give it a name of prompt, which is the only possible option. And let's go ahead and give it a render. Inside of here, a form item, a form label saying prompt, a form control and text area. Let's give this everything necessary from the field. Let's give it the class name of resize none, columns 30, rows 5, And let's go ahead and give it a placeholder.
So for example, we can write anything we want here, you know, a descriptive, a description of wanted thumbnail, Like this. And let's go ahead and pass in form message here as well. And still inside of the form element here, add a div with a class name, flex justify end, and render the button which will say generate and give it a type of submit. So before we move forward, let's go ahead and render the thumbnail generate model component inside of form section here. So I'm going to render it right next to our thumbnail upload model, so thumbnail generate model, and let's import this from ./.components thumbnail generate model.
Let's copy this state and this will be thumbnail generate model open and thumbnail generate model open. Set thumbnail generate model open. Great. So now in here, where I render it, I'm going to modify the thumbnail generate model to use thumbnail generate model open and set thumbnail generate model open. So just be careful, make sure you don't mix and match invalid states here, right?
Video ID can stay the same. And now we have to reuse the set thumbnail generate model inside of our sparkles icon AI dash generated drop down menu here. So instead of this, what's going to happen, it's going to set this to true. There we go. Let's see what kind of error I have now.
So I have generate thumbnail here. We can remove this. We are not going to need it here but we will need it inside of the thumbnail generate model. So you can add it here if you want to generate thumbnail and let's see do I have TRPC? We have it from the client.
So add it here. Generate thumbnail TRPC videos, generate thumbnail. There we go. And on submit we are going to call the generate thumbnail that's fine. Let me just remove...
Oh did I... Okay it should be here and not in the form section yes okay now I believe if I go ahead and click AI generated it will open upload a thumbnail perfect So let's go ahead and just finish up our on submit method to accept the values of z.infer type of form schema. And what we're going to do here is simply call generate thumbnail, mutate, and pass in the prompt to be values prompt and ID video ID. Now we are getting some errors here because we have to modify our generateThumbnail procedure And what we're also going to do, well, we don't have to do anything. We don't need to use the utils at all, but we just have to fix this prompt thing.
So let's go ahead and go inside of the TRPC videos generate thumbnail protected procedure. So whenever you find this, you can press command or control and click here and it should lead you to that exact procedure it's located inside of videos servers procedures so besides ID we are also going to accept a prompt which will be a Z string with a minimum of 10. So it's the same thing that we set here. Great, we now have this. So besides user ID and video ID, let's also pass the prompt from input.prompt.
I'm not sure why it didn't, when I wrote it like this, it looked like prompt exists. It looks like prompt is a reserved function. So that's why there was no error. So be careful, yeah, you have to restructure the prompt from the input. I did not know this, so maybe prompt isn't the best name to use, but make sure that you pass input.prompt.
So when you hover over here, it should be string and not that function. Well, yes, prompt is a pretty popular method, but I have not used it in ages. Great, so we now have generate thumbnail and let's change the workflow to go to thumbnail because that's the workflow we care about. And now it's up to us to create that. So let's go inside of source, app folder, API, videos, workflows, and let's copy one.
For example, let's copy title and paste it here and rename this the thumbnail. Let's go inside of the thumbnail one and now it's time to build this route. So first things first, besides having this, we're also going to have a prompt. So let's add that. We can remove this default prompt because the user will type their own.
And here we can also destructure the prompt now. And let's start as we usually start, which is by getting the video. So in case we can't get the video we're gonna stop here, we're not gonna run any other steps, right? Now we're not gonna need any transcript. At this point we can already generate but we're gonna generate a little bit differently.
So OpenAI SDK from Appstache, at least in my version, does not have it for anything other than chat completions create. So what we're gonna do is the following. Let's go ahead and delete this And let's go ahead and do context.call, generate thumbnail, sorry for the pop-up, generate thumbnail like this, and then pass in the second argument. And in here, we can go ahead and pass in the API OpenAI.com version one images generation. So I cannot exactly guarantee you that in the future, if you're watching this a year or two from now, this endpoint still works and still uses the same API.
So if you want to, you can go to OpenAI documentation. Let me just load the platform here. And if I click on image generation, images API, there we go. API, openai.com, We want images generation. So find in the open AI platform images, and in here, you should be able to see if this still exists or not.
So you can copy it from here and also add it right here. There we go. And then you can follow the prompt, the model, right? Let's go ahead and pass in the body from here. So let's assign the method to be post.
Let's assign the body to have our prompt like this. And then you can go ahead and pass in some other things. So N will be one. I'm not sure what this actually means. The number of images to generate.
Okay, so we are going to default to one. And now for the size, let's go ahead and see. So it looks like we can modify the size. In my original source code, I was using this one, but This is not the greatest resolution to use for thumbnails. So perhaps if we change the model, let's see.
It looks like if we use Dell E2, it's only square images. But if we use Dell E3, we can use this type. So let's try that. How about we give this a model model and now let's give it a size. We're going to try both of this.
So how about we try this one first, like this. And now we also have the passing the headers here, authorization, make sure you don't misspell this because this is not strictly typed. Pass in the bearer and then process environment and you have to add from your .environment open AI API key. Like this. There we go.
Perfect. So after this, if you go ahead and go, you will have the body here and what you can do is you can get the temporary image, temporary thumbnail URL like this and you can extract it from body data.url. At least this is how it works here. So body data, and then first in the array.url, right? So I'm not sure how we can set, you know, can I go ahead and give this a type of body, which is going to be this URL string and then an array?
Will that work? Let's see. Did I do this correctly? Body. Okay.
So maybe just this, maybe just URL inside. My apologies. How about we do like this? Body data doesn't exist on type. Let me try and strictly type this.
Oh yeah, my apologies. So we just need data and data needs to be an array of this. There we go. So if you want to, you can add your own types for what the return is going to be and if you don't like this return you this array you can also do this yeah that's the same thing great So we now have this temporary thumbnail URL. In case we don't have it, we can immediately throw this as a bad request.
And now what we have to do is upload it to upload thing. Right? So we are going to do the same thing that we did in our, is it a webhook? In here, when we do asset ready, we upload from the temporary thumbnail URL. So we're gonna do the same thing here.
The reason we did it in here is so we don't have to rely on mux changing their API in the future and then all of our images get lost, right? And to own our images. But the reason we are doing it here is because this URL which OpenAI generates will only last for some time. It won't last forever. They are not a storage company, right?
So we shouldn't expect them to store these URLs. You should always treat them as temporary thumbnail URLs. So now let's go ahead and do the following. We're now going to create uploaded thumbnail URL and let's do await context run upload image, upload thumbnail. Let's do that.
Asynchronous like this and let's define UTAPI to be new UTAPI from upload thing slash server like this and then let's go ahead and destructure data from await UT API dot upload files from URL and passing the temporary thumbnail URL and just return back data. There we go and in case we have an error we can just throw new error. Can I pass that here? I can't. Error.message.
Maybe we can do that. There we go. Or simply, if we don't have data, we can go ahead and say bad request. There we go. So now this will be the uploaded file data.
So maybe we should just call it uploaded thumbnail. And then what we have to do is, let's see. So in here we confirm the existing video. So we have that here. What we are supposed to do now is we are supposed to do the cleanup.
So let's do the cleanup now. Await context run, and this will be delete, or let's call it thumbnail cleanup, or cleanup Thumbnail, so we stay consistent with our names here. Asynchronous. What we're going to do here is define another UT API. Perhaps we can just move this here in the beginning because we're going to use it throughout so we don't have to define it many times so what we're going to do is we're going to check if video has thumbnail key in that case await utapi delete files by video thumbnail key.
That's what we're gonna do. And then await database, update videos, set thumbnail key to null and thumbnail URL to null, like this. Where, and, and passing our two equals here, we can just copy this. There we go. And after that, this should be fine.
And one thing I actually wanna do, I wanna move this step before we upload the thumbnail. That's what I wanna do. So I first want to clean up our old ones, just in case this never succeeds, and then we're left, or this succeeds, we upload. Basically, I'm failing to explain what I mean. I first want to do the cleanup.
Before I upload anything new, I want to delete the old. That's it, that's what I wanted to say. So only then are we gonna go ahead and upload new things here. And finally, update video, this can stay the same. And this time we're changing the thumbnail key to be uploaded thumbnail.key and thumbnail URL, uploaded thumbnail.url.
There we go! That's the workflow. Perfect! So just confirm that it's called thumbnail and confirm inside of your procedures here in the video that you go to workflows slash thumbnail. So this will be quite interesting.
I'm not sure which is the better resolution, this one or this one. I never know what's the height and what's the width of these things. Let's try one, right? So I'm gonna go ahead and just try a very, very simple one. An image of Swiss Alps.
Let's try that. Background job started. This may take some time. So we also need to like close this thumbnail. Let's work on that while we wait for our thing to happen.
So thumbnail generates model here. We're gonna go ahead and do the following. Let's do form reset here and let's do an open change pulse. There we go. That's going to happen in the future.
And let me just see. There we go. So run has started. It got the video and now it's generating a thumbnail. I'm very interested in this because this is not like my source code.
I'm using a different model here, so I'm not sure if this will work. If it doesn't, I will just show you exactly what I have in my source code. So we're going to see what's going on here. I'm now going to pause and wait for this to finish. So it seems to be working.
Let's go ahead and see what it created. I will refresh this And let's see if we have a thumbnail or not. Looks like we have a new thumbnail and it's so cool. Amazing. Looked like that was the correct resolution.
Perfect. So we can now use AI to generate thumbnails as well. And here's what I'm interested in. It's my upload thing dashboard that I'm interested in. Is it getting properly cleaned up?
So I'm gonna go ahead and go inside of files here. And it looks like I have some things left here, but I'm not sure if this were... This might have been left because of the 24-hour deletion thing. Because remember, I did delete some videos directly from my database, which doesn't trigger the webhook, and then things get left inside. So this is how I'm going to test this one more time.
I should have done this in the beginning. What I'm gonna do is I'm gonna remove all of my videos from my database and I need to have my Drizzle Kids Studio running here. So let me just refresh this. I want to test it out because I want to make sure we are not doing any mistakes here. So I'm going to remove all of these videos here.
I'm going to go inside of Upload Thing, and I will remove all of my content here. I'm going to go inside of upload thing and I will remove all of my content here. I'm going to go inside of mux and I will also remove all assets here simply so everything that I have is synchronized And this one as well. And now let's go ahead and try this again. So into my content which is now completely empty.
I'm gonna go ahead and I will simply upload a video. What should happen now is that in a few moments, an image should be generated and uploaded here. There we go, animated.gif and thumbnail.jpg. And if I now refresh this, I should see them here as well. And now let's go ahead and let's try to generate another one.
So an image of aquarium. Maybe we can try that. And let's click generate. And one thing that we forgot to do in the thumbnail generate model is disable the button while this is going on so disabled it will be generate thumbnail dot is pending like this there we go And now if we've done this correctly, I should have another run started here. And after generate thumbnail, we have the cleanup.
Or the cleanup already happened? No. So it's working on generate thumbnail now. Yeah, it's first generating it and then it's cleaning up. Okay, so it's working on that now.
What I expect to see is that the thumbnail.jpg gets deleted. That's what I expect to see and get replaced with my new one. So let me refresh again. Looks like it takes around 34 seconds. So this is a pretty good example of a long running tasks, right?
To generate a thumbnail, it seems to take 34 seconds. I'm pretty sure this is outside any timeout limits on some of the more popular hosting providers. So this function would fail on a majority of them, But thanks to our background jobs, you won't have to worry about that. Let's see if we've done this correctly or not. There we go.
Thumbnail has been replaced. And now we have this newly created AI aquarium image right here. Amazing. So in order to wrap up this chapter and in order to wrap up form section in general, let's go ahead and let's build a skeleton component so that when we load here, it doesn't have this, you know, loading dots here. So I'm gonna go ahead and close everything here and go inside of form section.
And we're gonna go ahead and build the proper skeleton here. So just go ahead and import the skeleton. Of course, if this doesn't matter to you, you don't have to do this. You know, I wanna give you a proper project. I wanna give you the exact thing I've built, but if you think that you wanna focus on some other things, you can just speed this part up or just copy it from the source code.
So what I'm doing now basically is I'm just looking at what I did, how I built the original one, right? And then I'm just kind of mocking this elements with skeletons. Here's a hint, AI is extremely useful when it comes to creating these things. So feel free to ask AI to use it. So I'm adding a space too, and this is like the title and the description.
And then I'm just going to add one big skeleton below that. So let's see how this looks for now. There we go. Not below that, next to it. So I'm just representing the button, no need for both the button and the little dropdown.
Now, what I'm going to do is I'm just gonna copy like my grid structure here. Grid, grid cols 1 and enlarge grid cols 5, gap 6. Then I'm gonna open my first column here. Then I'm going to close it and then I'm just gonna go ahead and add the first element of the form. So this would represent the title and its input.
And then below that, I'm going to represent... Oops, let me just add it here. I'm gonna represent the description and its text area. There we go. And you can choose how granular you wanna go with here.
So now I added a new one and in here I'm representing like the thumbnail and in the last one here I'm representing the category selection tool. There we go. So that part is finished. So if you want you can leave it like this or you can go ahead and create another side. So I'm going to go ahead and do that here.
And I'm going to go ahead and open like this gray box that we have, which renders our video, right? So now when I refresh, the box should appear here. I hope I did everything correct here. I think it's just not visible because there's nothing inside. So now I'm just going to add a skeleton with an aspect of video and this will represent like this video here.
There we go. That works fine. And then below that, I'm going to go ahead and open a div where I'm going to put all of this statuses, right? So this part will represent the video link like this. Then below that, we're going to do the video status.
And below that, we can do the track status, which are exactly the same. And all that's left is the visibility component. So we can do that one, two, three here, like this. There we go. A very, very detailed skeleton.
You don't have to build them like this, of course, but yeah, try out AI. It can help you with this kind of things. It can speed up this process very, very fast. Great. So you've learned all about background jobs now, and we've used Upstash's workflow, which is their new product, which I really, really like.
Everything Upstash builds is always a very good developer experience. We know that already from their rate limiting feature, from their Redis feature, And I'm sure this will only continue to go in that direction. I advise you to also, you know, explore more about background jobs. I think they're a very interesting solution. And I also advise you to Maybe try for a challenge and change something.
For example, inside of my, what's the name? Webhook. Inside of my videos, webhook, right? Maybe change this, this entire thing where I upload and extract the new thumbnail key and the thumbnail URL, maybe move all of this to a new workflow and leave it to be a background job, right? Inside of here, you would simply do await workflow.trigger, and then just pass in the URL to, of course, our process environment, ngrok, blah, blah, blah, slash API, and then maybe slash videos.
And you can do, I don't know, however you wanna name this route. It can be initial thumbnail or something like that, right? You can do that if you want to or challenge. Other than that, we are finished with the form section and what we're going to work on building next is this, the video link. So we're finally going to go ahead and see this video And then we can start building views, likes, comments, and slowly wrapping up with the search results page and the playlist.
And I think that one more thing that I have here, which I did not show you, is that I keep this button disabled unless you change something in case that's something that interests you. You can go ahead and find that button and disable it if we are pending or if not, form state is dirty. Right, so when you initially come here, you cannot save anything, but if you change something, then you can save it. In case you think that's a cool thing to do. Great, amazing, amazing job, and see you in the next chapter.
So just to wrap this up, I love doing this part. We've created the thumbnail prompt, the generation workflow, and we've added the skeleton, and we've finished the entire form section. Great, great job.