In this chapter, we're going to add WAPI functionality to our widget application. So in the chapter 6, we actually started developing this, but there was one problem. If you go back all the way to chapter 6, I'm going to help you and remind you here. So what we did is we developed our first AI voice assistant. We did this by creating a VAPI account and we followed their instructions to set up a customer support agent.
We added some files and tools to their knowledge base and well to their dashboard. We then tested that agent from their dashboard and then we tested it again from the client SDK. But there was one issue with this. The issue was we had to use our assistant ID and we had to use our API keys. And this is where I explained white labeling for the first time.
At this point, we didn't really have any of that. But now we do. Each of our organizations now have their own API keys, allowing them to have their own knowledge base, phone numbers, assistants, and tools. So now we can finally go back in this chapter 28, and we can combine all of that and enable each organization to display their own assistant and their own phone number. So let's start by creating a public secrets function which will allow us to decrypt AWS secrets for a specific organization.
And don't worry, we're not going to allow anyone to decrypt the private API key, only the public API key, since this is a public function, meaning it will be used in the UI. So it's a public key for a reason. It is to be stored in such a way that can be retrieved from some API. So we're not breaking any security rules here. We're not going to leak the private API key onto the frontend network.
So don't worry about that. Let's start by building the public secrets functions. So inside of our packages backend let's go inside of public here and let's create secrets.ts. Let's go ahead and let's import values from convex values. Let's import internal from generated API.
Let's import action from generated server. Let's import generate, my apologies, get secret value from libsecrets and parse secret string. So we again need to access those utils that we created the first time we added AWS secrets manager from our secrets lib. Now let's export const. Get Vapi secrets action.
So it needs to be an action because it's going to access a fetch to a third party API. Let me just fix the typo Action. The arguments that we're going to accept is organization ID, which is a type of string, and a handler, which is going to be an asynchronous method. The handler, as always, will have access to the context and to our arguments. Now we have one problem and that problem is actions cannot directly query the database.
If you try and get our plugin by doing await context.database.plugins it does not work. So what are plugins? If you remember, we have the plugins table in our schema and it's basically what allows us to store the secret name, the type of the service, and the organization ID that it belongs to. So inside of your VAPI plugin, whenever you create a connection, what you actually do is you create a plugins schema in the database. And inside of the secret name here we store well the secret name from AWS secrets manager so we never actually store the API keys directly in our database we just store a reference to decrypt them later when we need them.
So now we need to access these plugins so we can obtain the secret name. The problem is we don't know how to do that. Well, luckily for us, we can use run query and in here we can use our internal system. So internal dot system dot plugins dot get by organization ID and service. Looks like we already have this because we used it somewhere else.
So we pass in organizationId to be arguments.organizationId and the service that we are retrieving for this organization will be WAPI. If there is no such plugin, let's immediately return null. There is nothing we can fetch here. So let's just go ahead and quickly revisit this. So inside of my convex system, in here I have plugins.ts and I have get by organization ID and service internal query and in here we're using the index to quickly fetch a plugin and get the secret name for that service and for that organization ID.
So if I search through my code you can see that I already used this a couple of times, right? And now we just added this into this new secrets public function. Now let's go ahead and continue. Now that we have our plugin for this organization ID and for WAPI, we can get the secret name and we can do that by accessing plugin.secretName and then we can get the secret by doing await getSecretValue secretName. This will use our AWS API keys to decrypt this secret, basically this one right here.
We're going to try to decrypt this. And once we have decrypted that, we can finally get this secret data. Now, the secret data will very simply use the parseSecretString and pass in the secret from above. The problem is right now it's a type of any object. We can make this a little bit better by actually giving it the type of what it will return.
How do we know what it will return? Well, we can simply look at the retrieved value here. So we have private API key and public API key. So for now, we are going to map both of them here because we will decrypt both of them for now, but we are not going to return the private one. We are only going to return the public one because the private one should only be used within our API functions.
It should never be returned to the front end, right? Even if it's technically safely encrypted, it's not exactly laid out plain in front of you, which still should not leak it into the front-end network because someone with a high skill set can grab that and that would be well a bad thing to happen. So if there is no secret data at all let's return null. If there is no secret dot public key, my apologies, secret data public key, let's return null as well. Let me just see if I did something incorrectly, so it's public API key, my apologies, return null, let's duplicate this, private API key, return null.
And what we are actually going to return is just the public key. So secret data, public API key. And let's actually also call this public API key so we are consistent everywhere. I just want to double check that we are consistent. So let me search this.
Yes, it's called public API key everywhere and private API key everywhere. Perfect. So there is no instance of private key nor public key, only public API key and private API key. Exactly. We are consistent everywhere in our code.
Now that we have this, let's go ahead and see what else we have to do here. Let me quickly check. So we just created the public secrets functions. Now we have to load those WAPI secrets using our loading screen the same way we just did the widget settings. In order to do that we need to prepare the atoms.
So let's go inside of apps, widget, modules, widget, atoms, widget, atoms. In here let's go ahead and next export const, wapi-secrets-atom, let's make it an atom by default, whoops, let me just do this properly, So it's going to be null by default. But the actual type of this will be a type of public API key, which is a type of string or it's going to be null. Like this. And I'm trying to think if there's a way we can infer the return type of our newly created function but I think this is okay.
But yes, if in the future you ever change your public secrets to return something else here, you should also be mindful and change it in the WAPI secrets atom. That's why I always try to use the infer method here. As you can see, I never really type manually what we are going to store here, but for this specific case, I think this is okay. So I'm going to leave it like this. Make sure that you write public API key inside of an object like this.
Make sure you didn't misspell anything the same way we're doing it here. Once we have this prepared, it is time for us to go back to our widget loading screen. So let's go inside a widget loading screen located inside of the modules widget UI screens widget loading screen and let's quickly check our steps here. Organization session settings and Then we have a WAPI. So let's go ahead and prepare that.
Here it is, step 3, load widget settings. Let's go ahead and just go here. Step 4, load WAPI secrets. Just like that. So let's define a constant getVAPISecrets to be useAction, which we already have here, and let's use API public secrets getVAPISecrets.
In case you are getting type errors here, as always, double check that you have TurboDev running and that your workspace back in here, the last thing you should see should be convex functions are ready. That way you won't have any problems here. Great, Now let's add a new use effect here. The first things first, let's check if we are on the correct step. So if this step is not WAPI, let's break this immediately.
The same way we did in all other steps here. But if it is WAPI let's go ahead and change the loading message here to be loading voice features or you can write loading API keys whatever you prefer. And let's now call getVAPISecrets here and pass in the organization ID. Though this should only be if we have organization ID, I think we can add that here. And this is still showing me an error.
Let me try and see. Maybe if there is no organization ID return. Yeah, I think I use the invalid one here. Let me see. No?
Okay. Not sure what I'm doing wrong, but looks like this is working too. Basically in order to even attempt to load this we need organization ID and this is safe to put here because if there's no organization ID nothing else will work anyway. So I'm okay with blocking this entire flow if it happens that we are missing an organization ID. All right now that we have this let's go ahead and close this and let's do dot then and then in here we have the secrets and now we have to reuse that atom which we've just created here.
So let's do const setWAPISecrets useSetAtom WAPISecretsAtom. Make sure you have imported the WAPI secrets atom. Now we have set WAPI secrets and then in here in the then we can do set a WAPI secrets and pass the secrets and you can see that this is a type of secrets public API key and that's the exact thing that this accepts. That's why you need to be careful with the types. And in case an error happens, we're going to set the VAPI secrets to be null and well that's okay.
This can fail because this is optional, right? This isn't required for the chatbox to load at all. So yeah, I think this should all be okay. The next step here will be done. And the next step here, whoops, will be done as well.
Just like that. Now let's add all the dependencies here. So step organization ID, get WAPI secrets, set WAPI secrets, set loading message, and set step. I think those are all the ones we've used. And in fact, if there is no organization ID, let me just see here.
I still don't like how I did this. Yes, I should find a way to kind of assert that the organization ID will always exist. Let me just find so we do that here. Set there again once we validate the organization. Yes.
So at all of these other hooks, we almost have a hundred percent chance of organization ID existing. Yeah. So I don't like that we have to do this again, but I would rather we do set screen error, set error message. Let me just check. So the same thing that we are doing here, let me just check this one, Basically this.
Yes, this is the exact thing I want us to do. Even though I can't imagine how this can happen really. But yes, let's go ahead and do this. Organization ID is required. Set screen to error and return.
Great. And now what we have to do is we have to go back to step 3, load a widget settings. And Once we finish loading the settings, whatever the result is, null or the settings object, we actually set the next step to be VAPI. So that is quite important, right? Make sure that nowhere inside of this use effect for the load widget settings do you set step to done.
You should only set step to WAPI. So this use effect gets loaded and then we attempt to get WAPI secrets and regardless if this succeeds or if it fails we set step to done because this is not required it's just an extra to make our chatbox even cooler. And at this point if you try the chatbox widget here maybe you will see yes you can see it for a second loading voice features at least I can see it. So after we do that it means it successfully loaded the voice features meaning it loading the API keys and in combination with our widget screen widget settings we can now change the selection screen. So let's go ahead back inside of the selection screen so widget selection screen right here and now we're going to add some more buttons besides the start chat button but only if specific things are loaded here.
So let me go ahead and add const widgetSettings useAtomValue widgetSettingsAtom. Make sure that you add the import for the widget settings atom like this. And now let's also just do one more thing. Let's go inside of our widget atoms And just to make things easier for us, let's do export const hasWAPISecretsAtom, atom, get, get, wapi-secrets-atom and just check if it's not null. So do we have the secrets or do we not?
Now this way in the widget selection screen what we can do is we can use that new atom here. So let me just add has VAPI secrets use atom value has VAPI secrets atom the same one we just created make sure you import it from the widget atoms now that you have the widget settings and the has VAPI secrets you can now safely decide what to show to the user so let's go down here Right now the only thing we do is we display this button. So let me go ahead and copy this button and paste it below. And you will see that now we have two instances of StartChat. So let's go ahead and change this one to be start voice call and this to be microphone icon from Lucid React.
Mic icon, like that. And you can see how that's going to look like. But we should only display this if we have WAPI secrets and if widget settings? WAPI settings? Assistant ID exists.
So regardless if the user has connected their voice assistant, If they haven't changed their widget customization settings, we have no idea what assistant they want to use, right? So you can see that since I have added an assistant, I have this. But if I go inside of my widget customization here, And if I change this to none and save settings now, you can see that immediately when I refresh here, at least what should happen is this should be empty exactly, and when I refresh here, there we go. That button disappears. So even if you have connected your VAPI integration, only when you go to the widget customization and actually select an assistant and select a phone number and save those settings will you see on the next refresh those new features.
So now let's go ahead and do the exact same thing here. But for the phone number like this and this will be call us and is going to be using the phone icon. You can import this from Lucid React as well. And there we go. So if you try and remove the phone number, this button will hide itself as well.
Now what we have to do is add the two new additional screens that will appear when we click on these buttons. The first one is going to be the voice screen. So let's quickly go back inside of our views folder in the modules widget and let's go inside of the widget view. And in here we have voice to do voice. Great.
So that's already set. Make sure you have the voice inside of your screen components here. And now in the widget selection screen you can modify this to show voice. So I'm going to go ahead and let me just find so in here they say handle new conversation but I'm going to make this simpler. Do I have set screen here?
I think I do and I will just choose a voice. So make sure you have set screen in the widget selection screen. You should definitely have it. It's basically the use set Adam using the screen Adam. And now when you click start voice call there we go to do voice.
So you have to refresh to restart this and the same thing should happen if you go here and let's do what is it set screen. It is contact. Yes I think this is the one And then you click here to do contact. Great. So now we have to develop both of these.
We're going to start with the more complicated one. So we complete this fun feature. That's going to be the voice call. The good thing is we already have the hook ready to do this. We just don't have the, we just didn't, this is the hook I'm talking about, use a VAPI, right?
So you already have this, not use VAPI data, use VAPI in the apps widget. We developed this in chapter six, if you remember. And in here, we used our own API keys as well as our own assistant ID. So we're going to have to modify this use WAPI hook and we're going to have to use the atoms which we now have from the widget settings and from the WAPI secrets and that will allow us to speed the process up. So let's start by developing the widget voice screen.
So I'm just going to close these things because I have a lot of them open instead of the widget modules. UI screens. I'm going to copy the widget chat screen. I'm going to paste it and I'm going to rename this to widget voice screen like this. And I'm going to simplify it a lot but it does have some elements that we need.
So, okay, you know what, I think this might be a bad idea because voice screen is significantly easier. So what you can just do is just make the widget voice screen completely empty and let's go ahead and just import a line by line what we need. Arrow left icon, microphone icon and microphone off icon. We then need the button And then we need the AI conversation, conversation content and conversation scroll button from components AI conversation. And then we also need the message, AI message and AI message content.
And then we need our useVapi method from modules. Widget hooks useVapi. So this is the one I was just showing you, right? The one we developed quite early on. And one thing we also need here is the widget header.
So let's export const widget voice screen like that. Let's just prepare set screen here to be used set atom from Yotai and screen atom from atoms. So that's one more thing that we need. And then we can prepare useWapi here. We're not going to do anything now.
Let's go ahead and let's return a fragment here, widget header. Inside of here let's add a div with a class name flex items center gap x of 2 and inside of here I think we can just copy something let's see from our inbox screen. Yes it's the exact same thing here. So you can copy everything inside of the widget header actually and just put it here. And this will bring the screen back to selection and instead of inbox this will be voice chat like that.
And then let's go ahead and do the following. Outside of the widget header, not a paragraph, my apologies, add a div here and give it a class name flex-height-full-flex-column-items-center-justify-center-gap-y-of-4. Full flex column items center justify center gap y of 4 another div inside with a class name flex items center justify center rounded full border background white, padding, 3. And render a microphone icon inside. We already have this from Lucid React.
Give the microphone icon a size of 6 and text muted foreground. Then add a paragraph transcript will appear here and let's give this a class name of text muted foreground now that we have this I think I think this is enough for us to go back inside of the widget view and replace the voice with the widget voice screen which we've just created, right? So screens, widget, voice, screen. We still have some unused components. We are going to add that.
And there we go. We have something that says transcript will appear here. The only thing I think it's missing here. Let me just check. How do we.
Okay. So we also need to add the widget footer. I forgot that. So here at the bottom let's add widget footer and let's import that from components widget footer. Okay.
The thing that seems weird here is it's not taking full height for some reason. So let me just debug a little bit. All right. So I found out that we can make this full height by also adding flex one like this to this div and then it will take the entire screen. Now One thing that I also want to add here just above the widget footer here is another div with a class name BorderTopBGBackgroundPadding4.
And inside of here a div with a class name flex flex column items center gap y4 and then in here we're going to add a div with a class name flex item center gap x of 2 and then let's add a div which will be a self-closing div so it's going to be like a little dot. We're going to style it now so it looks like a dot. Size 3, rounded, full, and let's give it animate pulse and background red 500, like this. So like a little dot at the bottom and besides that dot a span which will say assistant speaking like this and give this a class name text muted foreground and text small like this assistant speaking. And now let's actually go ahead instead of this use WAPI and let me check.
Let's try and get isConnected, isSpeaking, transcript, startCall, endCall, and isConnecting. Great. So we have all of those instead of useVAPI. IsConnected, isConnecting, isSpeaking, and the transcript. And we return all of them.
Perfect. So we can actually enhance this already and make it make a little bit more sense here. So this is what we're going to do. First let's change the text. If is speaking then change the text to assistant speaking.
Otherwise listening. So we indicate whether it is our turn to talk or is it the assistant's turn to talk. And then let's go ahead and wrap this class name inside of CNUtil. Let's import CN from workspace lib UI lib utils. So this will be the default classes like this.
Let's go ahead and remove background red and animate pulse and let's do if is speaking then do animate pulse BG red 500 otherwise simply do BG green 500. So for now it's going to be listening, right? But this entire thing will only be displayed if it's connected so if we're not connected we're not going to display this at all like this so right now this is just empty so instead what we're going to do is just below this let's add a new div here with a class name flex full width justify center is connected question mark We're going to do a button which will say end call. Otherwise, we're going to do another button which will say start call. There we go.
So start call right now. And now let's go ahead and style this. So the class name will be full with disabled will be if is connecting. Size will be large on click will be for now an empty arrow function. There we go again.
Keep in mind this will be a very small like this. And let's for now just change this to true so that we see the end call button. And now let's do the same thing for end call. So class name here will be full width. Size will be large.
Variant will be destructive. And on click here will be just an empty arrow function. There we go. So now we have end call. Change this back to is connected and you should have start call.
Now let's add some icons here. So this will be microphone icon and in the end call add mic off icon. There we go. Perfect. And we actually don't need the widget footer.
You can remove it. And you can remove the import for the widget footer as well. So just like this transcript will appear here and the button to start the call. The problem is if we attempt to start the call now it's actually not going to work. So we have the start call button, right?
So we can use it actually. Let's find this start call. Let's just do start call here. The problem is we are going to get an error. As you can see, Assistant or squad workflow must be provided.
That's because currently instead of our use WAPI, we have empty start and we have empty API keys. So if you want to, you can use your own API keys just like we did in chapter six. Why would you do this? Well, if you had some trouble with AWS, yeah, feel free to just, you know, for fun add your own API keys and your own assistant ID here just so you can follow the tutorial along, right? But for those of you who are able to set up the AWS secrets thing we're going to have to go inside of use Vapi and we're going to have to actually load those API keys.
Just before we do that let me add the end call method here. So in here end call like this. Now let's go inside of the use WAPI and let's add the atoms here. So I'm going to do const WAPI secrets useAtomValue from Yotai and pass in the WAPI secrets atom. There we go.
So use atom value and WAPI secrets atom. Besides that, widget settings. Use atom value, widget settings atom. From the same place. And now that we have both of them, let's go inside of the use effect here and first things first, if there is no WAPI secrets, let's immediately return there's nothing we can do here.
We can now remove this comment and we can now initialize the new WAPI with WAPI secrets dot public API key and you have officially learned white labeling from start to finish. Now each organization can bring their own API keys and see it in their chat box. Amazing job. But still we have to actually make this work so let's quickly go down here where we use the VAPI start and in here we're going to do a similar thing so set is connecting to true sure but then actually no don't even do it. So before just check if there is no WAPI secrets, simply return.
Or if there is no widget settings dot WAPI settings dot assistant ID. There's nothing we can do that. And then you can finally just use the widget settings dot WAPI settings dot assistant ID. Perfect. And now you can remove this comment too.
And let's go inside of the widget voice screen here and let me just find a nice place to render the transcript. So we can do that just below the widget header here. Let's just do json.stringify transcript null and 2. The transcript is extracted from the uswapi hook. So I'm going to refresh.
I'm going to go inside of start voice call and I'm going to attempt to do this right now. You can see this is an empty array. So I'm going to pause the video and you will either see the transcript or maybe an error. And what we got was a combination of both the transcript and an error. So I'm not sure if you got this error, ignoring settings for browser or platform unsupported input processors audio.
I'm not too sure what this is, but it looks like it's working even with that. So I'm just going to Google this for a second just to see if we're doing something incorrectly. Maybe it's because whenever you start a voice call you actually have to like interact with the website. Let me see. Can I get that error again?
So I'm starting call now. Same thing. I'm getting the error and now the assistant is actually speaking and you can see things here. It's kind of working but I'm not sure what this error is. Let me just quickly research this.
All right so I've searched for that exact issue and apparently it could be just a web browser thing. I'm not a hundred percent sure and I'm not even sure if you will experience this. It could be the microphone that was selected for my browser here. Maybe something is wrong with that. I'm not too sure, but I can continue building forward so it's not really a problem for me.
But all I know is that I did not have this error when I first started, when I first built this. I'm not exactly sure why it's happening right now. I will definitely try to give you some more answers in the next chapters but I hope that you are in the same situation as me at least so that you can at least see the transcript here because there is one more thing we have left to do here and that is display the transcript in a nice way. So for now I'm just going to ignore this because this doesn't seem like a breaking error. It just seems like a warning like it's ignoring the settings for browser unsupported input processors.
So that is definitely talking about the microphone. So maybe this is only for me. Maybe you don't even have this. I have no idea. That actually does make sense because I'm now speaking in my recording microphone but when I developed I was using my Macbook microphone so maybe it's about that.
I'm not sure. I just hope that you can see the transcript. I'm going to ignore this error for now and I will give you some answers in the future chapters if possible. For now let's go ahead and let's make this transcript appear in a nice way. So let's go back inside of the widget voice screen and now what we're going to do is just after the widget header let's go ahead and do the following.
If the transcript.length is larger than zero, in that case we're going to display the transcript. Otherwise we're going to display this transcript will appear here, div. So let me just quickly add AI conversation here and let's go ahead and give it a class name height full flex one and inside AI conversation content transcript dot map. Dot map. Let's go ahead and let's do message and index.
A.I. Message A.I. Message content. Message dot text. And let's give the A.I.
Message from to be message dot row and key. Let's just make it message dot row. Let's do a combination of things. Let me just try and do this like that. Let's do an index dash message dot text.
I think this should be somewhat unique. Great. And let's also add AI conversation scroll button, which is a self-closing tag. And now you should have your messages displayed like this. So I'm going to start the call again and I'm going to test it out.
And here we go. So you can see that my bubbles are displayed in blue, their bubbles are displayed in white. So same like our chat interface here. The only thing I don't like is how it scrolls. So it should only scroll within this area.
So let me see, can I somehow fix this overflow? Why auto maybe here? I'm not sure. I think it's mostly about this flex one thingy. Not sure I will have to kind of debug this because I don't like how it's scrolling right now.
Let me try. Scroll. No. No. It's not fixing right now.
Anyway. Yeah. Let's leave it at flex one for now. Perfect. But this is where I want to left this.
Leave this chapter. Now We do still have this call us but that's super simple. We can just do that in the next chapter is just some UI and the display of the phone number. But in between those two chapters I want to kind of give you some answers about this even though I'm fairly certain this is the microphone that I'm using. And I also want to find a way to fix that scrolling issue so it actually displays in a nice way.
And then we are going to wrap it up with this. But so far, amazing, amazing job. So let's just quickly see in here. So we added the WAPI secrets in the loading screen. We modified use WAPI hook to use organization secret and we modify the selection screen UI to display a voice option.
So we did this, we did this and we developed the voice chat. So exactly what we actually had envisioned here. In the next chapter, we're going to do the call us or contact us screen, which is far more simpler than this. Thankfully, we were able to reuse our use VAPI hook from before just by plugging in the new WAPI secrets and the new assistant ID from here. So now the user from their dashboard can choose exactly what assistant they want to use and what phone number.
And this is basically such an insane improvement over just us wrapping the VAPI API and trying to replicate all of their features. Because now, you know, our users can just head to VAPI. They can go inside of their dashboard and in here, you know, they can just set up whatever they want and just load that assistant here and load that number here and that will that will be a much better experience than us trying to replicate the VAPI right. Perfect. So now let's go ahead and merge this.
So 28 a widget API. Let me go ahead and close the graph. I'm going to stage all the changes when the 8th widget VAPI. Let's go ahead and commit. I'm going to open a new branch 28 widget WAPI and I'm going to publish this branch And now let's open the pull request.
So here we go, compare and pull request, 28, a widget WAPI, that's correct. And now let's review our changes. And here we have the summary. We added voice chat to the widget. We can start and end calls.
We get the connection status, as well as live transcript display. We introduced a dedicated voice screen with controls and real-time conversation view. The selection screen now shows voice call and call us options, but only when voice features are available. The enhanced loading flow to fetch voice feature access and proceed accordingly with graceful error handling. Amazing, amazing, amazing job.
Let's go ahead and merge this pull request here. And now let's go ahead and see if that's all we have to do. That's right. We just had to merge the pull request and as always, whenever you merge, head back instead of your IDE, go back instead of the main branch and click synchronize changes just to make sure that your main branch is now up to date with your GitHub repository and check the graph here to confirm that the latest thing you just merged is 28 widget WAPI. I believe that marks the end of this chapter.
Amazing job and see you in the next one.