In this chapter, we're going to learn how to build an AI voice assistant using WAPI. More specifically, we're going to build a bank customer support agent that will be able to verify our account using the last four digits of our phone number and then give us back our bank balance. We're going to test this agent first using the WAPI dashboard and then using the client SDK for our Next.js application. And after we finish that, I'm going to briefly explain how WAPI's API work and our multi-tenant needs. I'm going to try to explain how we're going to white label WAPI's API, allowing our customers to bring their own API keys, making our app that much more flexible because every single customer will be able to configure WAPI the exact way you're going to see we will be doing it now.
And that is an amazing feature of this B2B software as a service. So let's start by creating a WAPI account. You can use the link on the screen to let them know you came from this video. After that, let's go ahead and create an account and head to our dashboard. Once you land in your dashboard, in the upper right corner, you will find the documentation button.
In here on the left side, in the sidebar, click on the WAPI guides. I highly recommend that after you finish this chapter, you try doing one of these on your own to get more familiar with WAPI, because this will explain to you all the things that your customers will be able to do thanks to the white label method that we are going to be doing. For now, we're going to build one of these together, specifically the inbound support. We're going to be building a banking support agent with function tools and knowledge bases. So let's go ahead and first download the knowledge base using the spreadsheets.
If you cannot find them for whatever reason, using the link on the screen you can access my public repository echo assets and in here find the wapi folder and you can find the same spreadsheets here as well as all the prompts and descriptions needed for this guide so now let's go ahead and upload these files to our WAPI dashboard. So head to the files in the sidebar. You can see that I already added them. Make sure you have them as well. After that, let's go ahead and create an assistant and call it Tom.
So head into your assistants, create an assistant, call it Tom and select blank template. Once you have created your assistant you're gonna have to modify the first message and the system prompt. You can find both of them inside of my public assets folder, first message, system prompt or simply follow the documentation along. I'm going to modify the first message here and I'm going to modify the system prompt. Just like that.
After that, if you want to, you can also configure the LLM settings to choose a different model, but I'm fine with this. So I'm going to click publish. Once you've done that, let's go ahead and let's add tools to our assistant. So head into tools and click create tool, select query. Once you've created this new tool, let's go ahead and rename it to get balance let's go ahead and paste the function description here and let's go ahead and click add knowledge base let's go ahead and call it accounts Let's give it a description of use this to retrieve account information.
Let's go ahead and select the files, accounts and go ahead and click save. After that, let's create Get Recent Transactions tool. Again, a query tool. Let's go ahead and give it a description. After that, let's add two knowledge bases.
First one will be access to the accounts just as in the previous one with the same description and same files. Then add another knowledge base transactions. Let's go ahead and give it the description and let's select transactions and let's click save. Let's go ahead and create our last tool, which will be called lookup account. Let's rename it to lookup account, let's give it a description and let's go ahead and add a knowledge-based accounts and just as in the previous examples, let's add a description and let's select accounts and let's click save.
After you have added those three tools, let's go ahead and add them to our assistants. So head into assistants, select Tom, scroll down until you find tools and go ahead and select lookup account, get recent transactions and get balance and hit publish. After you've done this, you are ready to test your assistant. You can do that using the Talk to the Assistant method. I would highly suggest that for the first run, you try the following.
In the first question, ask it to verify your account. After it asks you to provide the last four digits of your phone number, respond with 1234. Why that number? Well, if you take a look into our accounts.csv, you can see that that is the only available account. So that's how the lookup function, the lookup tool will be successful only if you pass the proper account here.
And then when it asks you what do you want help with, I highly suggest asking it to get balance. I've had it fail a few times with recent transactions, which highly depends on what model you select and what CSV file you have added. So for best results, I highly suggest asking it to get your balance. I'm now going to click talk to assistant and I'm going to pause the video after each checkpoint. So after I have clicked on start call, I first had to allow access to my microphone and then I asked Tom to verify my account.
After that, I told it 1, 2, 3, 4 and after that my account was successfully looked up. And now I have to ask a question again, but that is only true for this test call, not in the actual call. You can see that after I have asked to give my balance, again, it shows me that it ran a tool. So now I have to ask again and it will print out the balance. This will not happen in a real example.
And there we go. So every time that it activates a tool you're gonna have to ask the question again. Again this only happens in this testing mode, not in the real mode where your customers will talk with this AI. You saw it in the demo how well it works. So after I asked it, what is my balance?
It basically told me the exact balance that I have inside here. So I highly suggest that when doing testing, you test using this script. Hey Tom, can you verify my account? One, two, three, four, and then can you give me my balance? Perfect.
Amazing. So what did we just achieve? We achieved a very, very simple VAPI assistant. But as you saw in their guide here, in their guides, they have much more complex workflows here. You can do appointment scheduling, you can do medical triage and scheduling, and all of these things, as you can see, you can do no code, drag and drop things with nodes and all of those things.
You can see they have much more complex examples. And I highly recommend that you play around and try and create one of them on your own so you see the true power of Vapi. And this is what each of our customer will be able to do, right? So we are not going to limit them by just wrapping, just being a wrapper around WAPI. Instead, we're going to be a white label provider so that whatever customer knows how to use WAPI like we just learned to do, we're just going to give our app API keys and that will automatically pull all assistant IDs like Tom, who we just created, and all phone numbers.
So phone numbers is something that we haven't done yet. So let's just quickly do that. Head into your phone numbers here. You can see I already have one because I just tested this a second ago. You will probably have none.
So just click create a phone number and the trick is right now they only allow US phone numbers. You can still create them outside of US but you can probably only call them if you are in the US otherwise you will have high costs. I don't know what is the area code for the US so I will just use whatever one they recommend. Let's try 928. There we go.
And then simply call this Tom Wapi Bank Hotline. Something like that, and select your assistant to be Tom. So this is what your customers will be able to do. They will be able to create phone numbers, workflows, tools, files, assistants, all of those things on their own using Vapi. And we are going to tell them, great, just give me your API keys and I'm going to fetch all of those and display them in your dashboard.
And you will tell me which one, which assistant you want to use and which phone number you want to use. And I'm going to display them in the customer facing widget on your website. So that is how this is going to work. I think that's a super cool feature and something we've never done on this channel before, bringing this app even closer to enterprise grade software. So let's go ahead now and test out this voice agent the same way we just did using the WAPI dashboard, but let's do it inside of our Next.js application.
So we're not going to do this inside of our web app, we're going to do this inside of our widget app, because this is where customers will be able to talk to AI assistants and AI chat. So let's go ahead and do pnpm filter widget add at wapi-ai forward slash web. So this will now install that package specifically in the widget application. So just make sure you have added that package JSON here. Perfect.
And now let's go ahead inside of our apps, widget and in here let's go ahead and create a new folder, modules, let's create a widget module and let's go ahead and create hooks inside usewapi.ts Let's go ahead and import WAPI from our new package and let's import useEffect and useState from React. Let's go ahead and create a type called the transcript message with a role of either user or assistant and the text of type string. Let's export const useWapi. Let's go ahead and let's first define const wapi and setWapi inside of a useState. Give it a type of wapi instance or null.
Now we're going to create three booleans. All of them are going to be the same. IsConnected and SetIsConnected and same thing for IsConnecting and IsSpeaking. So just be mindful there is a difference between these two. All of them will be false in the beginning.
And the last state here will be the transcript which will be a type of transcript message but in the form of an array. Now let's create the use effect. Go ahead and give it an empty dependency array and now let's create the WAPI instance using new WAPI. Inside of here we now have to add our public API key. So head inside of your WAPI API keys and click add key.
I'm going to call this widget echo tutorial And I won't select any of these other ones. I will just click create public token. Widget echo tutorial. I'm going to click copy. Make sure you did this for the public API key.
And simply paste it inside. So why am I adding this like that and why am I not adding this in the environment file? Well, I'm going to add a comment as to why. Only for testing the WAPI API. Otherwise, customers will provide their own API keys.
That is why, because as I just explained, we are not going to use our own WAPI keys. Each customer will have to add their API keys, allowing them to create agents of their own to create workflows of their own and phone numbers of their own making our app much more flexible and for us a great lesson in white labeling other APIs So let's now set wapi to be wapi instance. And now let's go ahead and let's do something on the wapi instance call start. So when the call start happens, let's set isConnected to true, set isConnecting to false, and set transcript to an empty array. Now let's do the same thing on call end.
Set isConnected will be false, set isConnecting will be false, as well as set isSpeaking. Now let's add two very simple ones. On speech start setIsSpeaking to true and on speech end setIsSpeaking to false. Now let's also add the error state and I also recommend adding a little console.error here. Now let me just obtain the error and just pass it here simply so you can see if something happens and you cannot debug it.
And now let's go ahead and do WAPI instance on message. Now in here you have to be a little bit careful because message is a type of any. So what I'm going to do is carefully type if message.type. The thing is you can add this and it won't throw you an error, right? So make sure you don't misspell this.
If message type is transcript, you can also misspell here, so be careful. Transcript. And if message.transcript type is equal to final, same rules here, only then call setTranscript. We're going to get the existing values in the transcript and we're going to return an empty array. Then we're going to reuse the existing values in the transcript and we're going to add a new one.
We are going to assign a role for this transcript. Who is speaking? By checking message.role and checking if it is a type of user. If it is, assign user, otherwise set it as an assistant. And the text will be message.transcript.
Make sure you are not misspelling transcript in any place because this is a type of any so you can misspell things very easily here. Before we end this use effect make sure to add a return method which will call a VAPI instance question mark dot stop. Now let's go ahead and let's do const start call method. Set is connecting, whoops, to be true, and call if it exists vapi start. Inside of start we have to define which AI assistant we want to call.
So head into your assistants and copy the ID of Tom and paste it in here. I'm going to copy this comment from above one more time. The reason we are pasting it like this is because this is only for testing the WAPI API, otherwise customers will provide their own assistant IDs. So that's what white labeling will do for us. And now let's add one more function end call and finally from this hook let's return all of those things is speaking is connecting is connected transcript start call and end call Now let's go ahead and use this hook.
I'm gonna go ahead and go inside of apps, widget, app, page.dsx. And in here I'm gonna actually remove everything besides the button here. And I'm just gonna leave the button and nothing else like that and now I'm gonna go ahead and I'm going to call my new hook useWapi from modules.widget.hooks.useWapi So this is directly inside of the widget folder, modules, widget, hooks, use WAPI. Now let's go ahead and let's destructure everything that's being returned from WAPI. IsSpeaking is connected all the way to end call, like that.
And in here, let's call start call and the button will also say start call. Ummm sure this can stay like this. Below it add the end call method and call. Give it a variant of destructive. Let's also give this a maximum width of MD, MX auto and full width.
And then let's go ahead and add a paragraph here is connected and inside of here open backticks and render is connected and then the same thing is connecting and is speaking and then let's go ahead and let's json stringify our transcript like so. I believe we now use everything from here. Make sure you have use client at the top right here and now let's go ahead and do turbo dev in the root of our application. Since we are working within the widget dev we have to go to localhost 3001 and we should now have start call, end call and indicators here. Make sure to open your console so that you can see if any errors get logged here.
When you click start call, you will most likely have to allow access to your microphone, so be mindful of that. So I'm going to attempt to have the same conversation that I just had in the WAPI dashboard. I will ask it to verify my account and then I will ask it to check my account balance. And here we have our conversation. So we started with, hello you've reached WAPI Bank customer support.
My name is Tom, how may I assist you today? Hey Tom, can you verify my account? That was said by the user. Sure, please provide the last four digits. I said, one, two, three, four.
Did you verify my account? Yes, I verified your account, John Doe. How can I help you today? I said, I would like to check my balance. And after that, it told me your current balance is $2, 534.
So the exact same experience that we just had on the WAPI dashboard has been transferred to us here. Perfect, so that is what we wanted to achieve, amazing. So now let's go ahead and see what we've done. We've created a WAPI account, we've set up customer support agent, we've added the knowledge base, we've added some tools, we tested the agent from the dashboard and from the client SDK. But now let's talk about WAPI's API.
So initially my idea was, okay, now what I have to do is I have to add my secret API key for WAPI and use WAPI's server side SDK and for each one of my customers, create their phone numbers and create their assistants and all of those things. But then it hit me. Why am I actually doing that? Because all I'm doing is I am recreating the entire VAPI dashboard. That makes no sense.
Like WAPI's dashboard will always be one step ahead of mine. So it doesn't make sense to do that. So I thought, well, how about a white label WAPI instead? So we are going to learn for the first time how this works. But there's another problem at hand, right?
Another reason why I chose the white label. So right now, each of these entities, phone numbers, assistants and tools, All of them are tied to one API key and they are not separated by tenant. So from a security standpoint, I would have all of my knowledge bases in my account. And that is a kind of weird thing to do to your customers, you know, coupling all of their accounts into just one knowledge base. There's a couple of things that can go wrong here.
So I thought, okay, how about instead of one WAPI key, one tenant and single phone number, single assistant, single tools, how about I allow my users for my B2B SaaS application to white label WAPI's API and I tell them, give me your API keys. And then I'm going to fetch all of your phone numbers, your assistants, your tools, your workflows. So I can now have infinite number of tenants and all of them will have separate knowledge bases, separate phone numbers, separate assistants. That is where the idea to white label comes from. And if you're wondering how do we store these ABI keys that sensitive data we're going to be exploring Amazon Web Services more specific more specifically the Secrets Manager that is where we are going to store our keys.
Amazing amazing job! So as I said I now recommend after you finish this chapter that you go back into the WAPI documentation, head into WAPI guides and try and do one of these on your own. I will probably play around this some more and then you can create as many of them as you want. And all you will have to do later is add your API keys to our customer support app. And we are automatically going to fetch all the assistants you have created and all the phone numbers you have created, allowing your users to use your VAPI account directly.
So go ahead and try to build one of these on your own, if you want to, of course. Now let's go ahead and commit these changes here. So yes, we can leave them like this for now. So let's go ahead and go inside of here. I'm going to select all of these changes.
Let's see this chapter. 06 AI voice assistant. Like so. Let's commit. I'm going to click on the main here I will create a new branch 06 AI voice assistant like that and I will click publish branch after that let's go inside of my repository here.
Let's go ahead and open a new pull request And let's review our changes. Even though most of this is just for test, but I do want to see our hook reviewed just in case. And here is the CodeRabbit summary. We introduced voice call functionality with controls to start and end calls. We display a real-time call status and a transcript of the conversation.
So the changes introduced a new custom React hook called useVapi to manage voice call interactions using the VAPI AI Assistant within the widget application. That is exactly what we did. In here we have a sequence diagram explaining what happens. When we click start call, we use the use VAPI hook and we initiate the call with the assistant id. The assistant then emits events such as call start, speech start, transcript, etc.
And we update the state accordingly and then we render the call state and the transcript. And call works very similarly. And in here it gave us two comments and that is not to use hard-coded API keys and assistant IDs obviously we are not going to do that I just added it as an example. But yes, make sure that this isn't a public repository. I should have definitely told you this sooner.
Or what you can do simply, if you have committed this already, just go inside of your WAPI API keys and delete them, right? So in case this is a public repository you're working with, so no one can steal your public API keys. Even though public API keys are kind of meant to be public, right? It would still be a good idea not to exactly share them with everyone. Great so we're going to remove that in the next chapter so we no longer have that issue and for now you can just click merge pull request and confirm merge and after you've done that I'm going to change my branch back to main and I'm going to synchronize my changes.
After I have synchronized my changes in the graph below, we should see an example. We detached to do AI voice assistant and then we merge that back into main. Amazing, amazing job. Perfect, so now we have a base model for our voice assistant and how our customers will create support agents and then give us their API keys so that we can call this use WAPI hook with their assistant ID and with their next public API key. That's how this is going to work.
And just, you know, a small note here, in case you had any trouble finishing this and you're worrying, will you be able to continue the tutorial if this, for whatever reason, doesn't work for you? Maybe you don't have a microphone or something like that, don't worry. This is a very cool additional part of this project but you will be able to continue this even if you had some trouble doing this. Amazing, amazing job and see you in the next chapter.