In this chapter, we're going to continue working on node execution by implementing templating language. This will be a particularly powerful feature of our project, and I'm so happy that we are finally in this chapter, because I think it's very, very interesting how we are going to achieve this. So the first thing we have to do is we have to refactor one implementation from the previous chapter. In the previous chapter we added variable name which basically fixes the key collision issue which we discovered two chapters before this. So let's go ahead and quickly fix that before we discuss exactly how we are going to implement this.
So we're going to go ahead inside of source features executions components HTTP request executor.ts. So what problem do we have? Even though we have an if check for the data variable name missing and we throw an error, still if I remove this if clause right here, we get an error. And because of that, we have this weird solution where we wrap this instead of an if clause, and then we fall back to this one. Since we made a decision that variable name will be required, we can safely remove this fallback now.
And in order to fix this, we can very simply define inside of HTTP request data which ones are actually expected to be required. Keyword expected. So let's remove the question mark for variable name and let's also remove it for endpoint. This way these two are expected to always be required and we can also do the same for the method actually. But even though we expect them to be required, the only way we currently validate that is on the frontend using React Hook form, which is fine but not good enough.
We need to be secure. That's why we are doing these things right here. We throw error if any of our required fields are missing. So let's go ahead and just do the same thing for the method here. And very simply, method not configured.
As simple as that. If you want to, you can continue adding like these type of errors, simply it's easier to find in the logs. Great, So now we do make these required in the types but we also do a runtime validation just in case they are not passed because users can easily bypass this. So we want to make sure that these are actually passed so that we can properly do the request. And now once you've done this, you can actually remove the question mark here because this will now always be required.
And let's see, the method, same thing, you don't have to fall back to get because method will now always be required. Perfect. So that's how we resolve that first issue. Our code is much cleaner looking now. So let me quickly mark that as completed.
So what is template syntax? How are we going to implement that and why do we even need that? Well, take a look at this scenario that I've prepared right here. So very similar to our previous project. I have one HTTP request with a variable name myAPICall1 with a get method and a simple API placeholder calling todos with an id of 1.
And I have slightly modified the second one. So this one is called myAPICall2, which instead of calling todos calls users. So let me rename this. This will be my users. Let's call it user.
And this will be my to do. Perfect. So make sure you click the big save button and go ahead and execute your workflow. Just make sure you have both Ingest and Next.js running. And let's take a look at what happens.
I'm going to expand this as far as possible. Let's go inside of finalization. In the result, we now have to do object with HTTP response data and you can see it has properties of the to do but we also have a user object with all of the fake mock user information. So great we can successfully fetch one hard-coded todo and one hard-coded user. But what if I specifically wanted to do this?
What if I first wanted to fetch the todo And then I wanted to use the todo user ID in the next node. So what if in here, I wanted to do this, I wanted to pass, so this would be todo.httpResponse.data.userID Quite long, we could maybe shorten that, but this is how it is accessible for now, I believe. Let me just confirm. To do, httpResponse.data.userId with capital ID. There we go.
So if I click save here and if I just try running this, it's going to fail, I believe. Let's see. There we go. You can see that the second HTTP request is failing with 404 not found. So I'm going to cancel it now so we don't waste any time.
So now we're going to fix that by implementing the template syntax. For that we're going to be using handlebars. So let's go ahead and install handlebars. So I'm gonna go ahead and open a new tab, npm install handlebars. And then I'm going to show you the exact version that I'm using.
Handlebars 4.7.8. And let me just quickly try importing that so import handlebars from handlebars and looks like type safety is here as well perfect so now that we have handlebars Let's go ahead and let's try and fix this. So what I'm going to do is I'm going to go to this endpoint right here and I'm going to call handlebars.compile. I'm going to pass in data.endpoint And I'm going to pass it context like this. So what's going to happen now is handlebars.compile will read data.endpoint, which is going to look like something, right, blah, blah, blah.
Let me try and mock this. And then it's going to have to do dot http response dot data dot user id right so handlebars dot compile will read this and then it will use the context to populate that. So what is the context? Each of our executor has the context. The context is basically the previous node data.
The context always gets updated with each sequencing node. So the first HTTP request that's done, if we tried, for example, if you tried to do this for this one, right, for the first one, it's always going to fail, right? Because there is no context behind the first one but the second one will have context it will have to do context because we named this variable to do so I think just by doing this this should already work and let me just try and do console log end point and let me try and do end point. I'm using capital letters it's easier to find it and if we've done this correctly we should see the end point logged out. Well if it works I mean we will actually be able to inspect the URL itself, right?
Basically, what I expect to happen now, make sure you save that file, I think I've done it correctly. Maybe I'm missing something, but I think it should work is there we go. If I use to do HTTP response dot data in here in the users, this should compile to users 1 because the todo 1 has a user ID of 1. So let me go ahead and click save here one more time. Let's go ahead and click execute workflow.
And let's see if anything will change now or maybe we'll need to fix something and given that it completed, I think we just fixed this. Let's quickly take a look. So this is the first HTTP request and it's stored into a to-do variable, HTTP response data, with user ID of 1. And then in the second one, let's go ahead and scroll down, we now have a user. And you can see how we have the context of the previous HTTP response which allowed us to populate the user ID and I think this works as well.
Let's just confirm by finding the ID here and it is one. Amazing. Can I directly see the URL somewhere? Let me go ahead and try here maybe. There we go.
Endpoint. JSON placeholder type of code to do is one and you can see this is the one. Users one. So it's successfully compiled to exactly what we expected. Amazing, amazing job.
It really is that simple to do that. Now the only problem is you currently can't do that with JSON objects, right? And I mean, right now you don't really need to do it. We use the super simple example, but what if you change this from get to post? Well, now things get a bit more complicated, right?
Because sure, this is as simple as before, but what if you wanted to use the JSON variable object? Well, for that we need to do some slight modifications here. We need to register a helper to stringify objects so I'm going to do handlebars.registerHelper let's go ahead and add json grab the context and immediately return json.stringify context and immediately return json.stringify context null and to. As simple as that. There we go.
Now once you've registered that, we can go ahead and go inside of our HTTP request step here, go inside of post, put and patch. And now for the data body, what we can do is we can first try and do the following. So let's do const resolved to be handlebars.compile data.body passing the context like that. And then let's do json.parse resolved. So this will basically protect us from any invalid JSON being passed.
That's why we are doing that. And then, instead of options.body directly being that, we can just do this. Great. So yes, let's also do, while we do post put patch, I also think, I wanted to add the body to be required, but I'm not sure if body is yeah I don't think it's a good idea to make body required maybe we should just treat it like an empty object if it's not passed at all I'm not sure let me just quickly that is json.parse let me zoom in here Can I do json.parse on an empty string? I cannot.
Okay, so yeah, maybe we should fall back data dot body to like an empty object. Actually it depends if handlebars can compile an empty string. You know what, let's just try it so I stop guessing things. I'm just going to check what is the proper endpoint URL to test this on. Let's check quickly.
I think we can just do to do's. Like that. Change the method to post and just pass something like this. I don't think indentation matters that much but what does matter is that you don't have like an trailing comma at the end. So only use commas if there's a new line after.
And make sure that your key values are encapsulated within a string. Otherwise, that's not valid JSON. So we have foo, We have bar and user ID. I think JSON does allow numbers. So this should work.
Basically this should now create a to do. I think. So let's go ahead and call this created-to-do. Like so. Oh great, our validation works.
Created to do. Perfect. Let's click save here. And now we should have one GET request and then one POST request. Right now, not using any variables at all.
So let's just execute workflow. Let's see, does that work? Looks like it's working. There we go. Created Todo has an HTTP response with a body of bar id 201 title of foo.
Amazing. And now let's go ahead and let's try reusing something here. So okay I can use this. How about we try user ID dash and then let's use to do dot HTTP response dot data dot user ID. So in the title of the newly created to do we're going to use the variable that we previously previously used here to fetch the user.
So doesn't make much sense we should technically use it for this but I think it will like be more visible if we do this. So let me go ahead and I'm just going to zoom out a bit so I can click save we should probably improve that. Let me go ahead and save the project overall. You always have to save it and then click execute workflow and let's see if that will maybe be more visible. Create it to do, there we go, title user id dash one.
So it is officially working. We can successfully template both in the endpoint object and in the data.body object. Now let me just check what happens if I pass an empty request body and if I click save. So I'm just trying to see what bugs can we encounter here. Now this is completely fine to fail but I just want to see why it fails.
Okay I see because of unexpected end of JSON input. I see. I'm not sure if we should handle this or should the user handle this. So should the user know that the request body always needs to be something, even just an empty JSON object. Because I think that now when you save this and execute it, it's still going to fail, but it should fail for a different reason.
Yeah, okay, it didn't fail. It managed to create it. So yeah, you can make a decision. Do you want your users to know that they should always add something to the request body when they select a post method even if it's just an empty object or will you detect when body is empty and then maybe change it to this perhaps if you do this then you kind of save your users the hassle. Let's see if I remove this now and click save here, click save here and if I execute it I think it will like save your users some headache.
Let's see yeah now it works. It works exactly the same as before. Perfect. But still, we haven't tested one more thing. And that is the variables, right?
So what if I wanted to, let's me, can I see some better example? This one. What if I wanted to add the entire data object? Like I literally wanted to create another to do with identical HTTP response here. Now, right now, what we can do is, let's see what actually would be the proper way of doing it.
I'm not even sure myself. So I think that What we should be able to do now is pass JSON to do HTTP response dot data. I think this should work now. I'm not a hundred percent sure. Let's see.
We registered the JSON helper. So basically whatever you wrote here, you can name this anything you want. It's a name of the helper. So it's a reserved keyword here. So if I name this JSON2, I would also have to register JSON2 in the helper.
That's why we call the JSON here. And basically, it should be able to just transpile that entire, I mean, compile that entire object and just create two identical to-dos. So the flow is now fetch the to do under ID one and then create a new to do with the JSON, entire JSON object that we fetched from the previous node. That is a pretty reasonable request actually. Let's go ahead and see if that works or do we have to do something else.
It looks like it has failed. Let's see why. Expected property name or that in JSON at position 4. All right, I think that we are making some mistake here. I'm just not entirely sure how do we maybe need to wrap this inside of curly brackets like this?
Maybe, I'm not sure. Let me try and add some simple indentation. Let me click save because I'm trying to think how does this compiling happen. Does it happen like parsing? Looks like not.
Again, syntax error. OK. The way I would debug this now is, of course, by logging. I want to see what exactly is going on here. So let me go ahead and console log result body.
And this way at least I will be able to see what's happening because that's always kind of the problematic part and I will remove curly brackets. I want to see it in a string object, right? I want to see what's going wrong. So I'm going to remove curly brackets again and I'm going to remove all white space too. So I only have this.
Let me click Save here, Save up there. And let me click Execute Workflow. Again, I am expecting this to fail. Perfect. We can actually cancel all of the running ones because we know they're not going to succeed so we don't waste any time and now I'm going to go here and I'm going to take a look at them now.
Oh that's interesting. Body. So it managed to do it but it's completely incorrectly parsing this. That is interesting. I would assume that we fix that right here.
I'm going to have to pause the video just a little bit and research and I will do my best to tell you exactly how I debug this. Of course, you can do it yourself. You can pause the screen too, if you like a challenge. Basically, the issue that's happening right now is we do manage to get the exact body from the variable, but you can see that the quotes are broken, right? These quotes should be actual quotes.
So I'm going to go ahead and try and resolve this. Right. I think I might have a solution. Unfortunately, not a cool solution. I mean, not a cool way of discovering it.
I just asked Chad GPT. So yeah, not exactly too cool. But let's go ahead and modify this a little bit. I never liked this direct returns. So I'm going to open it like this.
And then I'm going to do, let's go ahead and const stringify to be JSON stringify, pass in the complex null and two. And then I'm going to return new handlebars dot safe string. And I'm going to pass in stringified inside. Actually let me call this const safeString like this and finally return safeString. I always found these to be more readable and let's call this JSON string if I JSON string.
There we go. JSON string and safe string and finally return safe string. So I described the problem to Claude and it told me to try this. So I'm going to go ahead and execute this again with no code changes. I mean, with no changes.
And it looks like that actually fixed it. Amazing. That's really cool. So let's see. Created Todo has completed of false ID of 201 title, this one, user ID, this one.
And our previously fetched to do is identical except ID, which makes sense. We cannot force the ID. I think this is a public API, right? So that makes sense that the ID wasn't honored. That's perfectly fine.
Amazing. I think that we successfully completed templating right now. And it wasn't even that complicated, right? We had a little hiccup here, but thanks to Cloud Code, it was very easily fixed. This is the type of thing that I really like having the help of chat GPT and Cloud because you know, it's really the same as Googling around.
I could have just read the documentation, obviously, but it really took like five seconds with Claude. Excellent. Now that we have this, you for yourself can decide, you know, how deep do you want to go in this? Because technically, you could allow your users to do the same for variable name, right? You could change this to be to do and then use, what would it be?
To do dot HTTP response data user ID, right or like user this, change this to get and this will be users. And then the exact same code code as up here, right? If you want to allow your users to have dynamic variable names. But yeah, you can see that then this directly conflicts our regex rules, but obviously you could modify that if you really really wanted to right you would just have to make a more loose regex and then before you assign the data variable name you would kind of create some compiled variable name, handlebars.compile, you would pass in data variable name and you would pass in the context. And then the compiled variable name would be used here.
That's if you want to do that. I personally will make the variables as safe as possible. Again, this is also if you want to fall back to an empty object or not. Great. So we are halfway, I mean more than halfway, just like one more thing left for us to be able to go to other nodes.
The reason I'm not building any other nodes now is because I want to make this HTTP request an example where we have everything. And then when we need to create new nodes I can just refer to HTTP request node where everything works everything is finished so I don't have many bugs nor any other things. So that's why it's taking me so long to complete this now. It is because I want to make sure it literally has everything and because it's a simple example, but again, complex enough that it allows us to do the templating and all those other things. And I think this templating thing is actually the most crucial part of a useful workflow project, right?
Because if this wasn't possible, it really wouldn't be that cool. I think you will agree with me on that. And I believe that is all we wanted to do. We now allow dynamic body and we allow dynamic endpoint. We can do individual strings or we can do JSON objects.
I will of course test this a few more times just in case I've missed out something big and maybe you've noticed it and you're wondering if I'm going to fix it, I promise I will test this out until the next chapter. So in the next chapter, if I notice something, I will show you how to fix it if I find it. But I think it's working pretty well. I think it might be perfect as it is. Awesome.
Let's go ahead and push this to GitHub and let's go ahead and just quickly hear what CodeRabbit thinks about this because I am quite interested. So I'm going to be 20, no templating I believe is the branch name. I'm going to go ahead and just stage all these files, not too many changes, 20 node templating. I'm going to commit and I'm going to publish the branch. Once the branch has been published, Let's go ahead and open a pull request here.
Let's wait for the review. Here we have the summary by CodeRabbit. New features. We added template support for HTTP request endpoints and request body content, enabling dynamic value substitution. Improvements.
We enhanced HTTP request validation with more descriptive error messages. We improved response handling consistency for HTTP request execution. In here we have a sequence diagram but I think it is pretty clear how it works. So we execute HTTP request with context. The context is basically what is assigned from the previous node.
That's why I explained that for the first HTTP request there will be no context but for the second one there will be context which is the result of the previous node stored in the variable that we defined. And basically we then use that context to compile the endpoint template or to compile the body. So one of those or both and then we send the HTTP request with those newly generated body and endpoint variables and we return some response and then that response again gets attached to the context. So if we had a third node, right, if I go ahead and add a third HTTP request, whoops, and connect it right here, then this one would have access to both the response of created to do variable and of the initial to do and it could do anything in once with that. Awesome.
So let's go ahead and take a look at the comments here. So the comments are mostly to safely handle the JSON parsing and to throw new errors if JSON serialization fails. So yes, we could do that. Instead of the register helper, instead of just doing it like this, we wrap it instead of try and catch. So we control exactly what error is being thrown.
I mean, it will still behave exactly the same right now. It would just be a little bit out of our control where the error is being thrown. Same thing goes for compiling the endpoint right. Actually this is something different. So in here once we template the endpoint it suggests actually checking if that endpoint still is a valid endpoint after we compiled it through handlebars.
Because yes, probably some very smart user could maybe abuse the handlebars compile method if they know how it works and if they watch this video and they know that we use it. So yes, this is some penetration protection that you could be aware of, right? Basically, after you verify your end, after you compile your endpoint with the variable, you should still check, does it exist? Is it the type of string? And is it still an endpoint you can call?
So very good catch by CodeRabbit here. But I am satisfied with this as is for tutorial purposes, so I will merge this tutorial. My apologies, I will merge this branch. I'm going to go back to the main branch and I will hit synchronize changes. Once I hit synchronize changes, I always like to open my source control, click on the graph and just convince myself that it has been merged right here.
Amazing. I believe that marks the end of this chapter. And in the next chapter, we're going to be doing the last thing in regards to this HTTP request node, which will be real-time feedback. Once we do that, We will be able to reuse that code and create a bunch of other nodes like OpenAI request, Anthropic, Gemini request and then some submissions, I mean some triggers like Webhook trigger, Google form trigger, Stripe trigger and similar. And very soon you will realize there is no limit to how many nodes we can create.
It will all depend on the ones that you want. Amazing amazing job and see you in the next chapter.