So we've now developed the console panel which displays all steps from individual runs. But one thing we cannot use the inspector panel for is to review the replay of the entire workflow. And that is something that BrowserBase actually offers under their Observability API. Now even though we added BrowserBase skills in the chapter where we set it up, those skills actually only refer to the browser CLI usage. It doesn't actually refer to API or SDK programmatic usage of Browserbase.
So because of that, what I'd recommend is that you learn your way around browser-based documentation and go ahead and find their observability right here. So make sure that you have access to the URL of the observability and find specifically session replay as well. So make sure you can find those two documentation pages. To let our agent know exactly which documentation it should look for while we build the session replays, let's go inside of our agents.md and let's add a new block here. So I'm going to add browser base Observability.
Session recordings, replays, live view and logs come from the core BrowserBase SDK and then we name that package, not Stagehand. Before building any observability feature, consult BrowserBase's observability documentation and then I paste the overall, the overview of the observability. So that's this URL. Session replays specifically, retrieving a session's recording as an HLS playlist is documented here and then I paste that the session replay documentation. So when you copy and paste this from my agent's MD file, just make sure that the URLs haven't changed and that they don't point to 404.
And we also just give it some additional info. The retrieval needs the secret API key, so it must be proxied server side. This is also an important hint. Given that the session replay API provides us with an HLS playlist, we need to install a dependency called hls.js. So where do we start developing this feature?
Well, the answer is inside of our run workflow task. So similarly as to how we updated this long running task previously for our console panel with all the relevant information like node type, node title, status, duration in milliseconds error and the output, we now have to add one more thing to this long running task and that is the session ID every time stage hand is initialized. But here's the thing, So far we've only been adding properties to individual steps, but the session replay is the same for all steps because session replay is tied to the entire workflow, not just for individual steps. So unfortunately, yes, we cannot see a replay of every single step. We cannot just see a replay of open URL.
We cannot see a replay of only agent node, but we can see a replay of the entire thing and how it went together. So let's go ahead and construct a prompt to modify our run workflow to make that work. As always, I've prepared all the prompts that we're going to use in this chapter inside of the chapter 22. You can also find the agents.md, you can find what we have to install. So let's first capture the session ID and let's review this prompt.
A run records its steps, but not the browser session that it drove. So there's no way to replay what happened. So inside of our runworkflow.ts, capture the browser-based session ID when the run opens its session and return it alongside the steps. Then surface that session ID on each run inside of our workflows run provider. So remember that's our data flow.
It starts with the run workflow long running tasks and then the workflows run provider optimizes it or basically makes the data compatible with what we need in our UI. So a panel can read it in other words, right? It only exists once the run has finished. The recording lags on the session close. So read it from the runs final output not in the live metadata.
So this part right here is if you remember long running tasks from trigger give us an option to read the data either from the final output or from the live metadata. So we need to choose which one we're going to use depending on which one is available. But for the session replay, we have to rely on the final output because It's not going to exist during the run because it's not yet finished. It's not yet created. So that's why we explicitly write this.
Let's go ahead and run this and let's look at the changes. So the first modification you should have is inside of the run workflow file. So alongside initializing stagehand, we should now also initialize the browser-based session ID. And then once getStagehand runs and once we initialize it, all we do is we extract the browser-based session ID and we store it inside of that new variable we've created, browser-based session ID. And all we have to do in the end is return it alongside the steps.
That's it. That's everything needed modified inside of the run workflow. And inside of the workflows run provider, again, it depends how your agent is working with this provider. I'm pretty sure agents have various ways of achieving this, but as long as you can now see that your provider is accessing the browser with session ID, everything should be fine. You don't really have to worry about how it's doing that and it doesn't need to be exactly the same as mine.
So here's how mine did it. In this useConsoleRuns which it developed for me, it added a new property browser-based session ID so automatically all of my components which use this hook now have access to the browser-based session ID and it looks like it generates that using the session ID for run and in here it specifically goes through my run and then through the output. So that's that hint that we gave it, right? Don't look through the live metadata, use the final output. So the output variable in the trigger run is what's finished while the metadata is used to see if something is live, right?
That's why you can see how sometimes we look through metadata and then fall back to output if it doesn't exist. But as long as you can now see the browser-based session ID within your workflow runs provider, it means that it's been added here. Now in the same conversation, let's construct the API endpoint which is going to serve as a proxy for fetching the HLS playlist because we cannot directly just access this browser-based session replay. We need to create a proxy from our own server to achieve that. So given a browser-based session ID, play back that session's recording.
Browser-based serves the recording as an HLS playlist and fetching it needs the secret API key, so it has to be proxied server-side. Add a route at app folder, API, replays and then dynamic session ID that retrieves the sessions replay playlist through a browser-based SDK and returns it available only to the currently signed in organization. That's also why we need to create our own endpoint to make sure that no one can hijack another organization's workflow replay. That would be a very big breach. Our agents.md file, which we modified at the beginning of this chapter, points to the browser-based documentation for the retrieval API.
So that's why I told you to add that. The recording isn't ready the instant the session closes. Browser-based answers with a not ready status until it is. Have the route pass that through and build a session replay component in the workflow components folder that pulls the route until the playlist is ready and then plays it using new dependency which we've added HLS.js. So this prompt is both going to create the API endpoint and a new session replay component which we're going to use to render in the console panel.
And we also say don't wire it into the console yet. Just get playback working from a session ID. Okay, so no wiring of this component yet. I just want to see it created. Because we pointed the agent to our agents.md file, you should now see web fetches to those URLs which we've added.
You should now have a new API endpoint under API replace session ID. In here we import out from clerk, not found error from browser base SDK and here's one thing that perhaps your agent didn't do. So my agent decided to create a separate lib for browser base. And it's just a simple initialization of browser base. So I actually saw during its output, it took a look at how live blocks and resend was initialized and it decided, okay, I'm going to do the same for browser base.
So if your agent didn't do that and instead it initialized it here inline, it's honestly okay because it's actually the only place where we initialize BrowserBase like that. In our run workload task we need to initialize it in a function because it needs to be preserved through the entire task, right? So however your agent decided to add the browser base SDK in this API endpoint is correct. In here I have a small comment about what this actually does. So It's important that this is a get function.
We skip the first parameter which is the request and then we go into the actual params from which we can extract the session ID from its URL actually. Then we use clerks out to extract the user ID and organization ID and make sure to throw if it's not available. We extract the session ID from the params here and then we simply use the browser-based API to retrieve the replays. And then here's how it decided to find the exact replay we need. So it found the first page.
If it's not available, it threw. And then it found the playlist itself, I'm guessing using the first page variable from above and then passing the page ID along with the session ID. Then it turned that playlist into text which got us this constant right here and then all we do is we return that back with content type application and then this specific value inside cache control no store. So I'm assuming this is a mix of knowledge of Next.js and how it's supposed to return things and our agents.md which pointed it to the documentation page. So if your agent didn't add any headers, perhaps ask it do we need any headers for the m3u8 HLS playlist and then it will maybe add the correct headers.
But maybe it can work without headers as well and I also happen to have a catch error here and in case it's not found error we simply return this status. So that's how my agent developed the API route. Yours can of course slightly vary but as long as you can see browser-based session, replace, retrieve page and something like this happening it's probably correct. Now let's go inside of the second modification which is a new component called session replay inside of the workflow components. So in here what the first thing you should see is that it uses the import from HLS and however you're developed, the polling is okay.
So mine decided to set up a poll interval for every two seconds because the endpoint which we just developed, the API replace session ID might not immediately be available. So when the user renders the session replay what it's going to do is inside of the use effect here it's going to wait for playlist and I'm guessing this will repeat every two seconds until it reaches it and make sure that you have if you do have that kind of polling make sure that you have a proper cleanup function like this one. Great. So we have that we use the HLS library here to attach the media to the video which I'm guessing is a component like this one. So make sure that you just have some kind of component which renders the HLS here and then perhaps you did the loading status, perhaps it did the error status, perhaps it did the unsupported status, maybe it didn't, you can slightly nudge it if you want that to appear.
Perfect. So you should now have at minimum these two changes, new API endpoint and a new component and maybe even a third change if your agent decided to separate the browser based client initialization. So now how do we surface that new session replay component inside of our console component? Well my first idea was to have it available in every node that we click on. We cannot really make it only available for one node because again the replay is for the entire run not just for a single step.
Okay, so my idea is to add a dummy node at the bottom called replay. Pretend like it's a step. And it's always gonna be the last step of any workflow we create. And then once we click on that new step, which we are gonna create, which we're gonna kind of mock, instead of formatted output or formatted error, we're just going to render that session replay component, which renders the video, right? I think that's a good idea because then it is encapsulated and scoped per run in the UX isn't confusing and every user will see that there is like an additional step for every run that they do.
And since we're gonna name it replay, it's kind of gonna make sense. So let's go ahead and create a prompt for that. Once again, in the same conversation, so your agent has all the context available. Each finished run has a recording now. Let's surface it to the console.
So inside of our logs panel component, add a single replay row under each run that has a recording. Its session ID is present and the run has finished. So that is the criteria for displaying this fake step which we are developing. It sits with the step rows and is selectable the same way, but it stands for the whole run not one step. So exactly what I was just explaining.
This replay step is essentially just a good user experience so that it makes sense for them. Okay, how do I see this runs replay? Obviously it would have been easier if we had an option to see the replay of an individual step but unfortunately we don't so we have to figure out a solution like this. Selecting it should play that runs recording in the output panel. In the inspector panel.tsx component, Render the session replay for the run session ID instead of a steps output when the replay row is selected.
So exactly what we said we want. When you click on that new row, let's render that new component of ours and not the output or the error. The console currently tracks the selection as run ID plus node ID. That's the console panel. Extended, so a selection can be either a step or a runs replay with only one active at the time.
Perfect. That should wrap up this feature. So now go ahead and create a brand new workflow. Remember that every time we do some modifications in the run workflow task, since we don't really do backwards compatibility at this point, perhaps some older workflows might have some errors. So you're just gonna have to clean those up, but your new workflows shouldn't have any errors at all.
So go ahead and create something that opens the URL and perhaps maybe calls an agent. Then make sure that you have your dev server running and your Next.js server running and click on run. And now you should see a brand new step row after the usual nodes which you would expect. This is the replay row whose purpose is to load that proxy API route we've just created and then display the HLS video replay of what just happened. And then go ahead and git add, git commit and git push your changes.