Skip to main content
Record your actions in the browser and let your agent turn them into automation code.
Some workflows are too complex to describe in words alone, especially in healthcare software and legacy web apps where the interaction sequence matters and the UI has unusual patterns. Interactive script building lets you perform the workflow yourself while Libretto records every click, fill, and navigation. Your agent then reads the action log and translates what you did into a reusable automation script. This approach combines your knowledge of the application (you drive the browser and show the agent the workflow) with the agent’s ability to figure out the right selectors and write reliable Playwright code.

​
Steps

1

Prompt your agent

Describe the site, the workflow you’re going to demonstrate, and the input/output shape you want. Mention that you’ll show the steps manually.
Example prompt
Use the Libretto skill. I'm going to show you a workflow in eClinicalWorks to look up a patient's primary insurance ID. Turn it into a script that takes patient name and DOB as input and returns the insurance ID. URL is https://your-ehr.example.com
Include “Use the Libretto skill” so your coding agent uses Libretto’s CLI tools instead of writing a Playwright script from scratch.
2

Perform the workflow in the browser

The agent opens a headed browser window on your screen. If the site requires login, do that first. Then walk through the workflow exactly as you normally would: search for a patient, open their record, navigate to the insurance tab, whatever the task requires.Libretto records every click, form fill, and navigation in the background. You don’t need to do anything special, just use the app normally.When you’re done, tell the agent what you did and where the result ended up:
I’m done. I searched by patient name, opened their chart, and clicked the Insurance tab. The primary insurance ID is visible in the Patient Info section on screen.
Call out anything non-obvious about what you did:
I clicked the second search result because the first row is always a header. Also, the insurance tab sometimes takes a few seconds to load.
Take your time. The recording captures everything, so there’s no penalty for pausing to think or scrolling around. If you make a mistake, just undo it naturally. The agent can filter out dead ends.
3

Wait for the agent to recreate your workflow

The agent reads the action log that Libretto recorded: every click, fill, and navigation along with the selectors and page context for each one. You’ll see it run commands like:
tail -n 20 .libretto/sessions/interactive-flow/actions.jsonl | jq .
It uses this log as a blueprint to write a TypeScript workflow file that replicates your steps programmatically. The agent may also snapshot the page or test selectors against the live DOM to make sure everything works.If the agent asks clarifying questions about what you did, answer them. Usually the action log plus your summary is enough.
4

Review the generated workflow file

The agent writes a TypeScript file that exports a workflow(). Things to check:
  • Output type matches what you asked for
  • Selectors look reasonable (class names, data attributes, or ARIA roles, not brittle positional selectors)
  • Parameterization: values you typed during the demo should be replaced with input parameters, not hardcoded
  • Wait logic: if you mentioned something takes time to load, make sure there’s a waitForSelector or similar
If something’s off, tell the agent:
The patient name is hardcoded. It should come from the input parameter. Also, the insurance tab selector looks fragile, can you use the tab’s text instead?
5

Check the validation run

The agent runs the workflow headless with test parameters and shows you the output. Check that the returned data looks right.
# What the agent runs
npx libretto run ./get-insurance-id.ts --headless \
  --params '{"patientName": "Jane Smith", "dob": "1985-04-12"}'
If the output is wrong, tell the agent what’s off:
It returned an empty string for the insurance ID. I think the tab content hadn’t loaded yet when it tried to read the value.
The agent will re-inspect the page and fix the issue.

​
When to use this approach

  • The workflow involves many steps and is hard to describe concisely
  • You’re working with EHR, healthcare, or legacy enterprise software with unusual UI patterns
  • The workflow involves conditional paths that are easier to show than explain

​
Tips

After you log in, tell the agent to save the session with npx libretto save. Future runs can restore your login state so you don’t have to authenticate every time.

One-shot workflow generation

Let the agent explore a site and build the workflow without your involvement.

Debugging workflows

Reproduce failures, inspect live page state, and fix broken automations.