Your coding agent should open the browser before it says it’s done

I got tired of coding agents telling me frontend work was done when nobody had opened the frontend.

You’ve probably seen the routine. The agent changes some code, runs the formatter, the type checker and the tests, everything comes back green, and you get some version of “Implemented and verified.” Verified how, exactly? A passing test suite is useful evidence, but it isn’t the same as opening the app and using the thing that just changed.

I’ve had plenty of changes where the code made sense and the tests passed, and the UI still had the kind of problem a person spots in about three seconds. A control overlapping something else. A selected state that exists in the store but isn’t highlighted on screen. Reset working for one path and not another. A filter that updates state but leaves the old data on the page. Save looking fine until you refresh. A map layer drawn in the wrong order. A modal that technically opens but doesn’t fit on the screen.

The agent had tested its code. Nobody had tested the application. So I started making browser review part of the definition of done for anything user-facing, and a few weeks after first writing this up, that habit turned into a tool. I’ll get to that at the end.

The loop agents are missing

Coding agents got useful quickly in large part because they already have a decent feedback loop for code. They read, edit, run the formatter and tests, read the failure, and try again. Nobody has to get it right in one heroic attempt.

For frontend work the obvious missing input is the frontend itself. The loop I want goes a bit further: edit the code, run the tests, start the app, use the changed feature, look at what happened, fix whatever’s broken, and then use it again. That last step matters more than it sounds. Agents are very comfortable telling you a fix “should now work”, and I’d rather they just open the page again and find out.

Playwright MCP is a good way to give them that. I already used Playwright for E2E tests, but the MCP server is useful for a different reason: it hands the agent a browser directly. It can open localhost, read the page, find controls, click and type, check what changed and take screenshots.

That isn’t the same as asking the agent to run pnpm playwright test. The test suite tells you whether the scenarios you already wrote down still pass. Browser review asks what this particular change could have affected and goes to look. If an agent refactors some shared map-layer utilities, the tests might cover a few helpers, but from the diff it can also work out that the code runs during initial render, selection, filtering and reset, and try each of those. That’s a lot closer to how I review my own frontend work.

Setup is small. You need Node 20 or newer and an MCP-capable client:

{
  "mcpServers": {
    "playwright": {
      "command": "npx",
      "args": ["@playwright/mcp@latest"]
    }
  }
}

A quick sanity check is to ask the agent to open the Playwright TodoMVC demo, add a todo called “Browser QA” and confirm it shows up. If it manages that, it has a browser. What it does with one is the harder part.

A browser on its own isn’t enough

An agent with Playwright can still do hilariously weak QA. I’ve had reports like this:

Opened /route.
Page loaded successfully.
Browser verification complete.

My brother in Christ, you looked at the page.

The first rule I settled on is that the review has to come from the change. If it changed a filter, use the filter. If it changed reset logic, change some state first and then reset it. If it touched persistence, refresh the page. If it changed a modal, open and close the modal, and if it changed responsive layout, resize the viewport. That all sounds obvious when a person says it, but it isn’t obvious at all if your agent instructions just say “verify the app works”.

This paid off on a work PR that fixed the map cursor getting stuck as a pointer. The PR’s new unit tests passed against the old code too, so they didn’t prove the fix did anything. What did prove it was the agent opening the app and hovering across clusters, their count labels, buffers and empty map, and watching the cursor go to pointer and back exactly when it should.

The second rule is to check the end result rather than the interaction. “I clicked Reset and nothing threw” doesn’t tell you much. What you actually care about is that reset puts things back the way they were. Same with saving: the real claim is that you can change a setting, save, refresh, and it’s still there.

I also ask for different evidence depending on the question. Whether the URL changed, a button is disabled or a dialog opened is a question for the DOM and browser state, not a screenshot. Whether that dialog is clipped, whether two things overlap, or whether text wraps badly on a phone is a visual question, and that’s where screenshots earn their place. Sometimes you need both, since a modal can sit happily in the DOM while being unusable on screen.

While it’s in there, the agent may as well look at the console and, when the feature depends on an API call, the network. I only care about errors the change introduced, though. A warning that’s been on main for six months shouldn’t derail every review, and a client-only transition doesn’t need a network audit. If a save that “worked” actually got a 500, I want to hear about it.

Start the real app

I didn’t want to write “run pnpm dev” into every repository, so the procedure tells the agent to work out how the project starts. It checks the dev script in package.json first, then whatever the README documents, and only falls back to an existing Docker setup if the app can’t reasonably run otherwise.

What I really don’t want is the agent changing the app to make it easier to test. No temporary auth bypass, no fake server, no edits to .env, and no throwaway route that exists only because it couldn’t work out the real setup. The point is to exercise the software I’m actually shipping.

Where the instructions live

My first instinct was to keep adding to AGENTS.md. It already held the rules every agent in the repo should know: structure, conventions, testing philosophy, don’t commit for me, keep changes surgical, no pointless comments. I was about to add another hundred lines on starting the app, browser setup, what to click, screenshots, console errors and teardown, and at some point that file stops being instructions and turns into a small religious text.

What I landed on is treating AGENTS.md as the constitution and skills as the playbooks:

/
├── AGENTS.md
└── .agents/
    └── skills/
        ├── browser-review/
        │   └── SKILL.md
        ├── pr-review/
        │   └── SKILL.md
        └── grill-me/
            └── SKILL.md

The rule in AGENTS.md stays short and always visible:

## Browser Verification

User-visible work is not complete until the affected behavior has been
verified in the running application.

For user-visible changes, follow `.agents/skills/browser-review/SKILL.md`.

A successful build, type check, unit test suite, or E2E test suite does
not replace browser verification.

Do not commit browser-verification artifacts. Stop anything you started
for verification and clean up before hand-off.

The skill holds the procedure and only gets loaded when it’s needed. I use Codex and Claude Code, so there’s one global AGENTS.md that Claude’s CLAUDE.md imports, and one shared skills folder both of them read. Whichever agent I’m in that day, it follows the same playbook. Here’s a shortened version of the one I first wrote:

---
name: browser-review
description: Verify user-visible changes in the running application with Playwright MCP.
---

# Browser Review

## Before verification

Run `git status --short` and remember what was already there. Don't touch
pre-existing work.

Read the task and the diff. Work out the affected routes, the changed flow,
the loading, empty, error, selected and disabled states that matter,
whether anything needs to survive a refresh, and what nearby behavior
could regress.

## Start the application

Use the repository's dev script, then its documented local setup, then its
existing Docker setup. Don't invent a new startup config and don't change
app behavior to make QA easier. Record every process and container you start.

## Exercise the change

Don't stop once the page loads. Go through the whole changed flow and check
the final result. Use browser or DOM state for functional claims and
screenshots for visual ones. Report new relevant console errors and failed
requests, not old warnings.

If something is broken, fix the smallest cause, rerun the affected tests
and go through the flow again. Don't assume the fix works.

## Cleanup

Teardown is required even when verification fails. Close browsers, stop
servers and containers, and delete screenshots, traces and other artifacts,
but only the ones you created. Run `git status --short` again before
hand-off. Never run `git add`, `git commit` or `git push`.

## Report

Say which routes you tested, what you did, which states you checked, what
tests you ran and what you found and fixed. If you couldn't finish, say why.

None of that is clever. The value is that it’s the default expectation, so I’m not relying on the agent to decide by itself that it should open a browser.

The reviewer shouldn’t trust the author

The obvious weakness is that the agent deciding whether the work is done is the same one that wrote it. That’s still worth something, since I test my own code too, but it starts with a theory of how its implementation works.

So when a second agent reviews a branch, I tell it to work out the test scope for itself. My global rules now require a fresh reviewer with no conversation history: it gets the original request, the diff and the review skill, and nothing about why the author thinks it works. It decides what the code could affect and tests that, without leaning on the author’s screenshots, QA notes or green test run.

The best catch so far came from a work PR that made the E2E setup’s app URL configurable. The reviewer noticed the seeded test data still hardcoded image URLs to the old port, and running the app on a different port confirmed it. Three image requests went to the wrong port and failed, and the page quietly rendered fallback cards where the images should have been. Nothing threw. A test that checks the cards exist passes either way, because a fallback card is still a card. You only see it by looking at the page and the network tab, which is the whole argument of this post in one bug.

Clean up after yourself, and only yourself

Once agents can start servers and browsers, cleanup is part of the design. Otherwise every session leaves a layer behind: a stray node process, a Chromium window, screenshots, a Playwright trace, a temporary auth file, a Docker container.

”Clean everything” is the wrong fix, though. pkill node isn’t cleanup, it’s an agent going on a small rampage, and it’ll happily kill the dev server you started in another terminal. git clean -fd will delete an untracked file that has nothing to do with the agent. The rule is to clean up what you created and leave everything else alone, and the git status before and after is there so browser QA doesn’t quietly end up in my next commit.

I didn’t make this rule up in the abstract, either. While building the tool I’m about to describe, the agent testing it ran a cleanup command that closed every Playwright browser on my machine, not just its own. To be fair, it owned up to it in its summary.

When a skill wasn’t enough

The skill worked, in the sense that agents started actually opening the app. But what I got back was still prose. The agent would tell me it opened the page, clicked through the flow and everything looked right, and I had to take its word for it. It’s the same problem I started with, just one level up. I didn’t want a nicer sentence about the browser. I wanted the evidence: what was checked, what passed, what failed, and the screenshots and traces to back it up, in a form I could drop into a PR for whoever reviews it.

So I wrote a build plan and spent a Saturday building it, pairing with Claude Code through milestones and independent review rounds. It’s called katalepsis, the Stoic word for grasping something so clearly you can’t be wrong about it, which felt like a fair bar for a pull request.

You point it at a preview URL and it writes a report: HTML, Markdown you can paste into the PR, and a JSON manifest. It does two things. katalepsis run runs your existing Playwright tests against the preview, and Playwright’s result is final. katalepsis sorts, redacts and bundles it, and never reinterprets an assertion. katalepsis verify takes plain-English checks for behavior nobody has written a test for, and a model drives a real browser with your own Anthropic, OpenAI or OpenRouter key. There’s also an MCP server, so the agent that just made a change can run either one itself and read the report back.

A few of the decisions came straight out of the problems in this post.

Every check ends up passed, failed, blocked or not tested. If the preview is down you get one blocked run, not three failed tests that send you hunting for bugs that don’t exist. A test that only passed on retry is reported as flaky with every attempt listed.

The model in verify can click and type and read the page, but it doesn’t get to call something a pass. A check only passes if the model points to a browser assertion that katalepsis ran, that passed, and that shows something that wasn’t on the page before the model acted. Otherwise it comes back blocked. That last condition exists because one of the models I tested tried to prove a form worked by citing the label of the email field, which had been there all along. The first live runs also caught parallel tool calls stepping on each other, and a name ending up typed into the email field.

Before telling anyone about it, I ran it against two of my own projects. Conclave’s suite came back with 95 passed. A larger work project came back with 137 passed and one failure, a map zoom test expecting 13 and getting 12.37. That was a test I already had and hadn’t noticed was failing, so katalepsis didn’t find the bug so much as make it impossible to miss. Both projects also hardcoded their baseURL, which meant their tests never touched the URL I passed in. katalepsis now warns about that, because a green run against the wrong server is worse than no run.

After I posted it on LinkedIn, someone in the comments pointed out that the commit you have checked out isn’t necessarily the commit the preview was built from. They were right, so the report now records both and marks them confirmed, mismatched or unknown, and a mismatch blocks a run that would otherwise pass, since a green run against a stale preview tells you nothing about your diff.

I’m dogfooding it in the same instructions this post describes. The browser-review skill now tells my agents to run katalepsis first, and to fall back to plain browser automation for what it can’t check yet: field values, persistence, console and network, layout across viewports. The first Codex session to pick that up went looking for a “katalepsis skill” because of how I’d worded the line in AGENTS.md, didn’t find one, and fell back to its old browser review. Wrong for a silly reason, but the fallback did exactly what it was supposed to, and the fix was saying plainly that katalepsis is a CLI and browser-review is the skill.

A few days later I used it on a real work branch, a change that made GeoJSON the main download button and moved CSV and KML into a dropdown. The first attempt stopped before doing anything, because verify sends screenshots and page content to the model provider and I hadn’t said that was fine for that app. That’s the behavior I want. With seeded synthetic data and an explicit OK, the check passed: one visible GeoJSON button, and CSV and KML showing up once the dropdown opened, with the screenshots and trace in the report. The skill still keeps pages with real data on the fallback path.

Where I’ve landed

Not every task needs any of this. I don’t need Playwright to confirm a Markdown typo got fixed, and a backend refactor that can’t change what renders doesn’t need Chromium. I do want it for components, forms, navigation, dialogs, client state, maps, filtering, responsive layout, persistence and any user-visible bug fix. My rule of thumb is that if you’re making a claim about how the UI behaves at runtime, the runtime UI should be part of the evidence.

None of this replaces good tests, code review, or me looking at changes that matter. It adds the feedback loop that was missing. The agent already has the repository and the shell, and it knows what it changed. With a browser it can start answering whether the thing it built actually works, and with a report it can show me instead of telling me. If it can write the button, it can click it and show me what happened before it says it’s done.

katalepsis is MIT licensed and on GitHub. If an agent check ever gives you a result you don’t believe, I’d like to hear about it.