How to Verify AI-Implemented UIs with Playwright
What you’ll learn
- The different roles of structural and URL validation versus Playwright browser checks
- Which layout, interaction, and display states Playwright should verify
- Where browser checks fit in an existing verification flow and which judgments remain human
Use Playwright for Final Browser Rendering Checks
An AI-assisted web change is not fully checked until the rendered page and its interactions have been inspected after structural validation. Playwright automates browser actions and rendering checks, but it does not decide whether the content is correct or ready to publish. Separating structural checks, browser verification, and human judgment gives each layer a clear responsibility.
By the end of this article, you will have practical criteria for answering “Where should Playwright and human judgment sit after structural checks?” in your own context.
Playwright complements the final browser check
On this site, I do not move every quality check into browser tests. I first reduce issues that can be detected from Markdown, configuration files, and repository scripts. After that, I use Playwright as a way to complement the checks that require an actual browser-rendered page.
Playwright Test is an end-to-end testing framework for modern web applications. It can run tests against Chromium, WebKit, and Firefox in local environments or CI (the automated test and validation layer, often run through tools such as GitHub Actions).[1] However, this site does not currently use Playwright as a permanent E2E test suite. I treat it first as a browser verification layer between existing validation scripts and human visual review.
Run Structural, Link, and Configuration Checks Before Playwright
For this site, browser verification comes after checks that can be decided from the repository.
The main targets are:
- Markdown structure
- consistency between body citation numbers and reference sections
- reference URL format
- internal link format
- structural differences between Japanese and English article pairs
- consistency across harness (the set of rules, procedures, and validation that guides AI work in a project) and agent configuration
For example, npm run review:content checks citation and reference structure in article bodies. npm run review:references:live actually accesses external URLs and checks for broken or unreachable links. npm run harness:check checks shared rules, runtime adapters, Japanese-English structure differences, and related repository invariants.
The important point is to catch text-level issues before relying on Playwright. A mismatched citation number, an unlinked reference entry, or an article mentioning an npm script that does not exist can be detected without opening a browser.
Use Playwright to Check Rendering, Interaction, and Responsive States
Playwright is useful for the state that exists after rendering.
On this site, good candidates are:
- whether the home page, blog articles, and docs articles render
- whether Japanese and English internal links navigate as expected
- whether navigation and body text still work at mobile widths
- whether reference entries are visible as clickable links on the page
- whether Starlight sidebars, breadcrumbs, and heading links appear as intended
- whether console errors appear
These are difficult to judge by reading source files alone. Problems can appear only after Astro rendering, Starlight behavior, theme CSS, generated content, and published URL behavior come together on the actual page.
Playwright MCP fits agent-assisted checks
There are two different ways to use Playwright here: writing regular Playwright tests, and letting an AI agent operate the browser through Playwright MCP.
Playwright MCP provides browser automation capabilities through an MCP server using Playwright. An LLM can read accessibility snapshots and use element references to click, type, and verify page state.[2]
For this site, I separate the use cases like this.
| Usage | Best fit |
|---|---|
| Regular Playwright tests | Display, link, and regression checks that should run the same way every time |
| Playwright MCP | Exploratory checks where Codex or Claude opens local pages and inspects browser state |
| Human visual review | Design judgment, readability, and display checks that require interpretation |
I think the practical approach is to use Playwright MCP first for exploratory checks, then promote recurring issues into regular Playwright tests or existing validation scripts.
Place Playwright After Structural Checks for Final Rendering Verification
If I use Playwright on this site, the natural place is at the end of the existing verification flow.
First, I run checks that can be decided from the repository.
npm run harness:check
npm run review:references:liveThen I start the local preview and verify the rendered pages in a browser.
npm run devAt that point, Playwright or Playwright MCP can open representative pages. Instead of trying to cover every page from the beginning, it is more realistic to start with pages where breakage has a large impact.
/ja//en//ja/blog//en/blog/- newly created or updated blog articles
- docs articles whose structure changed recently
This order keeps browser verification focused on issues that only appear after rendering.
Do Not Delegate Publication Judgment, Content Accuracy, or Secrets to Playwright
Playwright is useful for checking whether elements appear, links can be clicked, and console errors are present. It does not determine whether an article’s claim is correctly supported by its references.
For example, even if a reference link is clickable and returns a normal HTTP status, that does not prove the linked source supports the sentence in the article. This is also not fully solved by npm run review:content or npm run review:references:live. A human still needs to compare the article text with the cited source.
So I separate the roles.
| Check target | Main owner |
|---|---|
| Markdown structure, citation numbers, reference format | Validation scripts |
| External URL reachability | npm run review:references:live |
| Rendering, link clicks, mobile widths, console errors | Playwright or Playwright MCP |
| Whether a source supports the claim, and whether the prose is appropriate | Human review |
Summary: Place Playwright After Structural Checks and Before Human Judgment
For this site, Playwright does not replace every quality check. It complements structural checks, URL verification, and repository consistency checks by covering the display, interaction, and layout issues that only appear in a browser.
When articles or configuration are updated with AI assistance, file-level correctness and rendered-page correctness need to be checked separately. Playwright is useful as a verification layer between those two states. Repeated checks can become regular Playwright tests, exploratory checks can stay in Playwright MCP, and judgment-heavy checks should remain part of human review.
The first action is to select one important page and reproduce its rendering, interaction, and mobile-width conditions. Browser automation cannot decide content accuracy, secret handling, or publication readiness, so those decisions remain with people.
References
- Microsoft, Installation | Playwright, Playwright Docs
- Microsoft, Playwright MCP, Playwright Docs