In “Vibe Coding a Legacy PHP App: Web2Project Edition,” I covered the code. After 401 commits and a brand new UI, the manual and the docs site still showed a theme that no longer exists and features that had changed in major and minor ways.
The code was done. The docs were wrong.
Docs fall behind because screenshots are tedious to redo. Give an agent a running system with realistic data and most of that tedium goes away.
What Changed (and Why the Docs Broke)
If you read the release notes for 5.0.0 RC1 and RC2, the list is long. A few highlights:
- Bootstrap 5.3 is the only theme, so the per-user and system-wide theme settings are gone.
- New sign-in, sign-up, and lost-password pages.
- PDF export is gone, replaced by a print stylesheet.
- File check-in/check-out is gone.
- The Folders tab is back in Files.
- The header search box runs SmartSearch directly, so SmartSearch disappeared from the menu.
Every one of those makes some paragraph wrong. Most of them make some screenshot wrong too. And with a new theme, every single screenshot of every single page was wrong.
We have two places to fix. First, the docs site (web2project.github.io). Second, the User and Admin Manual (web2project/manual), which you can also buy as a PDF on Leanpub. They cover a lot of the same ground, so they share a lot of the same images.
Screenshots are the Expensive Part
Text changes are find and fix. Search for “PDF”, edit the paragraph, move on.
A screenshot is different. You need a running install, realistic data, a user with the right permissions, and the screen in the right state with the right tab open. Then you crop it, name it, and put it in the right folder. Then you do it again for the next one.
In an open source project, nobody volunteers to retake a hundred screenshots. So they don’t get retaken. The text gets updated (sometimes) and the pictures stay frozen in whatever year someone last had the energy. In our case, it was when I did it in 2007-2010.
When we started, the manual had 77 images, and every one of them had an identical match in the docs site. The docs site had 132. The good news is the images existed. The bad news is many of them weren’t used where they should have been. A page described a screen and the image for that screen was already in the repo but never embedded. Odds are, that was my fault.
Give the Agent a Running System
My first piece of advice in the last post was to give the agent a running system with real or sanitized data. That applies to docs as much as to code.
Web2project has a predictable structure: companies have projects, projects have tasks, tasks have owners and dates. Because of that, Claude could create plausible sample data, build out whatever a page needed, log in as the right user, and take the screenshot.
That matters more than it sounds. An empty install makes useless screenshots. A Gantt chart with no tasks shows the reader nothing. A project with real-looking tasks, owners, and dates shows the reader what the screen is FOR.
The capture itself lives in a Playwright script in the main repo. Each screenshot is one entry in a list: the output path, which user to log in as (or nobody, for the sign-in page), the URL, and an optional interaction like clicking a tab or hovering over a menu. It can capture a full page, a single panel, or a slice. It writes the image plus a 250×188 thumbnail, then copies each one into the docs site or the manual, but only where a page already references that image.
There’s an agent angle beyond the screenshots, too. Claude read the release notes and the code to work out which pages changed, instead of someone walking every menu with a checklist. The release notes were already written. Using them a second time cost nothing. In this particular case – because we changed the theming system – it was every screen but the incremental changes from here should be trivial.
It Found Screenshots We Never Added
Along the way, Claude found pages that referenced screenshots that were never added and images that only one of the two sources used. Some of the docs drift had nothing to do with 5.0. It had been broken for years and nobody noticed.
So Claude didn’t just replace images. It embedded the ones that should have been there all along, so every image gets used and the docs site and manual match each other.
This worked in part because of a decision made long ago: images are named after the path of the screen they show, and they’re structured the same way across every module. With that, Claude could figure out which page to render for a given screenshot AND where in the docs a missing one belonged. If the images had been called screenshot-final-v2.png, none of this would have worked.
Boring conventions pay off years later. Usually when you’ve forgotten you made them.
So What Should We Do?
If you maintain docs for anything with a UI, here’s what I’d do.
First, keep a seeded demo install that matches what the docs describe. Not your production data, not an empty database. Something with enough companies, projects, and users that every screen has something on it.
Next, have the agent compare the release notes against the docs. Ask it to list every page affected before it changes anything. Review that list. It’s much easier to catch a missed page in a list than in a diff.
Next, make screenshot capture repeatable. A script with one entry per image turns the next release into a rerun, not a project. For bonus points, keep it in the main repo so it changes when the UI changes.
Finally, review every screenshot before it merges. Right screen, plausible data, nothing private. The agent is fast. It’s not your editor.
For more on what good docs need beyond pictures, I covered that in “Better API Documentation in 3 Steps.”
Docs fall behind because updating them is tedious. Hand the tedious work to the agent and keep the review for yourself.
The updated manual is on Leanpub and the source is on GitHub. If you find a screen that still looks wrong, open an issue or send a pull request.
