21 October 2026 · 5 min read
My workflow keeps my AI honest. And me.
Wrong AI output looks just as finished as right output. Why I stopped reading everything my agents write and started asking them for proof.
21 October 2026 · 5 min read
Wrong AI output looks just as finished as right output. Why I stopped reading everything my agents write and started asking them for proof.
Back when I was overexcited and running about a hundred things at once, I had agents updating my portfolio, case studies and CV on the theory that they had the copy and wouldn't make anything up. Days after it all went live, I was reading my own case studies and thinking: how did that get in there? Wrong AI output looks just as finished as right output, so the only reliable catch is a check on what was actually produced, built into how you work instead of left to memory.
An agent writing your CV is the most generous biographer you'll ever have. It knows the document is meant to get you work, so it writes the version of you most likely to get it. Pass that through another agent or two without checking, and each one adds its own little tale until it's a whole other story. By the end, I sounded less like a sole trader and more like a mid-sized consultancy.
The example I still come back to is a line about how long I'd been doing agentic engineering. None of my files held the real answer, so an agent supplied a plausible one: specific, confident and invented. It reached two files and shaped two rounds of research before I caught it by reading the output.
I've written that a keyboard doesn't make the sentence less yours. It works the other way too: if the tool is part of how I think, a fact it invents under my name is my false claim.
The obvious fix is to read everything, and I tried. Agents are verbose, so whole days went on reading and reviewing, and honestly, who wants that? I've called speech an ancient, low-bandwidth serial cable, and reading is the same cable: one word at a time. A day of agent output doesn't fit through it.
I'm not a more careful person than anyone else, so I stopped trying to be the check. I don't read everything any more. I ask for proof, and it usually takes one of two questions.
This one came from a proposal for a real client. An API claim nobody had sourced and a hosting figure nobody had calculated both made it into the document. The API claim happened to be right. The hosting number was wrong by more than half, and the return-on-investment argument built on it collapsed.
Right by luck and right by checking look identical until someone opens the source.
So no number and no technical claim goes into a client document now without a source someone actually opened. Every claim gets a row, a mark and a cost:
Verified source opened, link given
Assumed plausible, unchecked: say what breaks if it's wrong
Unknown say who finds out, and how
cost of being wrong on every row, whatever the mark
The cost column is the one I read, because I can judge commercial consequence without being a backend engineer. It's the idea from a source of truth is whichever file something checks, applied to claims instead of tokens.
My nightly backup to Google Drive said it had finished. I asked how it could upload anything with Drive switched off, and that's all I had to ask, because Opus is always extra. It went and looked, found nothing had reached Drive, and came back with the reason, a plan and the fix already in. With the Drive app quit, macOS had left an ordinary local folder at the same path, so the backup landed next to Drive, not in it. It now fails unless the Drive app is running.
These days it's usually an agent that catches the made-up bit before I do, in its own work or another agent's. They practically snitch on each other. When I ran one question through three different setups for an experiment, all three flagged the same half-finished edit in one of my own files, without being asked. It helps that my global agent instructions say an edit that reports success isn't a verified result, and that the output is what gets checked. I don't just open Claude Code and start talking.
A status script lists the signatures of a half-done edit under CHECK BEFORE SENDING, and my CV won't turn uncommitted edits into a PDF until I've seen the diff. Both live inside the system that runs my working day. I ask, and something goes and looks.
Honest state, not the pitch version:
Built: the three marks and the cost column, on client documents and on this blog; a rule that numbers about me get checked against the record the first time they're used; global instructions that send agents to the output; a backup that fails without the Drive app; the status script; and the PDF gate.
Not yet:
If you use AI for anything that goes out under your name, you don't need my scripts. Two habits do most of the work: mark every claim with whether anyone actually checked it and what it costs if it's wrong, and ask for proof instead of reading everything. Which did you open? Where's the actual thing?
Your CV will be a bit less exciting. It'll also be true.