An Agent's Self-Report Is Not Evidence
A sub-agent handed me a figure for one property in a CRM: what share of records had it populated. It was one of four numbers in a catalogue of how well the data was filled in — the sort of unglamorous audit you run so that nobody later builds a report, a segment, or a workflow on top of fields that are mostly empty. The number was carried out to two decimal places and delivered without hedging — the kind of precision that reads as measured rather than estimated.
Before it went into the document it went through a check, which in this case was a row count against the live data. The real figure was less than half of what I’d been handed.
The instructive part is what didn’t happen. The agent hadn’t invented the number out of nothing, hadn’t refused and covered for it, hadn’t padded a gap with something plausible. It had sampled real records and reported what it found in them. The other three numbers in the same catalogue held up under the same treatment, landing within a few points of what independent verification returned. The samples were a couple of hundred records each against a database holding tens of thousands, so a few points of drift is ordinary sampling noise and I’d expect it. One number out of four was badly wrong, and nothing about the way it was presented distinguished it from the three that weren’t.
An agent’s self-report on a data-analysis task is not evidence. It’s a claim. It arrives in the same register whether it’s right or wrong, and the register is confident because fluent confident prose is what these systems produce, not because the underlying work was sound.
The check is cheaper than the report
The system I run against my own work now has a standing step for this, and it’s deliberately boring. Agent output that’s going to be relayed anywhere — into a document, into a record, to another person, to another agent — passes a verify-before-relay step first. The step logs what method was used to check and what came back. That’s the whole mechanism. It’s what caught the bad number above.
The methods are the cheapest thing available that’s independent of the original work. A row count. A grep. Opening the file and looking at it. Asking a different tool the same question in a different way. None of this is sophisticated and none of it needs to be, because the point isn’t to redo the analysis to a higher standard — it’s to get a second, independent read on the same fact. If the two agree, the number moves on. If they disagree, that’s what gets resolved before anything is built on either.
What makes this affordable is that a spot-check doesn’t have to reproduce the work. A row count answers one narrow question in seconds, and one narrow question was enough to expose a number that was twice what it should have been.
The trap is checking with the same thing that produced the claim. Asking the agent that gave you a number whether it’s confident in that number produces a second self-report rather than a check, and the two are correlated because they come from the same run and the same reading of the same data. The check has to come from somewhere else. That’s the only real requirement it has.
Agents are usually honest about what they couldn’t do
This is where I’d push back on the way the failure mode gets talked about. The popular version is that AI makes things up, which leads to a mental model of the agent as an unreliable narrator with a motive. That isn’t what I see in practice, and the nuance underneath it changes what you should build.
Two examples from my own work, both of them cases where the agent behaved well.
The first: I needed a name attached to a deal that had stalled, something mentioned in a meeting roughly three weeks earlier and never written down properly. I put two independent background search agents on it, running a cheap model, and let them grep exhaustively through the CRM records. Both came back with nothing found. Not a plausible-sounding candidate, not a “this is probably it,” not a name assembled out of the shape of the surrounding data — a clear statement that the search space had been covered and the thing wasn’t in it.
They were right, and the reason they were right is the part that stayed with me. The answer wasn’t findable by search at all. My own note had mislabelled the issue — filed under the wrong problem category entirely — so the search was correctly and thoroughly hunting for something that had never existed under that description. The agents were honest about the limits of what they had searched. That honesty gave me no protection whatsoever against a wrong premise sitting upstream of the search, because nothing about a well-executed search can tell you the question was wrong.
The second: a personal automation setup of mine has several scheduled jobs in it, and one of them silently stopped working. The schedule itself was registered correctly — everything about the configuration said this job runs at this time, and by every visible indication it was set up properly. It simply wasn’t firing.
Working out that it wasn’t firing took more care than I expected. The first log search came back empty, and an empty result from a narrow search is not proof of absence — it’s proof that one specific pattern didn’t match. The search had to be broadened before the negative result was worth believing, and only then did the absence become evidence rather than an artefact of how the question had been asked. That’s a habit I’d now apply to any “not found” that matters: before accepting nothing, broaden the search once and see whether nothing survives.
The root cause turned out to need elevated system access the agent didn’t have. It couldn’t reach the layer where the answer lived, and it said so. What it handed back was a clean split — here is what I established, here is the evidence for it, here is the part I cannot reach and what it would take to reach it. It didn’t invent a diagnosis to round the task off. It didn’t apply a speculative fix and report the problem as resolved, which would have left me with a job that looked handled and wasn’t.
I’d call that a complete piece of work rather than an incomplete one. An agent that draws a sharp line between what it proved and what it couldn’t is giving you the most useful thing it has, which is an accurate map of where its own output stops being reliable. The difficulty is that this behaviour and the bad-number behaviour come out of the same systems, often in the same session, and from the outside they look identical.
The thing that goes wrong is confidence, not character
Put the three episodes side by side and the pattern is consistent. In the search case the agent reported its boundary accurately and I was still wrong, because my premise was wrong. In the scheduled-job case the agent reported its boundary accurately and the boundary was the answer. In the catalogue case there was no reported boundary at all — a sampled estimate came back dressed as a measurement, and the gap between those two things was never stated by anyone, including me, until a row count stated it.
The problem isn’t that agents lie. It’s that a self-report carries a confidence level it hasn’t earned, and nobody in the chain is obliged to supply the missing caveat. The agent doesn’t know its sample was unrepresentative. You don’t know either, because all you received was the number. And the number reads exactly like a fact.
This compounds with the failure I wrote about in the model will forget what you told it, but it isn’t the same failure and it wants a different remedy. Instruction decay is about a constraint quietly ceasing to apply mid-task. This is about a result that was produced correctly, by the agent’s own lights, being treated as verified when it was only reported. One is a gap between what you asked for and what happened. The other is a gap between what happened and what you were told happened. Restating your constraints doesn’t help with the second one at all.
What I do with a number now
When something comes back from an agent, the question I ask isn’t whether it looks right. Looking right is what these systems are good at, which makes it a weak signal. The question is whether anything other than the agent has confirmed it, and if the answer is no, it’s a draft figure regardless of how many decimal places it carries.
Then one check, from a different direction, before it moves. If the number is going into a document someone will read next month, or into a record another process will consume, or into a summary another agent will treat as settled input, it gets the row count first. If it’s going nowhere in particular, it can stay unverified as long as it’s labelled that way.
Negative results get the extra step. A “not found” is only as strong as the breadth of the search behind it, so it gets broadened once before I act on the absence. And when a check can’t be run at all — when the answer sits behind access nobody in the chain has, as with the scheduled job — the correct output is the split, written down as a split. What I could establish. What I couldn’t. What it would take.
Where this stops being worth it
Not everything has a cheap independent check, and pretending otherwise produces theatre rather than verification. Some claims can only be confirmed by redoing the work, and if the work was expensive, that’s a real decision about cost rather than a discipline you apply by reflex. When that’s the case I’d rather mark the claim as unverified and say what confirming it would take than run something that’s really the same analysis a second time. A check that isn’t independent costs time without telling me anything new.
The other limit is distance. Output that stays in front of you while you read it doesn’t need ceremony around it, because the problem described here needs an interval to live in. It shows up once a number leaves your attention: written down, relayed onward, or consumed by something downstream with no way to notice it’s wrong. The further a figure travels from the one person who could have caught it, the more that single cheap check at the start is worth, and it costs about the same wherever it’s applied.
If you’re building on agent output right now, the audit I’d run first is to go back through what’s already been relayed and mark which of it has anything standing behind it besides the agent’s own account. Whatever comes back unmarked is a draft, whatever it looked like when it arrived.