My README said Parker understood dysarthric speech "at least 90% of the time." I wrote that line. I believed it when I wrote it. It was wrong.
Parker is a voice assistant I built for my dad, who has Parkinson's. The whole point is that it hears him when other systems give up, so the comprehension number is the product. A round number like 90% should have made me suspicious of myself. Nobody measures a messy real-world thing and lands on a clean 90.
I hadn't lied on purpose. I wrote the README early, before the corpus eval existed, picked a number that felt safe, and never went back to reconcile it against what the harness actually reported.
The last real-audio eval told a different story. On dysarthria corpora, baseline comprehension was 58%. With the repair loop and n-best rescoring, it came up to 82%. A 24-point recovery lift, measured across 656 tests on real audio, not the flat 90 I'd rounded to somewhere in my head.
I found the gap by opening my own benchmark/reports/ the way a stranger would. Someone skeptical clicks into that folder, sees 82, scrolls up, sees me claiming 90 on the front page. Two minutes to catch me. The part that stings: Parker ships an overclaim-guard eval. I built a thing to flag inflated numbers and then inflated one on my own landing page.
The fix had nothing to do with the number. I deleted the claim the evidence didn't support and wrote down the one it did.
What I keep noticing is that the true story sells harder than the round one. "58 to 82 on real dysarthric audio" says the repair loop does real work, that I measured it, that I'll show you the tests. "90%" says trust me. A skeptic believes the first and waves off the second, and the skeptic is the reader I want.
So the rule now: if a headline number is round, I go find the eval that produced it before it ships. No eval, no number.