What the dashboard said
Our cameras run two models in sequence. A fast object detector watches the video and labels what it sees: person, car, dog, bird. Before any animal reaches the wildlife log or anyone's phone, a second, larger vision model looks at the snapshot and confirms what the animal is. The detector is quick and gullible; the second model is slower and is supposed to be the skeptic.
In June 2026 the wildlife log filled with birds from a single outdoor camera. I counted three days of its events.
What one camera reported in three days
Two failures, stacked
The first failure was ordinary. The detector labelled the boulder a bird, with confidence between 52 and 73 percent. Object detectors do this; it is the reason the second model exists.
The second failure was the interesting one. The skeptic was agreeing. Working with an AI coding agent, I had it pull the exact frames the second model had been given, and the cause was printed on them. The detector draws its own label and box onto each snapshot before passing it along. The checking model was receiving a picture of a rock with the words "bird: 52%" written across it, and it was reading the caption.
The checker was grading a test with the answer written on it.
The frame scored at 73 percent contained no bird of any kind, only the rock, and the second model still answered "bird". Its instructions made matters worse. They opened by telling it that it was a wildlife identifier and asked it to name the species, so it had been told in advance that there was an animal to find.
The fix, tested before it shipped
The checker got new instructions: ignore any boxes or text drawn on the image; expect that most frames contain no animal at all; answer "none" when unsure.
Before anything changed in the live system, the old and new instructions were compared on the real failing frames, through the same path production uses, against the same model:
- The 52 percent rock: bird before, none after.
- The 73 percent rock: bird before, none after.
- A frame of real geese: still identified, correctly, as Canada geese.
The results held across repeated passes. That third line mattered as much as the first two. A checker that says "none" to everything would also have fixed the rock.
Then the problem was closed at the source as well. The detector now ignores the bird label in the small area the boulder occupies, drawn so that it stops short of the ground where the geese actually stand. The mask handles this rock. The skeptical instructions handle the next rock, or shadow, somewhere else.
The one I could not fix
In the same week, a cat crossed the far edge of the camera's view and was not recorded at all. It was tempting to blame the new, stricter checker. The logs said otherwise: the detector had never fired. At that distance a cat is too few pixels to clear the confidence threshold.
The obvious remedy, lowering the threshold, would have been a mistake. The rock problem calls for stricter detection and the cat problem calls for looser detection, and one global setting cannot do both. I left the thresholds alone and recorded the cat as a limit of range that needs a closer camera, not as a setting to tune.
And one we got wrong first
Midway through, a health check reported that the model server was down. It was not. The check had been run from a machine the firewall does not admit, so it saw empty responses that the real caller never sees. The lesson went into the record with the rest: test a service from where its real caller sits.
What I took from it
- A checker must never be shown the answer it is checking.
- A model told to find animals will find animals. Instructions are part of the measurement.
- Test a fix on the real failures, through the real path, and include a case that must still pass.
- When two problems pull one setting in opposite directions, the setting is not the fix.
"Truth becomes fiction when the fiction's true; Real becomes not-real where the unreal's real."