In Part I, every health check was green and the thing was genuinely broken. This one is worse, and it happened the same evening. Every health check was green, the thing was switched off, and my house had been quietly telling me it was fine for six weeks. Not because anything lied — because nobody had ever asked the question.
The thing I was checking
The setup is unremarkable. If the camera sees a person while nobody is home, three things happen: it records a short clip to the NAS, saves a still, and pushes an alert to my phone. It also writes the time of the sighting into a small note that my daily house briefing reads out, so that when I get home I hear "the camera saw someone at 14:32 while you were out".
I hadn't touched it in a while. I went to verify it — not because anything seemed wrong, but because I'd written down weeks earlier that it was unverified, and unverified is a debt.
It had been off since the third of July
The automation's state was off. Its last run was 3 July. It was mid-August.
Six weeks. During which: the camera was up. The person-detection sensor was live and firing. The other camera automation — the one from Part I, which writes the descriptions — was running perfectly normally. My service monitor showed 32 of 32 checks passing. Nothing, anywhere, in any dashboard or alert channel, indicated that a security feature had simply stopped existing.
The part that actually bothers me
The alert not firing is bad. The quiet consequence is worse.
My house briefing builds its camera line from that small note of sighting times. If the note is empty, it says: "Security camera: all quiet, no activity while away."
But the only thing that ever writes to that note is the automation that was switched off. So for six weeks the briefing had been confidently reporting all quiet — not because it was quiet, but because the pen had been taken away from the only thing that could write. And "no sightings recorded" and "nothing happened" produce byte-identical output.
An empty log is indistinguishable from a quiet week. If absence is your evidence of safety, you have no evidence at all.
I don't think anything happened during those six weeks. That's not the point. The point is that I had no way of knowing, and my house was reassuring me daily in a confident voice.
The tell wasn't in software
Nothing in the software stack was going to reveal this, because from the software's point of view nothing was wrong. What gave it away was a folder on the NAS.
The clips directory held exactly two files. One still image from the afternoon of 3 July. And one video file with a .tmp extension — a recording that had started and never finished, from that same afternoon, still sitting there six weeks later like a half-eaten meal.
That directory should have grown, occasionally, over six weeks. It hadn't grown at all. Nobody was watching it, because who writes a monitor for "does this folder get bigger sometimes?"
A directory that stops growing is a monitor nobody built.
Verifying it properly, in both directions
Switching it back on isn't verification. So I tested it two ways, because each one alone can pass while the thing is broken.
Does the gate correctly refuse? The alert is supposed to fire only when everyone is out. At that moment I was home and my partner was elsewhere — which is the one arrangement that can actually distinguish "everyone must be away" from a subtly different reading of the same rule, "at least one person is away". Those two behave identically when both of us are home and identically when both are out. Only the split case tells them apart. I fired the automation with its conditions evaluated: it correctly declined to run. Good — the rule means what I thought it meant.
Do the actions actually work? Then I fired it again with the conditions bypassed, and checked the result of every step rather than the return code of any of them: a finished video file on the NAS (playable, correct length, and crucially not a .tmp), a refreshed still image, the alert present in my notification server's database, and the sighting time written into the note.
Then I cleared that test entry, so my briefing wouldn't cheerfully narrate a sighting that never happened. A fake event left in a real log is its own small future bug.
The bit that's easy to miss
While I was there, I looked at every other automation sitting in the off state. There were two, and both turned out to be fine — because both are declared off in configuration. They start disabled every time the system loads, deliberately, and their configuration says so in writing.
The camera one declared nothing. It had simply been switched off at some point, by a hand, and nothing recorded that.
That turns out to be a clean rule, and it's the most portable thing I got out of the evening: a thing that is off and says so is documentation; a thing that is off and says nothing is a question. Intent that lives only in someone's memory of a Tuesday in July isn't intent, it's an accident that hasn't been noticed yet.
Two failures, one shape
Part I was a system that was running and useless. Part II was a system that wasn't running at all. Completely different faults — and in both cases every instrument I owned reported green, honestly, because every instrument was answering a question adjacent to the one that mattered.
"Is the process alive?" is not "is it fast enough to be useful?". "Are all checks passing?" is not "is every feature I rely on still turned on?". Both gaps are invisible until something makes you look, and the thing that makes you look is almost never a dashboard.
The practical takeaway I've actually adopted: for anything that's supposed to produce artifacts — clips, log lines, backup files, notifications — the artifact is the health check. Not the service, not the endpoint, not the exit code. Did the thing that was supposed to appear, appear? That question has no green-light failure mode, because there's nothing to be green about. There's just a folder, and it either grew or it didn't.
Comments