In Part I, a network card vanished. This one is subtler and, honestly, more maddening — because nothing ever broke. The WiFi was always there. My laptop just refused to use it properly: clinging to the far access point on a weak signal, stuttering on video calls, occasionally dropping mid-handshake. The network worked. The roaming was haunted.
The setup
I run two OpenWRT nodes — a main router and a dedicated access point at the other end of the apartment — with proper fast roaming (802.11r/k/v) and a band-steering daemon called DAWN tying them together. The idea is lovely: as you walk around, your phone and laptop hand off cleanly from one AP to the other, always landing on the strongest radio. When it works, you never think about it.
It was not working. My laptop would throw 4WAY_HANDSHAKE_TIMEOUT, churn between the 2.4 and 5 GHz bands, and sometimes sit stubbornly on the distant AP while a much stronger one sat ignored three meters away. So I went ghost-hunting. There were three.
Ghost #1 — the amnesiac
DAWN works by keeping a shared view of every radio in the mesh — who's connected where, on what signal. Steering decisions come from that view. And on nodes with long uptime, that view would, with no warning, go completely empty. Zero radios.
The daemon was still running. Still reporting healthy. It had simply forgotten the entire network — a quiet self-lobotomy mid-life. There's a self-heal that runs at boot, but the nodes hadn't rebooted in weeks, so it never fired. A process that's "up" by every metric you'd alert on, while being functionally brain-dead, is one of the more unsettling things to debug.
The scariest failures aren't crashes. A crash restarts. This thing kept running, kept passing its health check, and kept being wrong.
Ghost #2 — the phantom radio
Digging into DAWN's logs on one node, I found it crash-looping on a single line: Failed to lookup ID!. It was trying to query a radio interface over the system bus — an interface that no longer existed. A stale hostapd object sat on the bus like a phantom limb: the real interface was long gone from iw dev, but its ghost lingered, and every time DAWN reached for it, it fell over.
Exorcising it took a full reset of the radio stack rather than a polite reload:
wifi down
killall hostapd
wifi up
/etc/init.d/dawn restartThat clears the phantom and lets DAWN re-register against only the interfaces that actually exist. (It can come back after an AP reboot, which is foreshadowing.)
Ghost #3 — the port poltergeist
The third one wasn't even about roaming. DAWN's default peer-discovery uses multicast DNS, which binds UDP port 5353. That's the exact port my DNS ad-blocker uses for its own resolution. The two daemons fought over the socket, and the logs filled with thousands of sendmsg: invalid argument errors — a poltergeist knocking on the same door over and over.
The fix was to stop DAWN from using mDNS at all — switch it to plain LAN broadcast discovery and disable the multicast service:
# DAWN: use broadcast discovery instead of umdns
network_option='0'
/etc/init.d/umdns disableLesson filed away forever: two things wanting port 5353 is a classic, and it will hide inside unrelated symptoms.
The fix that mattered: give it a minder
Ghosts #2 and #3 were one-time exorcisms. But Ghost #1 — the amnesia — I could not make stop. The view just occasionally emptied itself, and I couldn't find the trigger.
So I stopped trying to cure the disease and treated the symptom instead. A tiny watchdog, on cron every five minutes, asks DAWN one question: how many radios can you currently see? If the answer is zero, it restarts the daemon.
# dawn-watchdog (cron */5 on both nodes)
# if DAWN's network view is empty -> restart it
[ "$(dawn_radio_count)" = "0" ] && /etc/init.d/dawn restartRestarting DAWN is safe over WiFi — it doesn't bounce the radios, so you don't even notice. The amnesia still happens; it just never lasts more than five minutes now. Not elegant. Completely effective.
The ghost I chose to live with
There's a fourth one I deliberately stopped chasing. The two nodes are supposed to exchange "neighbor reports" so each knows about the other's radios, including a DFS (radar-shared) channel. That sync would not hold across both nodes — "4 of 4 on both" was achievable for about thirty seconds before it flip-flopped back.
I spent real time on it before realizing: my laptop almost always associates to the AP, and the AP holds the full 4/4 view. The router being stuck at 3/4 changes nothing I can perceive. So the accepted steady state is "AP perfect, router slightly incomplete," and I walk past it every day without a twitch.
Half of senior debugging is fixing things. The other half is correctly identifying which broken thing doesn't actually matter, and having the discipline to leave it alone.
Two gotchas, free of charge
On this particular chipset (MediaTek MT7986, OpenWRT 25.12), two things will waste your evening:
wifi reloadis a trap. It can leave radios onChannel: 0— i.e. dead. Always usewifi down && wifi up(or justwifi) after a config change.bss_transitioncrashes all radios on this hostapd. You don't need it —ieee80211v='1'alone handles 802.11v transition management internally.
The takeaway
Three ghosts, three different shapes: a daemon that forgot, a daemon that crashed on a phantom, and two daemons fighting over a port. The cure for the unfixable one wasn't a cure at all — it was a watchdog that detects the symptom and resets, which on flaky embedded hardware is worth ten elegant theories. And the most senior move of the whole saga was deciding that one ghost could simply stay.
My laptop roams cleanly now. Somewhere, every few minutes, a five-line script quietly checks whether the network has forgotten itself again. Usually it hasn't. When it has, nobody notices. That's the dream, really — not a homelab with no ghosts, but one that keeps them politely contained.
Comments