The Watch That Never Slept: Victorian Lighthouse Discipline and the Cure for Modern Alert Fatigue

There is a particular cruelty in modern on-call culture that the lighthouse keepers of the nineteenth century would have found baffling. We build elaborate monitoring stacks, wire them to paging systems, and then sit back as engineers are woken at three in the morning by alerts that turn out to be nothing. We call it alert fatigue, as though the exhaustion were an unfortunate side effect rather than a design failure.

Trinity House, the corporation that has governed England's lighthouses since the sixteenth century, would have recognised the problem immediately. A keeper who cried wolf — or worse, who slept through a genuine distress flare — endangered lives. The service therefore treated signal discrimination as the core of the job, not an afterthought. Modern teams would do well to study how they managed it.

A Taxonomy Written in Light and Sound

The keepers operated within a strict, unambiguous signal vocabulary. A rocket fired with a specific pattern meant a vessel in distress; a different pattern signalled something else entirely. Fog signals, light characteristics — each lighthouse had a documented 'character', a unique flashing rhythm printed in sailing directories so that no mariner could ever mistake one beacon for another.

Contrast this with the typical monitoring dashboard at a British scale-up: dozens of alerts with names like high_cpu_warning_v2 and severity levels that everyone ignores. When everything can page you, nothing truly can. The lighthouse lesson is blunt: define a small, exhaustive taxonomy of signals, and make each one mean exactly one thing. If an alert cannot be described in a single unambiguous sentence — what it means, what to do, who acts — it does not belong in the paging path.

The Watch Rotation as an Escalation Protocol

Lighthouse staffing followed the principle that a watch must be fresh, singular and accountable. Keepers stood rotating watches, with a clear handover ritual at each change. The incoming keeper was briefed on the state of the light, the weather, and anything anomalous. There was never a question of who was responsible; the watch had exactly one owner at a time.

Modern on-call rotations, by comparison, often blur responsibility across a rota of half-distracted engineers. The fix is procedural, not technical. A proper handover — fifteen minutes, written, covering open incidents, acknowledged anomalies and known-flaky checks — converts a rota into a watch. And crucially, the lighthouse model enforced genuine rest: a keeper coming off watch slept. Teams that let off-call engineers 'just finish this ticket first' are sabotaging the very alerting system they paid to build.

Escalation Without Ambiguity

When a keeper observed something beyond their authority — a light failure, a suspicious vessel — there was no debate about the escalation path. The district superintendent was informed, records were entered in the station log, and the matter moved up the chain with timestamps. The log itself became an instrument of learning: the Northern Lighthouse Board's station records allowed the Stevensons, that great engineering dynasty, to refine their designs across generations.

Your incident escalation deserves the same rigour. Every page should have a defined next step and a defined time box. If no human action is possible within fifteen minutes, the alert is informational and should not page anyone. If an alert pages twice in a month without requiring action, it is deleted — not muted, not lowered in priority, deleted.

Building the Lighthouse into Your Stack

Three practical commitments, drawn from the keepers' example:

1. One signal, one meaning. Audit every paging alert against the question: could an on-call engineer act on this correctly at 3 a.m. with no context? If not, it is fog, not signal.
2. A written watch handover. Rotation changes are ceremonies, not Slack messages. Document state, anomalies and ownership.
3. A permanent log. Every page, acknowledged or dismissed, recorded with timestamp and outcome. Alert quality cannot be improved from memory; it can be improved from a ledger.

The lighthouse keepers never used the phrase 'observability', yet they achieved something most monitoring stacks never do: a system in which every signal deserved attention, every watcher was fresh, and every event left a record. The technology has changed beyond recognition. The discipline, it turns out, is entirely transferable.