How Complex Systems Fail: 18 Propositions on Failure and Safety

The site how.complexsystems.fail republishes a short 1998 essay on failure in complex systems as a list of 18 numbered propositions. Instead of treating accidents as anomalies, the text argues they are a normal product of how such systems are built and operated, and it is still quoted in discussions of safety and reliability.

Failure as a normal property of the system

The opening propositions state that complex systems — transportation, healthcare, power generation — are intrinsically hazardous: the frequency of exposure to hazards can sometimes be changed, but the processes themselves are irreducibly dangerous. Their high consequences drive the construction of many layers of defense, both technical and human, and those defenses normally work. Catastrophe requires several small failures to line up: each is necessary, but only their combination is sufficient, and the essay notes there are far more failure opportunities than overt accidents.

Because complexity makes it impossible to run such systems without flaws, they operate in a degraded mode, carrying a changing mixture of latent failures. Accident reviews, the text adds, nearly always find a history of earlier “proto-accidents” that came close to catastrophe.

The trouble with “root cause”

The essay rejects post-accident attribution to a single root cause. If overt failure requires multiple faults, then no isolated cause exists — only jointly are the contributing causes sufficient. Reasoning built on “root cause” reflects a social need to blame a localized force or event, not a technical understanding of failure. Hindsight then corrupts the assessment of human performance: knowing the outcome makes the precursors seem more obvious than they were to the operators at the time.

Operators carry two roles at once, producing the system's output and defending it against failure. Outsiders rarely recognise this duality: in quiet periods the production role is emphasised, after an accident the defensive one. Every practitioner action is a gamble under uncertain outcomes, and the essay points out that successful outcomes are gambles too — a fact that is rarely appreciated.

Safety is created continuously

The closing propositions argue that safety is an emergent property of the whole system, not a component that can be bought or isolated, and that it is dynamic because hazard and its management keep changing. Failure-free operation results from people continuously holding the system inside tolerable performance limits, often by selecting rehearsed routines and sometimes by inventing new ones. Recognising the edge of the envelope, however, requires intimate contact with failure — which is why better safety depends on giving operators calibrated views of the hazards and of how their own actions move the system toward or away from that edge.