"The normalization of deviance" is a sociological term describing how groups of people become accustomed to ignoring safety rules and best practices, becoming plagued with (sometimes fatal) problems that no one can seem to fix.
Normalized deviance has been blamed for the Shuttle Challenger disaster, and crises in "healthcare, aviation, mechanical engineering, aerospace engineering, and civil engineering."
Dan Luu uses a paper on normalized deviance in healthcare as a jumping-off point to describe how tech shops make the same kinds of mistakes, with the same kinds of disastrous consequences, and the same inability to fix those mistakes until they pile up so high that they threaten the whole enterprise.
Tech companies are especially vulnerable to this kind of corner-cutting, because of the emphasis on growth for early stage firms. When a company is small, it has nothing to lose and everything to gain by breaking the rules to get big fast. But once it's big, the culture of crossed-fingers-and-praying instead of methodical processes causes repeated disasters that end when the company melts down or (sometimes) fixes its culture.
Luu is interested in how companies cal learn from each others' mistakes, rather than making fresh mistakes for themselves at every turn.
People don't automatically know what should be normal, and when new people are onboarded, they can just as easily learn deviant processes that have become normalized as reasonable processes.
Julia Evans described to me how this happens:
new person joins
new person: WTF WTF WTF WTF WTF
old hands: yeah we know we're concerned about it
new person: WTF WTF wTF wtf wtf w…
new person gets used to it
new person #2 joins
new person #2: WTF WTF WTF WTF
new person: yeah we know. we're concerned about it.The thing that's really insidious here is that people will really buy into the WTF idea, and they can spread it elsewhere for the duration of their career. Once, after doing some work on an open source project that's regularly broken and being told that it's normal to have a broken build, and that they were doing better than average, I ran the numbers, found that project was basically worst in class, and wrote someting about the idea that it's possible to have a build that nearly always passes with pretty much zero effort. The most common comment I got in response was, "what kind of fantasy land is this guy living in? Let's get real. We all break the build at least a few times a week". This stuff isn't rocket science, but once people get convinced that some deviation is normal, they often get really invested in the idea.
How Completely Messed Up Practices Become Normal
[Dan Luu]
(via Dan Hon)
