As an airplane nerd I have been following the recent 737 Max 8 crashes, trying to figure out my own point of view on the tragedies. As details emerged I sketched out this diagram of two interlocking and reinforcing feedback loops, one showing how investing in ever more sophisticated technology makes it more useful, and another showing how more useful technology allows us to delegate more authority to technology. However, as technology gradually gains more authority, and become harder to understand, the invisible risk of fatal failure also gradually increases.
My point here is not that technology is bad. But I do think it’s useful to be aware of feedback loops that consistently push a system in one direction. ROI is always visible. Risk is not as visible until after it goes wrong. You can apply the same thinking to self-driving cars and the Nasdaq flash crash.
With much love