Fatal Abstraction

Software was supposed to radically improve society. Outdated mechanical systems would be easily replaced; programmes like PowerPoint would make information flow more freely; social media platforms like Facebook would bring people together; and generative AI would solve the world's greatest ills. Yet in practice, few of the systems we looked to with such high hopes have lived up to their fundamental mandate. In fact, in too many cases they've made things worse, exposing us to immense risk at the societal and individual levels. How did we get to this point?

In Fatal Abstraction, Darryl Campbell shows that the problem is "managerial software": programmes created and overseen not by engineers but by professional managers with only the most superficial knowledge of technology itself. The managerial ethos dominates the modern tech industry, from its globe-spanning giants all the way down to its trendy startups. It demands that corporate leaders should be specialists in business rather than experts in their company's field; that they manage their companies exclusively through the abstractions of finance; and that profit margins must take priority over developing a quality product that is safe for the consumer and beneficial for society.

These corporations rush the development process and package cheap, unproven, potentially dangerous software inside sleek and shiny new devices. As Campbell demonstrates, the problem with software is distinct from that of other consumer products, because of how quickly it can scale to the dimensions of the world itself, and because its inner workings resist the efforts of many professional managers to understand it with their limited technical background.

A former tech worker himself, Campbell shows how managerial software fails, and when it does what sorts of disastrous consequences ensue, from the Boeing 737 MAX crashes to a deadly self-driving car to PowerPoint propaganda, and beyond. Yet just because the tech industry is currently breaking its core promise does not mean the industry cannot change, or that the risks posed by managerial software should necessarily persist into the future.

Campbell argues that the solution is tech workers with actual expertise establishing industry-wide principles of ethics and safety that corporations would be forced to follow. Fatal Abstraction is a stirring rebuke of the tech industry's current managerial excesses, and also a hopeful glimpse of what a world shaped by good software can offer.