Will Increasing Complexity Lead To Collapse?

Clay Shirky writes a facinating post describing the erosion of complex business models. He recounts a talk he gave to TV executives worried whether online video will generate enough revenue to cover production costs. Drawing on Joseph Tainter's The Collapse of Complex Societies, Shirky examines whether media will simplify, or instead collapse:

Early on, the marginal value of this complexity is positive—each additional bit of complexity more than pays for itself in improved output—but over time, the law of diminishing returns reduces the marginal value, until it disappears completely. At this point, any additional complexity is pure cost.

Tainter’s thesis is that when society’s elite members add one layer of bureaucracy or demand one tribute too many, they end up extracting all the value from their environment it is possible to extract and then some.

The ‘and them some’ is what causes the trouble. Complex societies collapse because, when some stress comes, those societies have become too inflexible to respond. In retrospect, this can seem mystifying. Why didn’t these societies just re-tool in less complex ways? The answer Tainter gives is the simplest one: When societies fail to respond to reduced circumstances through orderly downsizing, it isn’t because they don’t want to, it’s because they can’t.

His conclusion is probably right. But I think the argument has two problems.

First, increased complexity doesn't necessarily create drag on the system. Put differently, increased complexity sometimes costs less than the status quo. My favorite example is virtualization. The back end is indeed complex, requiring advanced and well-designed software. But virtualized servers actually use resources more efficiently. And a properly configured Citrix desktop environment is faster and easier to work in than a thick client environment. Complexity is more efficient.

Second, improvements need not add complexity. Managing from a lean perspective, I'm always looking for ways to simplify. Those who identify and eliminate waste from a system make it less complex.

In other words, the best solutions are elegant, reducing complex problems to simple yet correct answers.

So on a fundamental level, I agree with Shirky: success lies ahead for those who can add value by making things simpler. The future is poetry, not prose.