The Principle of Organic Order: Our Journey So Far

Dear Reader,

Let's resume our quest to understand the VISTA software lifecycle by continuing our exploration of Christopher Alexander's small masterpiece, The Oregon Experiment. In this book he describes how architectural design used to be organized according to a timeless way of building called organic development and describes the problems that have accumulated in modern architecture as a result of abandoning it. Organic development is the same powerful process harnessed in the classic VISTA software lifecycle. Since the VISTA lifecycle's never been adequately described in writing, we're trying to understand it by reviewing Alexander's description of organic development and by comparing and contrasting it with VISTA's model.

Alexander describes organic development in terms of six principles, with a chapter on each. When we left off in September, we were in chapter one, which describes the Principle of Organic Order. Alexander summarizes this principle as:

Planning and construction will be guided by a process which allows the whole to emerge gradually from local acts.

To illustrate the importance of this principle, Alexander explores the nature of the modern architectural-planning process, in which planners imagine the problem of creating good architecture boils down to a struggle between chaos and order in which their job is to impose order (a vision, an aesthetic, a design) on an otherwise chaotic situation. Planners' main tool in this battle is the modern master plan, which attempts to fix in advance the more-or-less complete solution to the problems they've been hired to solve.

Alexander demonstrates that master plans themselves are deeply problematic, because in good architectural design there's a far more important quality at work than either order or chaos, what he calls organic order. Master plans by their very nature drive organic order out of a project because they're oblivious to it. As long as planners try to reduce their mission to a struggle between two choices, order and chaos, they'll remain unable to understand the damage they do with their master plans, because they'll only look for the elimination of chaos, the creation of order, and call that successful - even though the form of order they create causes worse problems than the chaos they've eliminated.

As with architecture, so with enterprise-scale software-development projects: the creation of a master plan is the first major milestone of any serious project, and the plan remains the fundamental organizing tool for everything that follows. To accuse master plans of being the root problem in modern architecture (or software engineering) is a radical act. It demands more than accusations, so Alexander supports his claim by examining the essential strengths and weaknesses of all master plans, to show that the problems they cause can't be avoided without abandoning this entire approach to planning.

In our review chapter one, we began by exploring the problem of chaos. Although chaotic situations do spontaneously generate forms of order, they also spontaneously generate forms of disorder, and the conflict between the two ensures that chaotically generated order cannot remain stable over long periods of time and also cannot create higher forms of order. For something as complex as medicine with stakes as high as life or death, master planners are right to reject chaotic order as a source of organization for software architecture. The fundamental strength of master plans is that they provide an alternative to chaotic order.

Their fundamental weakness is that in its place they impose totalitarian order, which is essentially hostile to organic order, so we next began reviewing the four unavoidable problems of master plans described by Alexander.

First, master plans are too precise about their solutions. They deal with unknowns by false prophecy, by trying to project a detailed picture of the future. They do this in part because their focus isn't on understanding the problem; it's on selling a solution, and to do that they have to paint a bold, emotionally compelling picture to win the contract. The buyers contribute to this because they're uncomfortable with the inescapable uncertainties at the beginning of any project, so they demand a false clarity to comfort them and give them enough confidence to proceed. The many details fixed in the master plan lend it persuasiveness, verisimilitude, but not truth. Later in the project those same details will tear the project apart as they collide with reality in precise and (only from the perspective of hindsight) predictable ways.

Second, master plans are too imprecise about the problems they mean to solve. As any experienced builder or programmer will tell you, only after you solve a problem do you fully understand it. The process of construction creates a flow of discoveries about the nature of the problems we face, surprises that reveal the many ways we're wrong about what we have to overcome to succeed. We know the most about the overall problem at the end, but master plans are created at the beginning, thus locking in the maximum amount of ignorance about those problems. Master plans propose solutions to problems the participants will never again understand so poorly.

Next we'll explore the third problem, alienation.

Yours truly,
Rick