What Does Fluxus Quo Do?

Perito Moreno Glacier Patagonia Argentina. Photo taken by Luca Galuzzi (http://www.galuzzi.it).

Dear Reader,

Fluxus quo acts upon our world through two kinds of dynamism: flow that results in motion, and flow that results in stillness.

First, everything is actually moving and changing even when it appears not to be. The appearance of the world is stability interrupted by intervals of change—punctuated equilibrium, as Steven Jay Gould called it—but the deeper truth of the world is flow. Look up from your computer screen and look about your room. Not a single thing you can lay your eyes upon is genuinely static. The glass is a liquid, ever so slowly oozing its way toward the bottom of the window, which is why old window panes are rippled. The paint, the wood, the fabric, all of these organic materials are slowly converting their volatile organic components via chemical reactions into gas, which is evaporating into the air, some of which you're inhaling. Every scrap of metal is like the glass, congealed into solid metal only because of how cold it is, but someday, sooner or later—a thousand years, a million, a billion—the metal will be hot again and will resume its liquid flow, and until then electrons readily flow within many metals creating electrical currents. Certainly all living things you can see are always engaged in both internal flow and many cyclic flows of interactions with the greater world around them. Seven years from now, some of their physical makeup will be the same—lead or mercury poisoning unfortunately—but most of their cells will have been replaced with new ones.

Stasis, things holding still or being what they are and only what they are, is partly an optical illusion, like a single frame of a motion picture, a moment caught in a strobe light. Seen for a brief enough instant of time, even a river appears to be a complex frozen ripple, in a photograph or the blink of an eye. Seen for a long enough period of time, the misunderstanding that a glacier is merely an enormous frozen river breaks down and the river of ice it truly is can be seen to flow, which it is always doing regardless of our limited perception of it. As Homer wrote, all human beings are ephemeroi, creatures of a season, like leaves on a tree. Our lifespans are far too short to see any but the fastest flows; the apparent stillness of so many things that our flashes of life illuminate are optical illusions, a trick of mortal perspective.

Heraclitus expressed this form of dynamism in several ways:

One can grasp no mortal substance in a stable condition.

It scatters and gathers, it forms and dissolves, approaches and departs.

Everything flows and nothing abides.

You cannot step twice in the same river.

Second, the pace of change does actually accelerate or slow in response to another layer of flow under the physical surfaces of the cosmos. Physical forces and organizational principles of the cosmos conflict with one another, subvert one another, tangle and release, and otherwise interact in complex, shifting ways so that the pace of change for any given thing is sometimes rapid, sometimes so slow as to create near stillness. When things are briefly still, it is because the forces within them are balanced enough to convert their dynamism into internal stresses instead of external motion.

Like a taut bowstring, these kinds of situations are pregnant with energies waiting to be unleashed. To a literal-minded person, a bow held undrawn and a bow held drawn are both just things, objects, static, but to one with eyes to see, the drawn bow is a wound-up explosion of change about to happen. Viewing the world in terms of the objects or things leaves you perpetually surprised by earthquakes, tsunamis, wars, terrorist attacks, and other kinds of sudden change, but truly it is during the apparent still periods preceding those disasters that the problems were created, that the invisible energies tangled up and pulled one another into tension. Part of the secret to the cosmos is that the visible things we cherish or fear are merely the side effects of the movements and conflicts of the deeper principles that underlie everything. That is, as important as it is to come to understand the flow of the things that make up the visible cosmos, it is the invisible flow of the principles and forces of the cosmos that actually produces everything we take for real.

Heraclitus wrote about this form of dynamism too, as an endless war that creates and sustains the harmony of the cosmos:

It should be understood that war is the common condition, that strife is justice, and that all things come to pass through the compulsion of strife.

Homer was wrong in saying, "Would that strife might perish from amongst gods and men." For if that were to occur, then all things would cease to exist.

It is in changing that things find repose.

Opposition brings concord. Out of opposition comes the fairest harmony.

People do not understand how that which is at variance with itself agrees with itself. There is a harmony in the bending back, as in the case of the bow and the lyre.

The hidden harmony is best.

Fluxus quo cannot be stopped to create stability, nor can it be ignored since it permeates and shapes all things, yet these are the two strategies most people try. Nevertheless, if we want to produce anything lasting in this world we cannot just go with the flow, or else the first form of dynamism created by fluxus quo will simply scatter our efforts in the river of time, but the second kind of dynamism it creates can and does paradoxically produce a dynamic kind of stability, what Heraclitus calls the harmony in the bending back.

This is the secret to harnessing fluxus quo, to achieving success with something as complex as VISTA. If we try to stabilize the things in the VISTA community that we want, our efforts will be swept away, but we can instead bend back the underlying forces that produce and organize those things so they conflict with each other to produce a harmony, a dynamic, self-reinforcing stability. In such a harmonic configuration, the flow of fluxus quo strengthens the stability instead of breaking it up.

A bow may be dynamically stabilized with two primary counter-reacting forces, from the wood and the string—with additional forces introduced when it is drawn, of course—but for something as complex as VISTA we need a more complex harmony between more forces. That would seem to reduce the odds we can keep them stabilized, but another field of study reveals that even complex harmonies can be achieved using a specialized method of bending back the forces upon one another.

The field is biology. The method is homeostasis.

Yours truly,
Rick

What is Fluxus Quo?

Supposed bust of Heraclitus from the Villa dei papiri in Herculaneum (bronze, Roman), Naples National Archaeological Museum

Dear Reader,

Just as Parmenides and Plato were the preeminent Hellenic philosophers advocating a static understanding of the cosmos, so Heraclitus was the preeminent Hellenic philosopher advocating a dynamic understanding of the cosmos. Although dismissed by the ignorant as a philosopher asserting that all things were made up of the element fire, Heraclitus actually strove to use the metaphors of fire, water, and many other things to try to capture the idea of the cosmos as a domain of radical transformation and flow, in which not only does nothing stay what it is eternally but also nothing is what it appears when you look below the surface, that everything that seems even briefly static is only kept that way temporarily through the intense dynamism of shifting, contesting cosmic forces.

Twenty-five hundred years of progress have not made any easier the difficulty Heraclitus experienced, the struggle to help people see past the sometimes static surfaces of things to understand the seething, roiling storm underneath. Even our very language works against us, as we find comfortable and therefore readily adopt terms like status quo that reinforce our attachment to the illusion of stasis while resisting terms like fluxus quo that might be the keys to unlocking our ability to see things as they truly are.

My first exposure to this term was in the summer of 1999 when I read a philosophical column in a magazine. Here was the crucial paragraph for me, a concise statement that captures the Hellenic view of the cosmos as a realm of change:

Nature is understood in at least two profound senses, becoming and intrinsic validity, which to the Greeks are equivocally the same. The first sense of nature, as physis—"becoming," "growing," the gerundive or process-form of the verb phuo—describes the domain of relentless, tidal mutation: nature is the realm of all things generated and perishable where nothing can remain simply what it is (we have our word "nature" out of Latin as Cicero's invention, by analogy with physis, from the past participle of the verb nascere, natus = having been born). All natural existence is pregnant with its other, incubating cryptic forms of future order and orientation which are presently unthinkable: to exist in nature is to be variable and subvertible—all that is natural changes, falls prey to the fate of alliosis or "othering." All of natural existence is thus in motion, on the way from one state into another: Heraclitus' incisive dictum Panta rhei—All things flow—captures for all time the quintessence of ancient dynamism: the world as "fluxus quo."

——from "End Times: Millennia in Microcosm, Ancient Civilization Part 1: Nature as Becoming and as Intrinsic Truth" (Kenneth Smith, originally written 15 February 1997, published in The Comics Journal #215, August 1999)

Like most reasonably educated people, I knew the term flux, certainly in two senses from the Oxford Dictionary of the English Language—(1) the action of flowing, and (2) a continuous succession of changes of condition, composition, or substance—but when I read Smith's definition of fluxus quo above it was a revelation.

Here, I thought, was a term we badly needed in English, a term that in a nutshell captured an essential but hidden truth about the cosmos, a term that was especially vital for understanding the VISTA software lifecycle. Eventually I came to understand that it was not just VISTA but all medical informatics that were driven by the principle described by this term. The practice of medicine continually changes and flows as our understanding of it improves, so medical software, too, has to continually change and flow.

Where status quo is best understood as things being the way they are because they are standing still, fluxus quo describes things as being the way they are because they are changing.

The best way, though, to understand this or any other cosmic principle is not through a description of it, since principles are not things, but through a description of what it does, how it acts, how it changes the world, because principles are agents of change that generate patterns of flow in the cosmos.

So what about fluxus quo, the principle underlying all other principles? How does it work?

Yours truly,
Rick

What is Status Quo?

Dear Reader,

The philosopher Plato did two-thousand-plus years of damage with his attractive but ultimately false view of the cosmos, and medical informatics suffers from his philosophy even today.

Plato was not comfortable with changes. He felt that a cosmos in eternal flux could not be comprehended. He wanted a world of fixed, stable, enduring ideas that could be used as a frame of reference, so he postulated the idea that change is an illusion, a mere flickering of shadows on the walls of the human cave, and the truth is fixed, eternal, unchanging—indeed, that the true cosmos is made of eternal unchanging ideas that are casting these shadows.

It's all very pretty and vivid but utterly false. The man was more poet than philosopher, in the end, and all his philosophies suffer from holding more beauty than truth (apologies to Mr. Keats).

Plato was not the first to try to bury his head in the sand - Parmenides before him had even more radically denied the existence of change and asserted that all is one and unchanging in defiance of the evidence of our senses - but it was Plato who developed the philosophy of stasis in the form that continues to trip us up today.

The problem with Plato's ideas is that they are an attractive nuisance. Any idiot can see change all around them all the time, but when someone is under the right kinds of pressure the rejection of change can be extraordinarily attractive.

Project managers, for example.

When you are being battered from every side to meet a deadline, the desire to pin down your targets to create a fixed frame of reference for managing the chaos can be overwhelming. In this way, most project managers become neo-Platonists, struggling against change.

The Oxford Dictionary of the English Language defines status quo as The existing state of affairs, a definition that is expressed in Platonic terms because it obscures the underlying principle unconsciously assumed to be at work by those who speak of a status quo: stasis.

status and stasis both derive from the same Latin/Greek root sta-, meaning to stand, as in to stand still, to hold, to be where and what you are in an unchanging fashion. You see, status quo seems to be a neutral phrase meaning "the way things are" because we ourselves are immersed in the philosophy of stasis—we take it for granted—and so cannot perceive that it really means "the way things are because they are standing still," that is, when things are the way they are because they are unchanging.

The extensive influence of mathematics in our education—which was advocated by Plato, by the way—exposes us deeply to simplistic math, the mathematics of is, of A equals A and A does not equal not-A, but cuts most of us off before we ever get into the mathematics of flow and change and flux from calculus on. We come to have a taken-for-granted worldview that understands a Newtonian and Cartesian fixed, mechanical universe far more easily than we can grasp the flow and flux of relativity and quantum physics because our math never went that far. Therefore, however much we may have thought we hated or resisted math in school, we pick up there and from the culture at large that the idea of a cosmos in which things are what they are and not what they're not just makes more sense than a cosmos in which everything is in continual flux.

The proof?

When under pressure, most of us seek security in some kind of fixed safety rather than by immersing ourselves more deeply into the flow of life. Stasis just feels safer to us, when push comes to shove. It feels more safe, more right, more natural, more real, more true.

So when we become project managers, we design our projects around fixed targets and seek to fight change to arrive at our fixed destinations. We are mercenaries who sell our services for money, and our services are that we promise to deliver a desirable fixed state by breaking down the gap between here and there into a series of manageable quantum steps. And at the end, we promise that the client will have been shifted into the new state they desire, they will have the things they want.

They will have a new status quo.

Too bad it's all a delusion, because Heraclitus was right: panta rhei, that is, everything flows.

Which brings us to the point of this series of essays: what is fluxus quo?

Yours truly,
Rick

It's the Culture, Stupid!

Dear Reader,

ACPE’s 2004 Technology Survey identified one bright spot in the health IT industry: the U.S. Department of Veterans Affairs’s VISTA system (Veterans Health Information Systems and Technology Architecture). VISTA has an unusually high user-approval rate, has won Harvard’s Innovations in American Government Award, and has dramatically and measurably improved the quality of healthcare at VA facilities over the last quarter century. When Hurricane Katrina devastated New Orleans, the only medical records to survive were those in the VISTA system at the Veterans Affairs hospital; VISTA was back online after only forty-eight hours of downtime.

Because VISTA is not only a high-quality health IT system but also a public-domain one, it is increasingly being adopted outside VA. The national health IT movement’s increasing awareness of the importance of IT is contributing to VISTA’s rising popularity both in America and abroad, which is coalescing into an international VISTA movement.

VISTA is not only an unusually successful software product but also an unusual health IT software-development culture that grew from an analysis of not only why most health IT projects fail so often but also why VISTA projects tend to succeed.

This same analysis explains why VISTA is an unusual success story for health IT: it is developed according to an alternative software-development culture that fits the needs of medical culture better because it essentially is the medical culture. VISTA is developed by a community of programmers most of whom began their careers as doctors, pharmacists, lab techs, or other medical professionals. Not only were they not steeped in the stasis-seeking software-development culture, they were steeped in the medical culture, which is used to a continuous state of changing needs.

Accordingly, the alternative software-development culture they created does not seek to achieve a perfect status quo but instead a highly responsive fluxus quo to keep up with the pace of medical change. Instead of seeking to avoid errors, this approach seeks to fix them quickly, something that would be impossible under the staggering quality-assurance (QA) overhead of the dominant paradigm.

Most importantly, in healthcare the stakes are too high to measure software correctness any way except by how well it meets the current state of medical needs; instead of measuring correctness by adherence to specifications this approach measures it by user satisfaction. Since medicine is far too complex for any individual or small body to authoritatively define, authority over what is to be done to the software is put in the hands of all of the users and the developers are given the authority to make whatever changes they need to whenever they need to in order to please their users. In short, authority is decentralized and the future of the software resides in the hands of an ongoing collaboration between the health professionals who use VISTA and the software engineers who develop it for them.

To allow the customization and peer review required in medicine, the source code for VISTA is open and the software itself is free. To avoid imposing any kind of penalty on efforts to make the software better serve medicine, adopters are not charged any kind of fees to report problems or have them fixed, nor are they charged to have improvements made. The economic model is instead (1) fee-for-service to set up a new VISTA site and train its adopters in how to use it, and (2) fee-for-relationship for ongoing support to encourage adopters to make as much use of support as possible, the better to channel their insights into the software development lifecycle.

In these and many other ways, the VISTA software-development culture is essentially the opposite of the dominant paradigm. It has more in common with more recent upstart methodologies like the open-source movement, rapid prototyping, agile programming, extreme programming, and so on, though it has been doing these and many other highly unusual things since long before any of these new methdologies had names.

The upshot of this VISTA analysis of the state of health IT can be summed up as follows.

Even if a perfect health IT solution were installed at a hospital, if the dominant software methodology is followed that software will become less and less able to meet the needs of its adopters as the state of medicine shifts out from under it until it eventually becomes a threat to the health of its patients. That is, good health IT software goes bad over time under the dominant software-development culture. Likewise, even if a dreadful health IT solution were installed at a hospital, if the VISTA software methodology is followed that software will become more and more able to meet the needs of its adopters and will change to take into account advances in the state of medical science. That is, bad IT software turns good over time under the VISTA software-development culture.

Yours truly,
Rick

Status Quo Kills

Dear Reader,

Most health IT projects fail because the software industry’s dominant response to the software crisis precisely collides with the unique demands of health IT.

Ever since Edsger Dijkstra began theorizing about how to solve the software crisis in 1972, computer science has focused understandably on the need to drive down the rate of errors in order to produce correct code. This focus on reducing errors has led to a focus on careful specifications and the careful coding designed to measurably meet those specifications. Under this paradigm, which is taught in all computer science departments across the country and followed by the software industry, success is defined as adherence to a software specification and errors as deviations from that specification. This error-avoidance philosophy is so ubiquitous, it is even implicit in the success and failure critieria used by The Standish Group in its CHAOS reports.

The resulting software-development process drives toward achieving the fixed goal of compliance with the specification, toward achieving a kind of crystalline, static perfection. That is, for the software-development process to avoid errors as much as possible, it has to impose extensive testing on all changes, which raises the cost and time required to make changes, which has the effect of discouraging changes. Indeed, everything about the dominant software paradigm rewards compliance with specifications and punishes attempts to modify the specification, especially in mid-course. It is a specification-driven, stasis-seeking methodology hostile to change. It dominates the software industry, including those companies engaged in producing health IT.

The problem with trying to apply such a model to healthcare is twofold.

First, healthcare is extraordinarily complex, essentially impossible to capture in a specification, which guarantees that any specification will deviate from the actual needs of the users. When faced with a choice between a clear specification and ambiguous statements of need, the specification-driven, stasis-seeking paradigm will always choose the specification over reality.

Second, even if it were possible to capture a perfect specification of the needs of health IT adopters, medicine changes continuously in response to the continuous scientific investigation into health and the resulting continuous stream of new discoveries that impact best practices. Hence, even a correct specification would become increasingly incorrect merely through the passage of time.

Healthcare is thus the worst possible case for the dominant software paradigm: an unspecifiably complex problem domain that continuously changes.

Unfortunately, health IT vendors have two overpowering incentives to disregard this fundamental mismatch of their approach and their subject.

First, the logic of avoiding errors seems irresistable, particularly so when backed by the consensus of an entire industry and its corresponding field of science; when struggling to gain control over out-of-control software projects, the retreat to controlling errors intuitively feels like the right move.

Second, the status quo is very, very profitable, especially because of the mismatch with the needs of healthcare. To help discourage changes to the specification in mid-course, it is standard procedure for IT companies to charge exorbitant premiums for such mid-course spec changes. The mismatch between health IT specifications and real healthcare information needs guarantees that health IT projects will involve a great many changes to the specification, for each of which the health IT vendor will earn a premium. The cost overruns that health IT adopters lament are the profits that keep health IT vendors in business.

In short, when applied to healthcare the dominant specification-driven, stasis-seeking, error-phobic software-development culture kills, but it does so both profitably and untraceably.

Yours truly,
Rick

Postscript: Let no one—no one—infer from this posting that I have anything less than respect and admiration for the great Edsger Dijkstra. My argument is that automating rapidly changing fields like medicine is a special case in which Dijkstra's prescription is not the best solution for addressing Dijkstra's concerns.

The Risks of Health Information Technology

Dear Reader,

Health IT success or failure numbers are very hard to come by, in part because some healthcare IT vendors require their customers to sign contracts banning them from publicly discussing the success or failure of their health IT, in part because the legal stakes are much, much higher with health IT failures which creates a disincentive to be forthcoming about dramatic failures.

Most numbers therefore are anecdotal. For example, the American College of Physician Executives (ACPE) surveyed its thousand members in its 2004 Technology Survey, and the results suggested that even successful health IT projects are often more expensive and less useful than expected, often permanently reducing physician productivity.

Informal surveys of non-management medical professionals at hospitals adopting IT reveal a much lower rate of satisfaction than that reported by those in management positions. The most frequent complaints are (1) a poor fit between the software’s design and the needs of its users, (2) poor responsiveness to requests to improve the software, and (3) hidden and horrendous costs that emerge in IT adoption during the process of trying to get badly fitting software changed to truly meet the needs of its users.

Off the record health IT programmers are often surprisingly candid about how bad their software is and about how resistant health IT company management is to making unprofitable changes that would improve its quality. Producing bad health IT software has been very profitable for a long time.

Health IT is the industry that results when one industry in crisis attempts to assist another industry in crisis, creating a collision of crises. The emergence of an American health IT movement is an important step toward improving health in this country, but unless the true state of health IT is understood and dealt with it will almost certainly be a very painful step indeed.

Yours truly,
Rick

The Risks of Information Technology

Dear Reader,

Unfortunately, missing from all this discussion is an appreciation of the dark side of health IT. Unless the problems with American health IT are as openly discussed, understood, and dealt with, as the problems with American healthcare, mixing IT with healthcare could deepen our national healthcare crisis.

The problems with health IT begin with IT itself. According to The Standish Group’s CHAOS 2004 survey results, only 29% of software projects come in on time, on budget, and with all the features originally planned. 53% of software projects fail at least partially, and 18% fail completely. Although people not familiar with the sorry state of the software industry find these numbers shocking, the reality is even worse.

The Standish Group’s numbers are oriented toward tracking the effectiveness of the project management, not the effectiveness of the resulting software. Any experienced IT user can tell you that many “successful” projects are actually failures from a usefulness perspective, bad ideas successfully imposed upon the organization. Similarly, these numbers do not factor in whether the budget was an effective use of the organization’s resources. Many, many projects are vastly more expensive than they should be for the value delivered. Once these crucial but omitted criteria are imposed on the CHAOS numbers, the numbers fall in line with the reality of an industry in crisis, The Software Crisis.

Since 1968, the software industry has understood itself to be immired in a crisis that defies solution. Forty years of innovation have still not managed to fix the kinds of problems measured by The Standish Group. Indeed, they have only clarified that at the root of the problem is a basic but paradoxical principle of complexity: Success breeds failure. Success with software development invariably increases the amount of software to be managed and taken into account in the next round of development. There is no limit to the potential complexity of software, but human capacity for managing complexity begins to fall apart at surprisingly low levels of complexity.

Successful software lifecycles become less successful over time. No wonder so many software adopters cyclically opt to replace what they have with something new! The temptation to blame the current software and start over with something else helps to fuel a major portion of the healthcare IT industry’s annual revenues. This wheel spinning may be successfully executed as projects and reported as such in the CHAOS reports, but it constitutes ongoing waste and failure disguised as success.

The consequences of our limitations are starkly illustrated by two parallel trends. First, at the same time that the hardware revolution has made computer hardware orders of magnitude smaller, faster, and cheaper, the software crisis has made software orders of magnitude larger, slower and more expensive. Second, the ability to succeed with software projects goes down dramatically as the scale of those projects increases, a factor noted in the CHAOS reports.

Yours truly,
Rick

The Importance of Health Information Technology

Dear Reader,

American healthcare is in crisis. It has been in crisis for decades, but thanks to the Institute of Medicine (IOM)’s Committee on the Quality of Health Care in America we can finally discuss it openly and objectively.

The committee’s reports—To Err is Human: Building a Safer Health System and Crossing the Quality Chasm: A New Health System for the 21st Century—have changed history by ending forever the viability of our industry’s de facto strategy of denying the problem. These landmark reports have also given us a framework for understanding the nature of the problems we face and the broad structure of the solutions we have to create and bring to bear to improve healthcare in America.

In Crossing the Quality Chasm, one of the IOM’s prescriptions for improving American healthcare is the use of information technology (IT): “IT has enormous potential to improve the quality of health care with regard to all six of the aims set forth in Chapter 2.” Indeed, the report’s ninth recommendation reads:

Congress, the executive branch, leaders of health care organizations, public and private purchasers, and health informatics associations and vendors should make a renewed national commitment to building an information infrastructure to support health care delivery, consumer health, quality measurement and improvement, public accountability, clinical and health services research, and clinical education. This commitment should lead to the elimination of most handwritten clinical data by the end of the decade.

This recommendation has launched a national discussion of the importance of health IT and strengthened the arguments of health IT advocates. Where health IT was once a technical topic of limited interest, it is now understood to be an essential component of our national strategy for improving healthcare. For example, the Commonwealth Fund’s 2003 Annual Report, Achieving a High-Performance Health System, uses language very similar to IOM’s in the report’s step four toward building a truly high-performance health system:

Invest in health information technology. Other countries are quickly surpassing the United States in adopting electronic medical records and prescribing systems. Their governments have invested in infrastructure and established the necessary standards, and the United States needs to do the same.

Increasing state and federal interest, funding, and mandates for health IT are symptoms of what has become a national health IT movement, much of which was catalyzed by the IOM’s reports. After all these years, there is a growing national consensus about the potential of IT to improve healthcare and a growing political will to do something about it.

Yours truly,
Rick

Fluxus Quo: A New Series

Dear Reader,

At Rodney Kay's request, let's postpone the continuing discussion of master planning in favor of what I agree is a long overdue topic for understanding VISTA and organic order: fluxus quo.

Since information out of context is noise, let's first explore how fluxus quo applies to health information technology (IT), to make clear why we care. Over the next few weeks, we'll run through a draft chapter of the book I'm writing on the VISTA lifecycle. This chapter makes the VISTA community's central argument about what's wrong with health IT and how to fix it. Fluxus quo is both the problem and the solution.

Yours truly,
Rick

Interlude: Horatio Smith on Shelley's Ozymandias

OZYMANDIAS
or
ON A STUPENDOUS LEG OF GRANITE, DISCOVERED STANDING BY ITSELF IN THE DESERTS OF EGYPT, WITH THE INSCRIPTION INSERTED BELOW

In Egypt's sandy silence, all alone,
Stands a gigantic Leg, which far off throws
The only shadow that the Desert knows:
"I am great OZYMANDIAS," saith the stone,
"The King of Kings; this mighty City shows
"The wonders of my hand." The City's gone,
Nought but the Leg remaining to disclose
The site of this forgotten Babylon.
We wonder, and some Hunter may express
Wonder like ours, when thro' the wilderness
Where London stood, holding the Wolf in chace,
He meets some fragments huge, and stops to guess
What powerful but unrecorded race
Once dwelt in that annihilated place.

The Principle of Organic Order: Our Journey So Far

Dear Reader,

Let's resume our quest to understand the VISTA software lifecycle by continuing our exploration of Christopher Alexander's small masterpiece, The Oregon Experiment. In this book he describes how architectural design used to be organized according to a timeless way of building called organic development and describes the problems that have accumulated in modern architecture as a result of abandoning it. Organic development is the same powerful process harnessed in the classic VISTA software lifecycle. Since the VISTA lifecycle's never been adequately described in writing, we're trying to understand it by reviewing Alexander's description of organic development and by comparing and contrasting it with VISTA's model.

Alexander describes organic development in terms of six principles, with a chapter on each. When we left off in September, we were in chapter one, which describes the Principle of Organic Order. Alexander summarizes this principle as:

Planning and construction will be guided by a process which allows the whole to emerge gradually from local acts.

To illustrate the importance of this principle, Alexander explores the nature of the modern architectural-planning process, in which planners imagine the problem of creating good architecture boils down to a struggle between chaos and order in which their job is to impose order (a vision, an aesthetic, a design) on an otherwise chaotic situation. Planners' main tool in this battle is the modern master plan, which attempts to fix in advance the more-or-less complete solution to the problems they've been hired to solve.

Alexander demonstrates that master plans themselves are deeply problematic, because in good architectural design there's a far more important quality at work than either order or chaos, what he calls organic order. Master plans by their very nature drive organic order out of a project because they're oblivious to it. As long as planners try to reduce their mission to a struggle between two choices, order and chaos, they'll remain unable to understand the damage they do with their master plans, because they'll only look for the elimination of chaos, the creation of order, and call that successful - even though the form of order they create causes worse problems than the chaos they've eliminated.

As with architecture, so with enterprise-scale software-development projects: the creation of a master plan is the first major milestone of any serious project, and the plan remains the fundamental organizing tool for everything that follows. To accuse master plans of being the root problem in modern architecture (or software engineering) is a radical act. It demands more than accusations, so Alexander supports his claim by examining the essential strengths and weaknesses of all master plans, to show that the problems they cause can't be avoided without abandoning this entire approach to planning.

In our review chapter one, we began by exploring the problem of chaos. Although chaotic situations do spontaneously generate forms of order, they also spontaneously generate forms of disorder, and the conflict between the two ensures that chaotically generated order cannot remain stable over long periods of time and also cannot create higher forms of order. For something as complex as medicine with stakes as high as life or death, master planners are right to reject chaotic order as a source of organization for software architecture. The fundamental strength of master plans is that they provide an alternative to chaotic order.

Their fundamental weakness is that in its place they impose totalitarian order, which is essentially hostile to organic order, so we next began reviewing the four unavoidable problems of master plans described by Alexander.

First, master plans are too precise about their solutions. They deal with unknowns by false prophecy, by trying to project a detailed picture of the future. They do this in part because their focus isn't on understanding the problem; it's on selling a solution, and to do that they have to paint a bold, emotionally compelling picture to win the contract. The buyers contribute to this because they're uncomfortable with the inescapable uncertainties at the beginning of any project, so they demand a false clarity to comfort them and give them enough confidence to proceed. The many details fixed in the master plan lend it persuasiveness, verisimilitude, but not truth. Later in the project those same details will tear the project apart as they collide with reality in precise and (only from the perspective of hindsight) predictable ways.

Second, master plans are too imprecise about the problems they mean to solve. As any experienced builder or programmer will tell you, only after you solve a problem do you fully understand it. The process of construction creates a flow of discoveries about the nature of the problems we face, surprises that reveal the many ways we're wrong about what we have to overcome to succeed. We know the most about the overall problem at the end, but master plans are created at the beginning, thus locking in the maximum amount of ignorance about those problems. Master plans propose solutions to problems the participants will never again understand so poorly.

Next we'll explore the third problem, alienation.

Yours truly,
Rick

Interlude: Do the Right Thing

Dear Reader,

Do you want VISTA to succeed, to help us drive medical error out of the top five killers in America? Here are seven steps to make that much more likely:

1) Cancel every VISTA-replacement project going on in the VA and never fund another.

2) Reassemble the primary package-development teams and resume the traditional cycles of rapid patch development and regular releases of new versions of all packages. That lifecycle made VISTA great; violating it crippled VISTA in the DOD, and is crippling it in VA. Put authority over the software back in the programmers' hands and leave it there; they know more about what's possible than you do.

3) Reorganize VA to put the majority of programmers back to work for local hospitals and regional networks. Put authority over the hospital computer systems back in the hands of hospitals and local networks; they know more about how to make VISTA serve their hospitals than you do.

4) Invest heavily in training up a new generation of VISTA experts. This is the second most important item on this list, just behind the next one:

5) Let users' requests for bug fixes and enhancements directly drive all VISTA development. Put authority over development priorities back in the users' hands and leave it there; they know more about what they need than you do.

6) Gut the Clinger Cohen Act so VA and IHS can do the right thing and invest in the people they need instead of being forced to outsource all their expertise and capabilities. The Clinger Cohen Act requires VA and other federal agencies to strangle themselves, which they've been working on doing since 1996. The results speak for themselves.

7) Invest heavily in rebuilding the tattered support community VISTA had before VA shredded it in the mid 90s, the support community the VA needs and cannot thrive without. VA needs to give up its siege mentality, needs to tear down the iron curtain it has built between itself and the rest of the world and start really collaborating with other VISTA organizations, beginning with IHS. Terrorists are not assaulting the VA, and VA's own VISTA experts are not the enemy; VA needs to spend less time and money investigating its VISTA experts for possible "collaboration without permission" and instead seek out and invest in opportunities to work together with others. Having frustrated, insulted, and driven away half of its top-tier VISTA expertise, VA will never again be the sole arbiter of VISTA's future. VA will never again be in a position to do as much damage to VISTA as it did over the last fifteen years. But if VA does all of the right things, if for example it does these seven things among many others, then VA could become a good VISTA organization again, instead of a hostile organization where good VISTA professionals sacrifice and struggle to do the right thing even though they know they may well be punished for it.

I'm hopeful about new VA CIO Roger Baker, but I'm looking for more than hope. It will be easy to measure whether he can fulfill the promise of his good words and good intentions, because deeply experienced VISTA hardhats know what he needs to do to turn things around.

"It's the thought that counts" is a pleasant thought, but in the end what matters is that we find a way to do the right thing. You're not in the dark. You're not lost. You're surrounded by people who know what you need to do to succeed with VISTA. Listen to them. Help them help you to do the right thing.

Yours truly,
Rick

Interlude: Shelley on Hybris

OZYMANDIAS

I met a traveller from an antique land
Who said: Two vast and trunkless legs of stone
Stand in the desert. Near them on the sand,
Half sunk, a shatter'd visage lies, whose frown
And wrinkled lip and sneer of cold command
Tell that its sculptor well those passions read
Which yet survive, stamp'd on these lifeless things,
The hand that mock'd them and the heart that fed.
And on the pedestal these words appear:
"My name is Ozymandias, king of kings:
Look on my works, ye Mighty, and despair!"
Nothing beside remains. Round the decay
Of that colossal wreck, boundless and bare,
The lone and level sands stretch far away.

Four Problems with Master Plans: (2) Imprecision

Dear Reader,

Yes, after the first criticism this second one doesn't seem fair, but it's true. Life, which isn't a logical syllogism, is filled with seeming contradictions. Here's one: not only are master plans too precise, paradoxically they also aren't precise enough.

Specifically while they're too precise about what the future solutions should look like, they're far too vague about the nature of the problems they're supposed to solve. As a result, even in theory master plans only address some parts of the problem and not others.

This is bad but understandable.

Master plans can't possibly identify, let alone address, the full scope of the problems they're supposed to solve because they're created at the beginning, before any serious effort has been made to solve the problem, and thus before Murphy's Law has had its hundreds of chances to teach us how wrong we are about the full extent of the problem. At the time we write a master plan, we just don't know enough about the problem to get its details clear in our heads, and without those details we can't really solve the problem, only take a clumsy swipe at it.

This is understandable but bad.

How bad?

Sometimes it's inconvenient.

Maybe while planning your day, you fail to pay attention to what you're doing at the moment and lock your keys in the car, thus ruining your plans.

Maybe to save three million dollars you replace a deep, stiff bridge design that lets the wind pass through it with a cheaper, shallower, more flexible one that makes the wind pass over and under it, creating aerodynamic lift so it flutters in the wind, tears itself apart, and collapses into the deeps.

Maybe you overplan your bombing runs by putting precisely the right amount of gas in the planes for them to fly out, bomb their targets, and return, but then they encounter a headwind that makes them burn up extra gas so they drop out of the skies before returning home, forcing the pilots to parachute to safety.

Sometimes it's catastrophic.

Maybe you use an explosive gas for airship buoyancy and then try to control all the many scenarios that could cause it to ignite, only to discover that sooner or later your planning or control slips, killing thirty-six people.

Maybe it seems inconvenient to standardize your fire-hose couplings, so you don't, so when your city catches fire the fire engines from neighboring cities arrive only to find they can't help you, and your city burns to the ground.

Maybe your regulations only require a total lifeboat capacity of 1,178 people, even though your ship can carry 3,547, causing the deaths of 1,517 people.

Details matter.

It is the details of problems that make them problems. Something as small as an O-ring seal can kill you if you get it wrong. Engineering is the discipline where Don't sweat the small stuff is a recipe for disaster (and where . . . and it's all small stuff is criminal negligence).

In medical informatics, more starkly obvious than in much of life, the problems we face seem to shift when we discover their nature is different than we at first thought, and they also actually shift as the nature of medicine and software shift around us. The details of our solutions have to be able to shift along with the problems or we can't solve them.

Master plans prevent that dance of solutions with problems in four ways.

First, they emphasize the big picture over the details, wasting energy on prophecy that should have been spent on understanding the problems better, so the necessary details of the solution are missing or vague.

Second, what details they do specify are shaped by design to create (i.e., to serve) the totality, not by discovery to actually solve the problems, so they tend to be out of sync with the reality of the problems (i.e., wrong).

Third, they rigidly lock in those details, preventing them from moving with the problems as the problems shift, soon making any accurate details outdated and incorrect.

Fourth, the nature and interactions of the problems we aim to solve are far too complex to capture in a plan to begin with, ensuring that no matter how much effort goes into a medical-informatics master plan, its worldview always ends up being a ridiculous cartoon of the situation it is meant to deal with.

Any one of these characteristics would be enough to cripple a master plan. Together, they create irresistible forces that cause all VISTA master plans to converge on the same brief lifecycle of (1) ballyhoo, (2) bogging down, and (3) breakdown. And yet the inevitability of failure evidently can't compete with the intoxicating feeling of control a master plan creates, judging by the relentless parade of VISTA-replacement (or "modernization") boondoggles. It would be tragic if after fifteen years it weren't so drearily risible.

Still, at least we can serve as an object lesson to validate Alexander's point:

Master plans fit the shape of the problems with too little precision, and so they fail.

Yours truly,
Rick

Postscript: And so, Alexander's critique of architectural master plans holds even more true for enterprise-scale medical-informatics than it does for architecture: Thus, as a source of organic order, a master plan is both too precise, and not precise enough. The totality is too precise: the details are not precise enough. It fails because each part hinges on a conception of a "totality," which cannot respond to the inevitable accidents of time and still maintain its order. And it fails because as a result of its rigidity, it cannot afford to guide the details around the buildings which really matter; if drawn in detail, these details would be absurdly rigid.

Interlude: Why Some People Just Don't See It

Dear Reader,

In response to my post on 30 August 2009, "Principle 1: Organic Order, part 1: The Three Kinds of Order", on August 31, 2009 at 6:58 AM Die Anyway wrote:

're:"...it's precisely because VA turned away from those processes toward more totalitarian ones that it lost the ability to effectively manage or develop VISTA."

'I see it and you see it so why don't the ivory-tower, pointy-hairs see it? Or do they see it and ignore it because they have an entirely different agenda?

'In any case, as a biologist and programmer I like the idea of organic design even if it did originate in those pre-historic times of the mid '70s.

'Eat well, stay fit, die anyway!'


My answer doesn't fit within a blogger comment, so I'll reply here, as this post:


Dear Die Anyway,

The VISTA managers in VA who try to overly centralize and control VISTA, i.e., who try to impose totalitarian rather than organic order, (1) are not necessarily the majority, just the most powerful or in-favor managers (I know some good VISTA supervisors and managers in the VA and elsewhere), and (2) they do not see what we see because they lack the proper understanding of their context.

They think they're facing an entirely different kind of problem than they actually are. If you interpret their actions from the standpoint of their worldview, their approach makes sense for a while. As we'll explore later in this weblog, their approach leads to a gradually increasing breakdown that eventually becomes so dire that even they can see things aren't working. By then they're usually too burned out and despondent to be capable of taking responsibility for their actions by steering the organization in a healthier direction. There's a certain amount of morale a manager has to have to be effective, and unfortunately when one is committed to a false idea one tends to use up one's effective energies on the dead ends.

The problem with the actual cosmos is that its forces and principles are subtle, easily overlooked. Any reductionist intellectual can mentally reduce an intricate organic system to a trivial mechanical one. When one is tasked with something impossible, like managing VISTA while pleasing Congress, one's mind finds otherwise implausible oversimplifications oddly irresistible, because they offer a desperately needed false hope.

Once people decide on an interpretation of reality, they have an astonishing ability to see every "fact" that confirms that interpretation, and an equally astonishing ability not to see everything else that disproves that interpretation. To the outside observer this seems to result in irrational behavior, but from within the interpreter's reality bubble, within the framework of interpretations, the behavior may be completely rational, even inevitable.

It is a great disappointment to discover that rationality has been overhyped. Reason can be used to reach the falsest or vilest conclusions through irresistible logic drawn from false premises.

Rationality is worthless, even dangerous, unless it is harnessed to two things grossly undervalued - even invisible - in our culture:

(1) profound insight into the essential principles at work in creating each situation, and

(2) mature, discriminating taste capable of balancing priorities among competing values to figure out which good must give way for which other good.

Insight is vital because the truth of hardly any situation is visible on the surface, but instead must be sought in the nuanced and subtle but powerful forces that create that surface.

Taste is vital because contrary to just about everything we teach through schools, the arts, and common sense, the bad things in the world do not result from a great conflict between good and evil, but from conflicts between various goods.

Values conflict, which the classical Greeks knew but which we do not, and which good deserves priority over the others shifts and flows from situation to situation depending on the hidden forces at work.

Neither the truth about the source of problems nor the two vital qualities needed to deal with that truth end up in position descriptions, evaluation criteria, or the law. Instead, managers inside the government and out are held to simple-minded, mechanical criteria, and if they do not bend themselves to fit those laws, criteria, and requirements then they cannot thrive in their careers. They have to believe what they're doing is right, so they do.

In short, I think the answer to your important question is answered best by Upton Sinclair:

"It is difficult to get a man to understand something when his job depends on not understanding it."

Part of why I love biology is that it is difficult to explain in simplistic mechanistic terms, so its study tends to compel you to develop an appreciation of deeper, essential principles.

Thank you for your comments.

Yours truly,
Rick

Four Problems with Master Plans: (1) Precision

Dear Reader,

It is simply not possible to fix today what the environment should be like twenty years from today, and then to steer the piecemeal process of development toward that fixed, imaginary world.

This escapes us when we agree to create a master plan. We forget the simple truth that we're not gods or prophets; predicting the future is never our forte but often our downfall.

Our grand designs fail in four ways.

Today we discuss the first: a master plan is too precise. By predicting the precise shape of the future - the following buildings or software modules will be built in the following ways, and will contribute to the design like so - it leads us into precise collisions with reality when some parts of the plan prove impossible, as always happens.

Murphy's Law isn't very funny to software engineers; it's the reality under which we operate, a reality that's too easy to ignore during planning but impossible to ignore later when our designs come crashing down around all our heads at great expense.

VISTA programming is designed to take advantage of Murphy's Law. It's a highly adaptive process in which we immerse ourselves in the problem with our adopters as guides, in which we do not set out to predict the future, only to solve some specific, immediate problem. A nonprogrammer simply cannot imagine how much failure is involved before one achieves success, but that string of failures is why the VISTA model works and master planning doesn't.

By making planning a cumbersome, expensive process that comes at the beginning, before Murphy's Law has taught us the many ways we're wrong about what's possible, master planning builds a profound ignorance about what's possible into the development process from the start. It saves the exposure of all its failures for the end, when the differences between what's possible and what the plan proposes finally accumulate so greatly that everything collapses in a great crash. Ironically, it's the act of struggling to preserve the plan's success that increases the scale of the failure, since covering up problems and investing more time and money help intensify the inevitable collapse.

By contrast, the VISTA methodology begins making mistakes up front, one at a time, while they're still small - a string of little crashes. We continue making mistakes the whole way, from beginning to end, but they remain small because (1) there is no big, expensive investment in a master plan to try to protect so we can and do change directions repeatedly, (2) the future user of the software is sitting right there the whole time saying "No, we can't do it like that," and (3) instead of struggling to adhere to an increasingly irrelevant plan the team is building up a map of the terrain of the project, a practical guide to many different ways to succeed or fail.

The contrast is stark. Master plans are prophecies. VISTA plans are histories. Master plans are expensive, produced separately from the software. VISTA plans are cheap, produced alongside the software. Master plans are written and finished at the beginning, when we know as little as possible about the problems and its solution. VISTA plans are written during programming and not finished until the end, when we know as much as possible about what didn't work and what did.

The excessive precision of a master plan comes from its focus on the solution, which means trying to pin down the future. VISTA plans avoid that by focusing on understanding the problem and letting the solution emerge gradually and often unexpectedly.

That is, in the master-planning approach we create the illusion of knowing where we're going without first really understanding the nature of the problems we claim to be solving. In the VISTA planning approach, we acknowledge from the start that we have only a vague idea where we're going, only that we know we're going to solve a specific problem by getting to know it very well.

The VISTA approach involves a lot of false starts and backtracking, a lot of mistakes - it looks sloppy and loose compared to the professionalism of a master plan - but that's its strength and the weakness of a master plan.

The VISTA methodology was first advocated by John D. Chase, Jack Brooks, and Ted O'Neill. Here's what Ted O'Neill and Marty Johnson had to say about the difference between the master-planning approach and their own, in their memo to Ken Dickie back on June 10, 1981:

Requirement specifications can be derived effectively and quickly by developing a basic functioning system with the intended user, then further refining the system through user-suggested modifications. The resulting system, in daily use, will provide a much more complete and accurate reflection of user requirements than any narrative description of a system that has not been built. Moreover, by using available modern techniques, such as natural language application generators, database management systems and other software tools, fully operational prototype systems can usually be produced more rapidly than can paper descriptions thereof. Requirement specifications that are produced before any prototype testing activity has been undertaken are now recognized by most computer professionals as inadequate.

Now go back and reread Alexander's quote from the top of this blog entry.

These are two different ways of saying the same thing. They have the same criticism of master plans, independently arrived at from two different perspectives. Our own history bears out the truth of this convergent revelation.

Master plans predict the shape of the future with too much precision, and so they fail.

Yours truly,
Rick

Postscript: Master planning isn't the only kind of planning. The Principle of Diagnosis involves a very different form of up-front planning that helps to create natural order instead of totalitarian order. There are also forms of community planning that focus on studying the problems communities face - the trends, developing problems, and so on - and then draw general, reasonable conclusions to help communities plan. Likewise, there are dynamic forms of project planning designed to accommodate the unpredictable path of research and development. Each of these approaches is the opposite of the master planning approach because they focus on what we can know best - the nature of our present and developing problems - and sketch lightly and prudently in recommending how and when those problems can be solved.

Principle 1: Organic Order: The Problem of Chaotic Order

Dear Reader,

Chaos is not what most people think it is.

To begin with, I'm not using the mathematical term, which sweeps up areas of mathematics outside the traditional disciplines to show there's mathematical order beyond that we usually consider. Although an interesting subject, it's only metaphorically related to true organizational chaos. I'm using the other meaning for chaos: the results of a complete lack of planning, the outcome feared by planners and managers everywhere.

With that in mind, I must next emphasize that chaos isn't the absence of patterns. On the contrary, even fully random processes produce emergent patterns. That kind of patternless chaos people imagine actually takes quite a bit of planning and control to pull off. True chaos always generates two kinds of patterns: accidental and intrinsic.

First, even the chaotic results of random processes produce both statistical ones (like bell curves) and specific ones (like the constellations among the stars) as a normal and expected result of the lack of planning. These macro- and micro-patterns can be used as the seeds for future organization.

Second, when living things are involved (intelligent or otherwise), their needs drive them to produce nonrandom patterns of behavior and organization so they can live and thrive. Firstly, the forces within living things aren't random but instead will lead them to organize their lives in ways that meet their needs - like returning to rivers to drink (or to hunt the drinkers) on a regular schedule. Secondly, living things inhabit ecosystems whose powers and forces reward and punish their behavior, leading living things to adapt in common ways to common forces. For example, even such different life forms as fish, reptiles, and mammals will gradually evolve into the same shapes if they occupy the same ecological niches because the same forces will work upon them over their generations. This is why the tuna, the ichthyosaur, and the dolphin end up closely resembling each other despite very different origins - they converge on the only viable solution to the pressures they're under.

Advocates of imposed order usually underestimate the value of these forms of chaotic order. A great deal of useful order can be achieved without imposition, simply because of people's intrinsic need for order. For example, despite all the screaming and panicking shown in disaster movies, which help to train into us a terror of chaos and of other people (and an artificial craving for imposed leadership that is well understood by juntas when they execute their coups d'etat), during and after disasters people frequently organize themselves to help out family, friends, and neighbors, without any leadership at all - not always, but more often than fearmongers would have us believe.

Advocates of chaotic order, such as anarchists, point to this kind of natural collaboration and contrast it with the inevitable evils of totalitarian order to argue their position that any form of central organization is both unnecessary and unwanted. Anyone who's bothered to study Nazi Germany or Stalinist Russia, or who's thought long and hard about how readily republics loose their democratic bonds to become fascist empires, or who's observed how the disturbing trend toward decreasing personal freedom and privacy changes our culture more and more toward the dystopia of Orwell's novel 1984, has to give this position more than a moment's consideration. There are serious points raised by advocates of chaotic order, and we must do justice to those issues or suffer the consequences of those who fail to learn from history.

Nevertheless, I'm not persuaded, and you shouldn't be either.

There are two excellent reasons why chaotic order is not sufficient.

1) The order produced in chaotic systems is generally offset by the disorder also produced by chaotic systems. Without central coordination, some enclaves will self-organize into more or less peaceful communities, true, but others will devolve into internicine feuds (Internet flame-wars, anyone?), and still others will become predatory. Just as I'll not grant advocates of totalitarian order their insistence on the absence of order within chaos, so I'll also not grant the advocates of chaotic order the absence of destructive consequences.

2) Chaotic order is almost always local, almost never universal. That's enough to solve certain categories of problems but not others, not problems beyond a certain scale. As a VISTA theorist, I am particularly bothered by this reason, because the complete VISTA lifecycle requires that (a) a great number of specialists be given the time and focus needed to master their chosen parts of VISTA, (b) a large number of autonomous teams follow a common culture to ensure that what they produce is compatible, that it creates a living whole, and (c) people not only act with good intentions but also actually do the right thing, since lives and privacy are at stake. These things require more than local or temporary order. These requirements unfortunately fall into chaotic order's demonstrated weaknesses, making it a poor choice of approaches for developing VISTA.

But beyond two there is an even better third reason to reject it for VISTA: it is predicated on a false dilemma, namely that one must choose between totalitarian order and chaos, or some compromise between them. As Alexander argues and I'll explore in this weblog, natural order represents a superior alternative to either of these dysfunctional approaches.

That is, since we already have a model for a system of organic order for managing VISTA development, we don't need to pay the price of chaotic order's weaknesses in order to gain its benefits.

Nevertheless, we're going to return repeatedly to the subject of chaotic order, because in management terms (and now for the punchline) chaos is just a pejorative term for freedom, reflecting a deep unacknowledged terror in all of us. Until we examine our ambivalent relationship to freedom, any cost-benefit analysis of chaotic order is at best highly suspect.

Yours truly,
Rick

Interlude: Ordering The Oregon Experiment

Dear Reader,

You can order a copy of The Oregon Experiment from your friendly neighborhood independent bookstore, or from any number of online book services, including:

A Libris
Abe Books
Amazon
Powell's

Yours truly,
Rick

Principle 1: Organic Order, part 1: The Three Kinds of Order

Dear Reader,

Alexander diagnoses a nearly universal illness in our systems of architectural planning: we use master plans to try to guide our development, to plan for the future:

. . . master plans, are intended to coordinate the many hundreds of otherwise independent acts of building . . . to make sure . . . that the many acts of building in a community will together gradually help to create a whole, instead of merely making up an aggregation of unrelated parts, a chaos.

. . . the master plan, as currently conceived, cannot create a whole. It can create a totality, but not a whole. It can create totalitarian order, but not organic order . . . although the task of making sure that individual acts of building cooperate to form a whole is real, the conventional master plan - based on a map of the future - cannot possibly perform this task . . . because it is too rigid to do so - and because, in addition, it creates an entirely new set of other problems, more devastating in human terms than the chaos it is meant to govern.

It's not enough to replace chaos with order. Order can be worse than chaos.

The problem with chaos is that the "whole" (really just a collection) is less than the sum of the parts. The entire reason for planning is to do better than this, but master plans only give us chaos's opposite: totalitarian order, when the parts are designed to support the whole. The problem with totalitarian order is that the parts don't adequately support themselves or each other, only the whole, and the whole doesn't support the parts, only itself.

Seen with a clear understanding of what we really want in the world, chaos and totalitarian order strongly resemble each other, because they both lack what we most need. What we're striving for isn't a compromise between order and chaos; it's nowhere on that spectrum, because that spectrum doesn't take into account what really matters.

What we need is what Alexander calls "natural order" or "organic order":

We define organic order as the kind of order that is achieved when there is a perfect balance between the needs of the parts, and the needs of the whole.

In an organic environment, every place is unique, and the different places also cooperate, with no parts left over, to create a global whole - a whole which can be identified by everyone who is a part of it.

This kind of order is called natural because it's the kind of order nearly universally created by natural processes, and only rarely by artificial ones. Living processes result in parts that are fully, distinctively, amazingly themselves and yet are equally completely part of and contributing to a greater whole.

The balance between the needs of the parts and the needs of the whole allows the parts to be simultaneously highly specialized, optimized to do their jobs as well as they can be done, while simultaneously helping to create a healthy whole that couldn't exist without those parts.

Consider the relationship between the parts and the whole in the human body. A stomach and a brain are astonishingly different, each highly adapted to its job - if you had to know how to digest an apple you'd starve to death, and we all know what happens to people who try to think with their stomachs - yet each organ fully contributes to the success of the whole. You need them both to live. They're indispensably part of a whole, yet wildly different from each other.

Or consider an ecosystem. A cheetah is an amazing creature. Watch a film of a cheetah chasing down prey and you can hardly believe what you're seeing is possible. It gives every appearance of being a completely original, independent design, a master at what it does. And yet, it is simultaneously utterly dependent on its ecosystem for its survival. Without enough habitat and prey, the cheetah cannot survive. Likewise, without the cheetah, its prey overpopulate and strip bear the vegetation, causing ecological collapse and the widespread death of its prey.

In both examples, the relationship among the parts and between the parts and the whole are nothing like those relationships in a chaos or a totalitarian order.

The parts in a collection (chaotic order) are unrelated, random things. They rarely work as a whole, and never for long; they work against one another at least as often as they work together.

The parts in a totality (totalitarian order) are cogs in a great machine, all resembling each other because they're designed by the same designers. They break down readily because they do not support themselves or each other, only the whole. Place a living thing within such an order and try to use it as a part and it soon perishes because nothing in a totality fully supports the needs of the part.

The parts in a whole (organic order) look very different from one another, each highly adapted to its situation in order to be great at what it does, but if you study them long enough you begin to see the subtle common principles of their designs and to understand the ways in which they're all dependent on one another and contribute to a common whole that in turn strengthens each of them as parts.

Put another way, in organic order each part is also a whole, identifiable and supported by its own parts, by the other wholes around it, and by the whole of which it is just a part. Conversely, each whole stands for more than just itself; it also contributes to the health of its own parts, to the other wholes around it, and to the greater whole they make up together.

This quality is what makes living systems so much more scalable than artificial ones. Each scale of organic organization supports not only itself but also the greater and lesser scales. This is why as totalitarian structures get larger they get weaker - the increasing demands of the whole parasitically sap the strength of the parts - but as organic ones get larger they get stronger - the existing whole and its parts strengthen each new part added to it, and each new part in turn strengthens the other parts and the whole.

This is why the cosmos creates wholes and parts that are so vastly more complex than anything made by man, and yet those systems of wholes and parts last for millions or billions of years instead of the mere decades and centuries we manage. Chaotic order and totalitarian order may be sufficient for the trivial levels of complexity and brief time frames people usually work in, but if you need to create something vastly more complex to last for much longer, then only organic order will do.

VISTA is at that scale of complexity and longevity. It only exists because it was created using living processes that resulted in natural order, and it's precisely because VA turned away from those processes toward more totalitarian ones that it lost the ability to effectively manage or develop VISTA.

Organic order is the grail. This is what you want, and nothing less. It's a key to everything in VISTA. It's what Alexander in later works refers to as a living system, what he uses in his new definition of life in The Nature of Order. Understanding what organic order is, learning to detect its presence or absence, figuring out why it matters so much and how to create it - these are the central challenges in becoming a great VISTA manager.

Yours truly,
Rick

Postscript: Throughout this exploration, I refer to the principal author as a shorthand for the team of authors who developed these ideas and wrote this book. To set aside expediency for a moment and give credit where credit's due, here's the full bibliographical entry on this book:

The Oregon Experiment. Christopher Alexander, Murray Silverstein, Shlomo Angel, Sara Ishikawa, and Denny Abrams. New York: Oxford University Press, 1975. ISBN 0-19-501824-9. Volume 3 in a series from The Center for Environmental Structure.

Our Story So Far . . .

Dear Reader,

There's an emerging order to this weblog. We began with a little introduction to VISTA before turning to the long project of investigating the VISTA software lifecycle. The first four chapters of that investigation are related in a flow of their own.

First, we started by breaking up the ground, by challenging the most common misunderstandings and filters that prevent people from being able to understand the lifecycle even when they're looking right at it. This was the chapter on the eight essential points.

Second, now, we're taking our first steps into understanding the lifecycle by using something similar that's already been well explained as a frame of reference. That's this new chapter on the principles of organic development as explained in The Oregon Experiment.

Third and fourth, after establishing that point of reference we'll drive right to the heart of the VISTA software lifecycle by examining its two essential characteristics that together set up the pattern that makes sense out of every detail of its architecture or policy: flow (fluxus quo) and life (living systems).

Thereafter, in subsequent chapters, with the main misunderstandings dealt with, a common frame of reference to compare and contrast with, and the two essential characteristics fully described, it should be easy enough to work our way through the details of how to manage VISTA effectively.

Like all plans, I'm sure this won't survive much contact with the subject matter, but let's find out.

Yours truly,
Rick