Point 7: Restart the Lifecycle with Fileman and Forum, part two

Dear Reader,

. . . continued from part one, yesterday.

Rebooting the VISTA lifecycle will also require the right license for the software. We can't use a license for the core infrastructure packages that discourages the widespread adoption of the common shared VISTA core, by adopters and developers alike. If that core isn't kept in common, particularly the infrastructure packages like File Manager and Kernel, then VISTA will continue to balkanize into mutually unintelligible dialects, drifting apart as the VA and DOD dialects have done.

Although there may be VISTA packages for which the GNU Public License (GPL) is a suitable license, it's certainly not suitable for the core infrastructure packages because of the extreme degree of integration these packages have with all other VISTA packages, an integration so extreme it pushes past normal definitions of such terms as "derived works" and "interfaces." This unavoidable mismatch between the GPL's terminology and VISTA's unique internal structure makes eventual court battles over licensing violations highly likely, and offers no guarantee that such lawsuits would be decided on the basis of a reality most people don't comprehend and the rest can't explain clearly. That is, although the GPL isn't inherently incompatible with File Manager, for example, its terminology misrepresents Fileman's relationship to the rest of VISTA so badly as to make it an unreliable safeguard of our intentions.

Whether the license for the VISTA infrastructure packages needs to be the Lesser GPL, the Eclipse Public License, or some other open-source license is a question that needs to be sorted out in short order so we can get on with the work such licenses are intended to safeguard.

Most people should never ever work on the File Manager or Forum software because of how complex they are and because of the potential of problems in these areas to affect every other VISTA package, but they must be our top priorities. With the right experts leading these efforts, they'll be the right projects to restart the lifecycle.

Yours truly,
Rick

Point 7: Restart the Lifecycle with Fileman and Forum, part one

Dear Reader,

It's taken roughly 1,000 programmers driven by tens of thousands of users over thirty-two years to get VISTA to its current level of sophistication. To achieve a VISTA-lifecycle renaissance will eventually require something similar.

But VISTA didn't start on such a scale. Back when Ted O'Neill and Marty Johnson first launched what they called the MUMPS Systems (VISTA's first name), they only had about twenty-four programmers responding to a small population of users. That was plenty to create a system easily recognizable as the ancestor of today's VISTA, and it'd be plenty to get things moving properly again.

To prime the pump, we should begin even smaller, with two small teams focused on two areas.

First, we should build on Medsphere's excellent File Manager work to create File Manager version 23. Fileman is the easy choice because it's the core architecture for all of VISTA and therefore the most important point at which to begin reinvigorating VISTA, since every VISTA package benefits from Fileman's improvements. That one package'll be enough for us to restart, test, and refine the VISTA software lifecycle.

Second, to refine that lifecycle we'll need to be able to fix or improve its software too - KIDS, NOIS, Patch Module, and the other packages that run on or communicate with Forum - as we proceed with Fileman, so that should be the other subject of our work, the focus of the second VISTA-development team.

This'll require funding for the core developers for each of those packages plus the small constellation of students we need to surround each one with. It'll also require funding for a Forum system manager and his students, a verifier and her students, and a database administrator and his students. The right fifteen to eighteen people could make this fly.

After the prototype lifecycle is up and running, we should plan to expand it to include the Kernel, Victory Programming Environment, and Laboratory packages (for reasons we'll explore in future postings), with a team of four to six people, mostly promising students working under the top gurus for each package.

To be concluded in part two, tomorrow. . . .

Yours truly,
Rick

Point 6: VISTA's Software Stream Has Many Tributaries, Part 2

Dear Reader,

. . . Continued from part one yesterday.

Although it hasn't happened much for the last fifteen years, a vital part of this software stream is that every year or two each package should release a fresh version of their entire package, an upgrade that goes through a more intense re-engineering, documentation, testing, and verification cycle than patches get. This helps keep each package's architecture fresh and provides a deep housecleaning period to flush out the subtler or more intractable bugs. So, the "patch stream" is actually a software stream that includes both patches and new versions.

So what about complete snapshots of VISTA? Most software in the world is managed by releasing entire new snapshots. What about VISTA?

Well, this is part of where the VISTA lifecycle turns everyone's expectations on their head, part of why our software lifecycle doesn't begin with a single code repository. Nobody upgrades VISTA with a complete snapshot. VISTA's code and data are and must be far too intricately intertwined to simply swap out all the code like most software does. Instead, we upgrade with incremental changes, with patches and new versions of individual packages. At best, VISTA snapshots are useful for starting a brand-new VISTA system from scratch.

Even if VISTA weren't so vast as to be unmanageable as a single codebase, this is the other reason we do not begin the software stream with a single code repository - because most VISTA adopters have little use for new snapshots. Instead of being the source of the stream, the VISTA platinum account (the clean and shiny VISTA codebase from which snapshots are created) is just another recipient of the software stream at the end of one of its delta channels, like any other VISTA system. That is, the platinum code repository is a recipient, not the source, of the software stream. It has to be.

VISTA can only be managed piecemeal, with all its separate threads of user feedback and incremental changes being woven through Forum to create the fabric of VISTA.

Yours truly,
Rick

Point 6: VISTA's Software Stream Has Many Tributaries, part one

Dear Reader,

In the classic VISTA lifecycle, each VISTA development team relies on user feedback in the National Online Information Sharing package (NOIS) on Forum to figure out its top priorities, but the changes are not made on Forum nor on any central code repository, as discussed in point two. Instead, each package-development team develops its changes in its own package-specific code repository. Developers do not need permission to make changes to their software; they are the tyrannical owners of that software, and they continue to be just so long as they keep their users happy.

Once the developers have solved a problem, they bundle it up using the Kernel package's Kernel Installation and Distribution System (KIDS) and e-mail it to Forum. Using another special Forum package called the Patch Module, they convert the KIDS distribution into a patch and distribute it for testing, verification, and eventually release. The Patch Module maintains a list of subscribers for each VISTA package, so when a patch is released it is automatically e-mailed to all of its subscribers. The power of this e-mail-based push mechanism is something we do not have time to get into in this posting, except to say this is what makes the pending auto-patch system work.

Conceptually, this means the software stream does not flow to a single mouth, a single repository where people have to go to find upgrades. Rather, after Forum the software stream splits into a delta that delivers a stream of software to each subscriber.

This explanation should make clear that the patch stream, the nonstop sequence of incremental improvements to VISTA, also does not begin with a single source, but with many sources, maybe between fifty and a hundred. Forum's role on this outbound half of the lifecycle is strictly that of a traffic cop who sequences the traffic, not that of traditional software manager who controls what gets done and whether it can be released. Forum's role here is to act as the point where the tributaries come together to form a single stream.

Although from the VISTA adopters' perspective Forum produces the steady stream of small advances they can trust to continuously upgrade their systems, from the developers' perspective it is still very easy to separate out, think about, and manage just the thread of development that represents their chosen package. This is how the unmanageable scale of VISTA development is broken down into incremental steps organized into manageable packages and corresponding patch streams.

Concluded in part two tomorrow. . . .

Yours truly,
Rick

Point 5: Users and Programmers Need a Shared Forum, part two

Dear Reader,

. . . resuming from part one, yesterday . . .

How to configure Forum and how it works to organize user requests and encourage dialogue are topics we'll spend a great deal of time on in this blog, eventually, but for now I just want to make three observations.

First, someone needs to host this system and supply a VISTA-savvy system manager to run it. This is a responsibility, and it supplies a service that everyone needs, but there is no strategic advantage to being the one who runs it. You can't charge for it without screwing up the VISTA lifecycle, and you can't tamper with the flow of dialogue between users and programmers. It's a responsibility that carries with it shared benefit but no advantage over the other VISTA vendors and organizations. At the moment, the network is testing out the Forum software on its Paideia educational server, and our community could start out by using that until we're ready for a more robust system.

Second, because VISTA is medical software, sometimes problem reports must include patient information to properly diagnose. The only way this is ethical and legal is if all of Forum's users (programmers and users alike) have signed Health Insurance Portability and Accountability Act (HIPAA) agreements to respect the privacy of all patient information they see there. That's only possible if Forum is not open to the world, only to the finite community of actual VISTA programmers and users.

Third, Forum is not an optional or replaceable part of the VISTA lifecycle, though it's the first part that open-source enthusiasts get excited about replacing, usually the moment they hear the words "problem reporting." Everyone wants to use either their home-brewed pet software or else the latest fad that's sweeping the open-source community. Such petty arguments over toolsets are at least half the reason why, eight years after I first proposed setting up a shared Forum system outside VA we still don't have one up and running.

So this time, let's sidestep this problem by starting out with the only hub ever proven to work with the VISTA lifecycle. Later, after the lifecycle is underway and we are all productively developing VISTA we can tinker with our toolset, but for now, I refer you back to the first point I made (five posts ago). Let's try to suppress our urge to tamper with the VISTA lifecycle before we truly understand it.

The shared Forum accomplishes half of the VISTA lifecycle: it gets the development priorities properly set in response to user needs. After that, the second half of the lifecycle is "easy."

Yours truly,
Rick

Point 5: Users and Programmers Need a Shared Forum, part one

Dear Reader,

A user-driven lifecycle begins with teaching your users how badly we need their ongoing input about what works for them and what doesn't in VISTA. This is much harder for them today than it was in 1994 because mass-market software companies have taught them that no one gives a damn what works for them as long as they keep forking over the cash, that the programmers only work for them in some abstract, marketing sense. In our experience, it takes some work on your part to overcome this learned passivity, but the users who break through tend to become persistent chatterboxes, which is what we all need.

Next, we assume the majority of your users' VISTA requests are reported and resolved by the hospital's own Information Resources Management or Health Information Systems personnel, or by your immediate support organization's expert troubleshooters. The vast majority of problem reports are resolved with training, improved documentation, or through making minor local configuration changes to VISTA, and still others can and should be resolved with forward-compatible local extensions to VISTA (class-three code). That is, a user-driven software lifecycle does not require you to air all or even most of your dirty laundry; each support organization handles the majority of its own problems. All of this can be handled through whatever local problem-reporting system you like.

But when it comes to problems that require shared changes to VISTA, problems requiring the attention of the VISTA package-development teams, VISTA problem-reporting cannot require the use of eighteen different systems for eighteen different organizations. We need to use a single, specialized problem-reporting system that's evolved over the decades to support our VISTA development teams, the one they're most comfortable and efficient with, the only one specifically designed to support the VISTA software lifecycle.

At the hub of the VISTA lifecycle is a single system called Forum. Forum is where users and VISTA teams carry on their continuous dialogue about how VISTA is working and not working for each user, where users formally make requests for changes to VISTA.

Forum is a VISTA system. It runs VISTA. Any VISTA system could be configured as a forum system, but we only need or want one to ensure we get all the user problem requests into one place where they can drive our development priorities. Configuring a VISTA system as the forum system mainly involves ignoring all the medical, financial, and administrative packages in VISTA and properly setting up its communication packages. For example, one of the special VISTA packages Forum uses that most VISTA sites ignore is National Online Information Sharing (or NOIS), the main problem-tracking package; most VISTA sites don't use this package, but on Forum it's one of the main packages used.

To be concluded in part two, tomorrow . . .

Yours truly,
Rick

Point 4: Users Must *Directly* Control VISTA's Lifecycle

Dear Reader,

VISTA is too important to healthcare to leave its fate in the hands of managers, programmers, politicians, or entrepreneurs. Really, only VISTA's actual users - nurses, doctors, pharmacists, lab techs, radiologists, and other medical and support personnel - know precisely how they use VISTA second-by-second every day, so only they can really know how it falls short in supporting their work. Of all the people involved with VISTA they are also the only ones directly contributing to patient care, which after all is the point of this entire enterprise.

Therefore, in the VISTA lifecycle model, although anyone can request VISTA changes, user requests trump all other priorities. Directly. Not as interpreted for them by managers, experts, committees, or other bureaucratic forms of user disempowerment. User requests are collected and used directly as the marching orders for the VISTA development teams.

Users are the fourth and most important element of the VISTA model of authority. It is the users who decide what each team works on, and not by sorting through or voting on priorities but simply by reporting the problems they are having and requesting bug fixes or enhancements. As the reports accumulate, the urgent problems identify themselves and do not require committees or expert panels to identify.

Many people find this element of the VISTA lifecycle idealistic, but since all of their proposed alternatives have failed over and over to the tune of billions of dollars wasted, and since this model not only worked for seventeen years but produced the greatest productivity and responsiveness the VISTA world has ever seen, I find the idea of doing anything other than user-driven VISTA development ridiculously idealistic. A properly organized VISTA lifecycle very nearly runs itself, using specialized tools and techniques of information management.

Yours truly,
Rick

Point 3: VISTA Requires Many Authorities, Not One

Dear Reader,

An unimaginable amount of expertise went into creating VISTA, and that same degree of expertise is required to manage it. Managing the VISTA software lifecycle requires more expertise than you have, more than I have, more than anyone has, but it also requires more speed and accuracy than any group is capable of. Any individual in charge of VISTA would be ignorant of 99% of the subjects needed to manage all of VISTA, and in any group large enough to include all the experts needed, 99% of them would just be in the way for 99% of the decisions. There is no form of centralized authority capable of managing VISTA effectively, no matter how you divide up the code into repositories.

That's why VISTA's second name, before "VISTA," was the Decentralized Hospital Computer Program, because no central authority can manage it, no matter how smart or powerful or well funded they are or how good their intentions. VISTA authority must be decentralized.

However, history has demonstrated that VISTA authority also has to be efficient and responsive enough to keep up with the changing needs of medicine, has to avoid the chaos of a free-for-all, and has to avoid the endless debates over trivia to which democratic communities are all too prone. Each VISTA authority must have near-tyrannical powers over their chosen part of VISTA to ensure maximum efficiency. That is, VISTA authority must not only be decentralized, it must also be concentrated.

VISTA also requires a third element in its governance. Each VISTA package is so complex it can only be effectively managed by dedicated expert programmers who can focus on mastering their chosen package over very, very long periods of time, like a decade or more. Most VISTA packages are too complex for any single expert programmer to manage, certainly too complex for dilettantes to manage effectively, so teams are required. And so, VISTA authority must also be expert.

The resulting form of governance, to have authority decentralized and concentrated into the hands of many near-tyrannical, permanent expert teams, is so weird there isn't even a name for it - for now let's call it the VISTA model of authority - but history has proven that whatever you call it, it is capable of managing VISTA better than any other authority structure.

So here's a vital part of the VISTA lifecycle recipe: decentralize authority, divide it up by subject, and allocate it to teams of expert programmers. Each team manages one VISTA package's code repository, with the senior experts in charge of each team.

This part of the recipe is necessary but not sufficient. After all, who is in charge of the senior experts? Who decides what the priorities are? That's the subject of my next post.

Yours truly,
Rick