The Received Methodology
The Received MethodologyTrantor Development Methodology and Revision Control
There is plenty of material out there in the world about how software *should* be built. There is plenty of material out there that is a postmortem analysis showing why things went horribly wrong. There are mountains of material showing the redacted version of how successful builders created things. However, except for anecdotes from some exceptionally humble and honest programmers (a rarity, but they exist), there is little available that speaks clearly to how most successful software is actually written in real life.
Despite all the advances over the years, as of 2009 software development still remains something of a black art. Transforming requirements and associated algorithms into software is still labor intensive. It involves much trial and error. The only proven way to develop working non-trivial software is to build, test and repeat. However, simply building a thing and testing it becomes effectively impossible as the size of the system increases.
The only way to minimize (but not eliminate) the impact of increasing size is to become more disciplined in reducing linkages between sections of code so that source objects, 'strings' of those source objects and the subsystems they compose can be tested separately without a complete test of the system. Testing at each point is critical and the sooner done, the better. Testing as soon as feasible mitigates the damage caused by failures.
Our formal 'methodology' follows the protocol below and is consistent with industry practice. Even with a formal development method, there is much re-design and re-write prior to delivering reasonable working software. Any novel, non-trivial software that actually performs a useful function must come as the result of a similar process. This process involves iterations of specification, design, development and testing.
Many here have had this experience, but I submit that it is intrinsic to software development generally.
Trantor’s methodology has the following stages:
Without our graphic, it is a little difficult to picture, but for each stage and sub-stage in the process, the existing work product is tested and sent back if it fails the test. It's status in the prior stage is examined and if the defect appears to be there, the work restarts there. If the defect does not appear to be there, then it is sent back a further step. This goes all the way back to the end and if the original rationale does not hold, the system is retired.
The cycles happen in repeated iterations many times. It is actually possible for something at the end of 'RELEASE' (actually deployed in the field) to get sent all the way back to 'INSPIRE'. That means one could build an entire system only to scrap it shortly after release into production. Does that happen in the real world? Yep. It may happen as much as 25% of the time or more with big systems. [See http://www.cis.gsu.edu/~mmoore/CIS3300/handouts/SciAmSept1994.html]
What I have documented here simply reflects what actually happens on projects that deliver working systems into production. If you work without such a plan in mind, you will be constantly surprised. Timelines will surprise you; budgets will surprise you. You end up proceeding more or less as above anyway, or you will not deliver a proper working system.
It is my opinion that communications documents (letters, etc.), archival sources and test data and working software embodies the best record of its own design and development. To that end, the use of a revision control system is an excellent addition to the development process.
A revision control system can help at every stage by capturing the actions that were taken, by showing the context within which they were taken and showing why those actions were taken. In our graphic (not shown here), just about every polygon should generate some kind of material that goes into revision control. In fact, if one were using a formal workflow system, even the lines would generate some storable item. It is implicit in the process that there must actually be tests and supporting material for how to do the tests, what was actually done and what happened.
It took me literally years to organize this simple overview of the process. Similarities of the stages reflect genuine similarities, but each item is subtly different as well. Other documented methodologies still either conflate different things or fail to make similarities apparent.
Although we follow a strict formal methodology, it is one that reflects that 'messy' business of software development realistically. It is not always inspired by something reasonable. It is not always a good idea, but will often proceed anyway. During early analysis, many things get changed. What may have been a poor idea can morph into a good one. Some projects that were ill-conceived at the outset will continue to move forward, even though most of the people in the trenches already know it is doomed. Our methodology is designed to minimize that from happening. Other methodologies have a built-in resistance to returning to prior stages. In some, the initial ill-conceived plans and designs are contracts and they doom the project to failure by disallowing fundamental corrections at a later stage.
At each stage moving forward, things start in a very immature state, have sketchy documentation (or none) and they change at each step. In the real word, systems rarely conform to the original design documents and in fact, they generally do not even conform to the final design documents once they are finished. Nearly all real-world documentation for deployed working systems is out of date. That is because unless there is an error in later stages, there is no will (or budget) to revisit the earlier stage.
Development becomes more and more defect driven as time goes on. This is because as the system grows, it begins to 'canalize' into the shape it takes on. It becomes more difficult to change earlier decisions as more and more later things depend upon those. There are, however, exceptions to this. Sometimes, something will become apparent at a late stage that either shows a much more elegant solution that is worth pursuing or (more often) a design defect as a result of an incorrect assumption will force redesign. I once worked on a project where the team had decided to implement using C++ templates when the compiler was slow to build using them. It eventually got to the point where the system effectively could not be built. The decision to use Templates like that was abandoned at a late stage and it required a significant effort to recover.
The fact is, whether they want to admit it or not, developers of working systems have a lot of false starts, iteratively return to the same ground many times and use whatever they have at their disposal to get the job done. That sometimes involves stopping to build tools, sometimes involves extensive manual efforts and always requires ongoing revision.
I'm at a total loss to explain how this differs from the WaterfallMethodology, which we know doesn't work on any reasonable scale requiring rapid time to market.
I have been trying to figure out where this fits in. How about the following. That looks to be reasonable as there are various pages on WaterFall in the same category. -- JohnFletcher
My bad, I suppose. It is not nearly as clear without the drawing. The stages do not fall through inexorably to the bottom. They could go back to the terminus and stop at any time. The goal is to reach pilot as soon as you can. It differs entirely from WaterfallMethodology. The Waterfall lives or dies at the end and because of that, it generally dies. I don't know that I have ever really seen a successful project that was a true Waterfall.
As I stated, the above is how I have observed *successful* projects deliver software from small-ish one coder 30K line products on up to $250 million dollar projects. One could quibble about semantics WRT the names of stages, cleavage, etc. However, I have given considerable thought to the above breakdown and it is still true to what actually happens.
Here are some hallmarks of the above:
It is, in practice, impossible to spec non-trivial software, in detail, up front. Until you build and deploy something, you really don't know for sure what it is you are actually trying to do. Non-trivial systems do a lot of stuff. It depends and depends and depends on so many things, it might well be unpredictable even in theory. That is, the empirical exercise of building and trying the software may be the only way to build non-trivial software that works reasonably.
What you *can* say about software is that there was a reason it was started. Somebody at least took a stab at justifying and characterizing it, some sort of design took place (if even just on a napkin), some sort of sub-system design took place (if only on another napkin), somebody coded little things and put them together into larger things, somebody built the components and then used them to build the whole, somebody tested it in some fashion, users passed some sort of judgment on it, it was released to a small pool at first and a larger one later. In general, those parts build upon one another, so they happen best in sequence. You can do them out of sequence a little, but to the extent you do, it usually does not work well for you. At each stage, there is always at least a little fall back to a prior stage. If done well, the fall-back is anticipated or even welcomed as an opportunity to fix things at the stage where the fix has the best impact and is less costly.
Somebody very good, with a lot of experience, working on a small tool might complete the whole thing above in a day or two. They could 'sort of' skip steps by doing stuff in their heads or going with a 'big bang' test of the entire system and skip unit test. However, that is still likely sub-optimal and only a very skilled programmer could pull it off. [As an aside, a lot of problems come from very talented journeyman programmers whose short-term memory is at a peak. They write code with brain-buster routines and macros that even they themselves are unable to understand at a later date. If there are bugs (it is amazing how programmer's continue to delude themselves that they don't make mistakes), they are sometimes more time consuming to find and eliminate than it is to simply rewrite properly (like, less brain-busterly) from scratch).]
Regardless of the skill of the team, if the project is a large one (even 100K lines, I would say), it will conform to the steps above, pretty much in that sequence. To the extent that it deviates, it will produce an inferior product dollar for dollar, and in my experience, they generally just fail outright.
To the extent that I am aware of the process involved in building UNICES of all flavours, compilers, languages like PHP, Perl, C/C++, Java, other operating systems (I was involved with beta testing and building applications with OS/2 early on, for instance), banking production systems, telecommunications production systems, small department specialty applications, vertical market applications, etc. This is how it happens. That may not have been the plan, but that is what actually happened.
I have seen a few 'post hoc' accounts that speak to a different method of development, or perhaps to more certainty going forward. I have worked on a couple of ISO 9000 projects and despite the nice reports showing steady forward progress, the situation 'in the pit' was decidedly different. I have also worked all the way through enormous projects that followed a stiffer process where going back was vigorously constrained. These were released to production and faired badly. The developers got paid, the users signed off and it went into production, but it was *not* pretty.
I will not name names, but in the early days of OOP, I reviewed a $250 million dollar OOP project using (well, I will name this name) Rational Rose and whatever OOP project management regime was in vogue at the time. More than one major vendor was involved in this project. Fifty million dollars into the project, I asked everyone involved (I reviewed the entire project) to show me what they had. The developers, in particular, with the best tools available, after consuming tens of millions of dollars were unable to demonstrate a single working subroutine. They could not fire up even one single thing and show it do anything on a screen. Nothing. Nada. Zip. [Note that it was on NextStep x86 boxes (OMG) and the GUI was a major part of the deliverable. They had nary a dialog to display.] Just about ten million dollars later, they canceled the project.
I have obviously not made it clear, but the methodology above produces demonstrable results *well* before consuming a million dollars. The OOP project named above was doomed at the outset. However, had they followed the working methodology above, they would have discovered it was doomed long before consuming $60 million dollars. The methodology above, allows or even demands that the INSPIRATION remain in place because it is constantly being revisited.
Think about it from another point of view. If you follow the method described (admittedly not well) above, you will discover at the earliest possible opportunity if something fails. You may not know for sure that the thing that 'passes' will go all the way to production and stay there, but you know for sure that the thing that fails ain't goin' nowhere. There will always be some 'gotchas' in production. Even mature systems get nailed in production. That means they have to go back to whatever point is required to fix the problem and rolled forward. Rolling forward is a heck of a lot easier if you have proper design, build, test and deploy regimes in place. If you follow the method above, you will as a matter of course. A passed regression test does not guarantee success, but a failed unit test guarantees that you either fix that or at least know it is an 'issue' at release.
What is represented above is the distillation of 'what is done that works'. One need only examine the history of major software and source code repositories of working software to see that this is basically what happens. To the extent that artifacts are left behind, they appear to support the notion that the stages above took place. Things like FireFox are the culmination of such a process over many years starting with and Mosaic and its antecedents.
One thing in particular, that is different about the above is the primacy of the 'INSPIRE' stage. This is arguably the most misunderstood, underrated and misrepresented stages of software development. Years before even Mosaic (predecessor of the predecessor of FireFox) was built, Tim Berners-Lee provided and kept alive the 'inspiration' that led to the World Wide Web and the birth of the modern browser. It began with hypertext as a solution to the problem of navigating through information. A working hypertext system existed for many years before Mosaic was given the go-ahead. It appears from the record (and makes sense to me) that the inspiration, the analysis/rationale and creation of requirements began a very long time ago and culminated in a series of iterations through to Mosaic. By Mosaic, it is fairly clear to me that Mosaic was a finished product, but that it also fed right back to the beginning of the cycle and that culminated in Netscape, which inspired Internet Exploder and the race to where we stand now with IE7/8 and FF2/3. It seems to me that FireFox has a significantly more sophisticated development and release cycle than the original text only hypertext browsers.
I might also note (again, it is not well expressed above) that as the iterative cycles unfold over time, the various formal aspects 'tighten up'. Initial rationale and requirements may be quite ethereal. It often happens (in corporate environments at least) that there is more of a 'vague notion' than an inspiration and more of a drive to justify after the fact rather than a true rationale and requirements are driven more to justify moving ahead than to really describe what is being done. As things move forward through prototypes, feedback to earlier stages, etc, the whole thing 'gels' (or does not and dies). Something successful in production enough to live another day will go back to have better specification, a deeper and more meaningful rationale, an inspiration bolstered (and clarified) by initial success and will have stages 'tighten' up as the repetition of tasks spurs forward greater specification, automation and understanding.
I hope others will weigh in here to help me (by responding to criticisms or requests for clarification) to more clearly express the notion of the 'received methodology'. I call it 'the received methodology' because I don't really take credit for inventing anything here beyond tidying up the process and describing it. It already exists in the wild on its own. It is 'what we do'.
There is, I think, some merit in formalizing some of the stages so that work is easier to characterize and carve off and so that developers do not have to make apologies for the real-world fact that things generally are not 'first time right' in the meaningful sense of being 'the best' or surviving in the marketplace without revision.
Things like 'pair programming', 'design patterns', 'RAD', 'Use cases', XP, etc, all have their place within this framework, if only because they *DO* happen and this is a description of 'what is' more than a description of 'what ought to be'. I would say, though, that empirically, since this seems to be what creates the world's working software, it is near what 'ought to be' as well.
I hope this does not sound defensive and that people will jump in to throw in their two cents. After three decades of development, I am quite convinced that this is a good way to view things. I welcome the opportunity to help correct misunderstandings, to argue the case where people have good points and even *gasp* make alterations to reflect valid criticism.
Fix Bonehead typo in Page Name. --BobTrower (AKA Bonehead)
EditText of this page
(last edited April 3, 2009)
or FindPage with title or text search