The Mythical Man Minute

TheMythicalManMinute is an article I wrote about SoftwareReuse.

It is about the problems, benefits and the implications of reuse. The main point is that a change of thinking is needed for effectively facilitating reuse.

The article is only in GermanLanguage till now (sorry!) (Maybe some day somebody might help me with translation of the content - for now, I don't think my English is good enough and my time is too limited). But I am sure that also some German reading people are here.

The translated full title is: "The Mythical Man-Minute: Mastery of Software".

Here is the original text:

http://www.2CentsOfWisdom.de/dev0005.html

For those, who are wondering about the title - yes, it is referring very much to The MythicalManMonth by FredBrooks.

-- JuergenLindemeyer

Hi all contributors!

I am really overwhelmed by the fast engagement of the WikiZens who started this translation project. I really appreciate that!

Of course I will look over the text and clarify -- maybe some comments will stay for a second roundtrip -- but I am sure, that we will get a good translation in the end!

Some remark about copyright: I am actually not sure, which copyright to use for this text (since I am not sure if I want to use it in a book).

If you want to OpenSource it, I suggest using <http://creativecommons.org/license/> to choose a license.

But I hereby grant permission to this Wiki to publicize the translated text in the Wiki. For others: Please ask for permission.

-- JuergenLindemeyer

Maybe we can run it through an automatic translator, and then various WikiZens can tune it up bit by bit. Machine translators generally don't do a very good job by themselves.

A hand-tuned version follows. Tuning questions / comments are in curly braces {}, the original author is requested to clarify.

The Mythical Man Minute: Mastery of Software Feb 17, 2005

How does the micro level of software development affect the macro level? Or: How can software development be made more cost-effective while yielding better quality software?

On the Search for the Holy Grail

Software development is time-consuming and error-prone. Often a software project lasts longer than was estimated, for example, the notorious case of Germany's "Toll Collect" system. Also in the USA, there have been substantial financial losses. For example, in Denver, Colorado, around $3.2 billion was lost because a fully-automated airport luggage system was operational 16 months later than predicted.

For years, in software development departments we have looked for the solution to one question: How can software development go faster, and at the same time the quality of the software be improved? Some believe we can achieve the goal with better programming languages; others think that software production must become automated. Neither camp has yet demonstrated significant success. Frederick Brooks wrote in "The MythicalManMonth" that there is no SilverBullet for the acceleration of software development.

There is still another camp which holds that not only can software development become faster, but that the quality of code can be improved at the same time. This camp states that by reusing program code, substantial savings can be made.

The principle is immediately obvious: code that can be used a second time, but doesn't have to be written again, costs nothing the second time.

If it were so simple, however, then everyone would do it. Maybe this is why, for most software development departments, up to now systematic software reuse has been more dream than reality. [ReuseHasFailed] [AreBusinessAppsBoring]

But, why is it like that?

Myth or Reality?

First, however, a small diversion into other areas of the economy. How are costs reduced there?

Car makers, for example Volkswagen, point the way. A car consists of thousands of component parts. It has been a long time since every component was made for exactly one model of car. It is not only engines that are designed to be used in several car models. For years, car makers have had a strict model strategy, where ever more components are reused between models. Volkswagen goes particularly far with its platform strategy. The Golf platform is used in many different car models: Volkswagen Golf, Audi A3, SEAT Altea, and Skoda Octavia all use the same base components. Also, the Passat platform is used by several automobile brands.

The reason for such strategies is simple: reliable components, which only need to be developed once, reduce development expenditures and improve the quality of the resulting products. Because frequently used components can be released more deliberately and action taken to eliminate weak points, the customer also profits from this standardization.

The construction industry is even older than the automobile industry, and we also find reuse here. In the building industry individual components are developed from scratch only rarely. Instead, finished doors, windows, roofs or even complete sets of structural drawings are used.

Even the ancient construction industry continues to improve itself. Building methods are still being steadily improved. While thirty or forty years ago one had to build with relatively small bricks, today it is more common to stack large, standardized breeze blocks on top of each other. Prefabricated buildings are built up from walls produced entirely in advance. Everything is oriented to saving time and work -- with the additional benefit of uniform quality of the final product.

Of course, there are also individualized houses, but when you look closely, you see that this individuality is made from many small standardized parts. That also makes sense, because who would want to relearn how to use a door every time they entered a new house? But we still often see such things in software, where the same task is often presented to the user in quite different ways.

In some countries, particularly the UK, modern building methods are often blamed for uninspiring, uniform housing ("ticky tacky boxes"). Really it isn't so much these techniques, but rather a lack of imagination in applying them on the part of housebuilders.

The Devil Is In the Detail

Reuse of previously developed things is worthwhile in itself; these examples from other industries show that impressively. But, why does the software industry lag so far behind?

If you try to reuse software today, you encounter many problems:

How do I find software parts that I can reuse at all? How do I find the correct (reusable) solution to my current problem?
If I find something, it is often not reusable for me, because it doesn't fit my problem 100%.
Even if it fits, does it have sufficient documentation to reuse?
Is the quality of the software component good enough that I can use it in good conscience?
How will this component be maintained? Who is responsible if a component which is used in five programs has an error?

Many development departments that have tried software reuse are confronted with these problems and more, and sometimes they throw in the towel, because reuse of software doesn't seem to work.

Whoever is true in the small ...

But it can work. Many developers have the experience that they can use functions of libraries again and again and save significant time. Also, the well-known C runtime library, which was a big part of the success of the C language, is a good example of reuse. Problems encountered by every developer, such as copying a string, are easily solved for all time, and can be used by innumerable programs.

This is not a convincing example. There are well-known security (BufferOverflow) problems with the string handling functions of the standard C library, which often lead each developer to roll their own versions of these functions. Standardized alternatives, such as strlcpy from OpenBsd, are not yet in common use. Ok, the example is somehow weak, I admit that, but it would be not easy to find a both well known and also convincing example. The C library is the most well known example I can guess for now. But this problem very good shows an other point of me: The devil is in the details. To really produce top reusable things is really, really hard! The most basic thing is the design. strcpy is broken by design and can not be repaired, because the interface must stay the same. Many reuse efforts are having this problem: Weak design (because of wrong people, bad education, to less time, ...) and this weak design undermines the whole software build on top of it. Ok, I could use strlen as better example --> but the learning factor of strcpy is compelling in my opinion. This is not the place for a flawed example. If you want the reader to believe that it is possible to solve a problem for all time, you must use a good example. It would be a perfect example for an expansion of the "whoever is true in the small ...is true in the large" idea, but add a small section to complete the aphorism: "and whatever is broken in the small ...is broken in the large". It shows perfectly the need to make the lowest levels of re-used software absolutely unbreakable and foolproof.

Many developers understand that the solution to a single problem should ideally be resolved in only a single place. One of the basic principles of ExtremeProgramming is OnceAndOnlyOnce. The proponents of this development method understood that knowledge about the solution of a subproblem should be focused in only one place; if the problem changes, then you only have to change one place.

The usual practice in software development is unfortunately different: even the smallest problems are solved repeatedly. Partly by different programmers, but also sometimes by the same programmer, because awareness of this fundamental concept is missing.

Here, basic knowledge from database theory could be quite helpful. One of the basic principles in relational databases is that the same information is to be recorded only once in the database, because otherwise you can end up with inconsistencies. Further, precious storage space can be saved this way. This procedure to avoid double storage and other anomalies is called "normalization".

Good software professionals use this same principle in programming. The "normalize" their programs in such a way that the cost of program changes due to changes in the problem definition is minimized. In the context of ExtremeProgramming this is called ReFactoring. However, what happens in reality is a limited form of reuse. Instead of inventing the same wheel in many places, it is implemented correctly in one place, and this wheel is used again and again, and becomes continuously better.

{Note: the following and previous subheading come from the German aphorism "Wer im Geringsten treu ist, der ist auch im Großen treu, und wer im Geringsten unrecht ist, der ist auch im Großen unrecht.", which means something like: "Whoever is true [loyal, faithful] in the small, is true also in the large, and whoever is wrong in the small, is wrong also in the large." I'm not convinced that is actually correct, but anyway...} {That's right. The origins of the German aphorism are still deeper - it comes from the bible verse Luke 16,10. That's the reason why I tried to fix the first version of the translation by using the English bible verse (I think it was a rather new translation), which I tried to modify in such a direction that it fitted. The actual version lacks the human factor in my opinion. The German aphorism is about persons - this version does not address persons, as much as I can see - I intented to address the developers with this aphorism - as source of reuse or non-reuse. Small vs Large was originally meant as substitutions for "details" and "macro level" ... I hope, that clarifies matters a little bit} {Then, we can simply change "whatever" to "whoever". "Whoever" means we are talking about people, and "true" in this context means loyal and/or faithful.}

... is true also in the large

What can work so well at the micro level, if the right developer sits in front of the program code, could also work on the macro level.

If you build all small subcomponents in such a way that each problem is solved in exactly one place in the code, and thus always in the same way throughout the program, then is is also possible to assemble larger components similarly. As in nature: the whole world is assembled from a small number of subatomic particles, which form a number of different atoms, and from those cells form, which organize themselves into complex things like plants, animals, and humans. In precisely the same way, software can consist of the smallest components inside ever larger components.

Indeed, it is already like that today; in the end, our software consists only of bits and bytes, thus there are very few primitive elements. But, above a certain degree of abstraction, the software world suddenly becomes terribly complex.

What went wrong?

In nature, we can see that the same construction principles are used over and over. But in software development? There new construction principles are used over and over, and therefore the complexity rises so fast that above the level of the programming language, things hardly fit together.

We have missed the opportunity to transfer the micro level reusability to the macro level because we did not pay consistent attention to reduce the number of construction principles (or concepts) and/or coordinate them.

If you wanted to reuse a program module today, it would usually fail immediately because it is incompatible with another program module. On the atomic level, the modules are compatible, but then so many incompatible levels interfere that this minimal compatibility no longer helps.

Instead of building "compatibility", and thus reusabililty into our software from the beginning, we create a scrapheap of modules, and in this chaos despair to find two software parts that can be used together. {The metaphor is just as mixed in German ;-} {I would suggest to use chaos or clutter or even confusion to unmix it -- I think that would be a better translation for "Durcheinander" here -- I twisted things around, because "Durcheinander" was before scarpyard.} ''{"A chaos" doesn't make sense in English. I am sure the new wording conveys your original meaning - a scrapheap is where we throw old cars, appliances, etc when we are done with them. The metaphor is different, but no longer mixed.}

A New Way of Thinking

This is exactly the problem with reuse: many think that reuse means finding by chance a suitable part that someone else has already built. Reusability of software is fact a consequence of building a software system with a consistent structure, as with the good old Tinkertoys {in the original: Fischertechnik} construction system where all parts fit perfectly into one another, because they were designed to do so through a great deal of creativity and work.

{Fischertechnik is an educational construction kit for children, basically a German equivalent of LegoMindstorms or Meccano; see <http://www.fischertechnik.com/>.} {the main point here is, that Fishertechnik is physically totally incompatible to the basic Lego system -- I don't know, if LegoMindstorm? is compatible to Lego, I guess so. Also Fishertechnik is much more advanced (and more expensive) then the basic Lego system. I am not sure, if the comparism to LegoMindstorm? (I just know it to less) is that good, because the basic Fishertechnik is not about electronics.} {Then, you could use the examples of Tinkertoys or Lincoln Logs for the US. These are fairly well-known, but completely incompatible with each other. Legos are also well-known in the US, more so than Lincoln Logs}

Nobody would think to give their child one part from a Tinkertoy system, one from a Lego set, and other parts each from a different manufacturer. It is clear to everyone that this would only lead to frustration. But, in the software industry, we apparently think we can carry on in this way.

We call the sum of the software concepts in a program its software architecture. In the architecture of software lies the key to reuse. <http://www.informationweek.com/708/08iuhid.htm>

But, who is ready today to invest in software architecture which begins with the details and continues up to the macro level? Where are the software architects who are able to construct software from the bottom up, such that everything fits smoothly together?

Macro-level reuse dictated "from above" is not the solution, because the impedence mismatches ("frictional losses") of adapting existing components to each other are large, and usually the awareness of genuine reusability of software is missing in individual developers.

Unfortunately, this awareness is also missing in managers. Frequently, developers are evaluated based on the number of code lines they can produce, rather than on conceptual points of view. How the program will behave in five years or how much additional expenditure it will consume in software maintenance is only very rarely noticed.

The result is often that we are surprised why a successfully running project ends up thrashing and so much additional expenditure and maintenance arises. Even the most modern software - naturally in the newest programming language - turns out to be an unexpandable dead end that devours enormous resources.

For these reasons, reusability of software can only prosper where a strong management commitment exists, and where all coworkers are conscious of the software architecture. Further, we need softwar architects who know how to build software architectures with uniform conceptual details.

But, the current trend goes straight in the wrong direction, because in place of experienced software developers, many companies look for "flexible" young developers who can work a lot of overtime. The fact that the software developed during this overtime will soon cause problems is never clear on a financial chart.

Mastery of Software Development

This article can offer only an introduction into this very complex topic of "reusability". I hope nevertheless that this article can set a re-orientation process in motion with some readers. Frederick Brooks was right: the "silver bullet", a magic technology which solves all our software development problems for us does nbot exist.

Instead, we must begin to take the details of software development seriously and to learn them anew from the bottom up. We must accept the fact that we can only reuse that which is reusable. We must also understand that we will achieve genuine mastery of software development only if we learn first of all to develop software really masterfully. This mastery must begin with the smallest components and continue up to the most complex modules.

Final remarks for the second round:

Looks very good to me now!

It seems that there are still some open points left by contributors. I added some further remarks from my side - some of the cases I can't decide, because I am lacking the English expertise.

Especially the two subheadings seam to me very problematic still. I agree that they are very difficult to translate because it is based on a German aphorism. But maybe somebody has an idea how to come near to the original meaning. Those subheadings are carrying their own message to the reader.

My new comments are inside brackets but without italics to be distinguishable.

-- JuergenLindemeyer

CategoryPaper