Edward Berard

Author of a paper and presentation titled "Misconceptions of the Agile Zealots", presented recently at the Atlanta SPIN (http://www.atlantaspin.org or http://www.cc.gatech.edu/SPIN/ver002/Archives/061703.html), Silicon Valley SPIN (http://www.gd-ais.com/sv-spin/Events/schedule.htm), and North Jersey SPIN (http://www.njspin.org/).

See http://www.toa.com (The Object Agency) and in particular his articles page, http://www.toa.com/shnn?searticles. This specific paper can be downloaded from http://www.toa.com/pub/Misconceptions.zip.

In 2000 he posted this to news:comp.object.

After you read several pages of misdirection and insinuation disguised as erudition, you will read 'I am not interested in any more "XP battles."'

So he seems to have reversed his stance, with his recent "AgileZealots?" career shift.

Battles I Cannot Win

"Those who cannot remember the past are condemned to repeat it."

-- George Santayanna (1863-1952), from The Life of Reason [1905-1906], Volume 1, Reason In Common Sense

Introduction

Over the past couple of years, I have watched as interest in Extreme Programming (XP) gradually consumed comp.object. In addition to the informational and question-answering posts, there has been a flurry of zealotry.

Zealotry takes many forms. Some posters felt it necessary to go far beyond merely advocating/prosoelytizing XP. They chose to directly confront not only existing object-oriented technology, but the established base of software engineering in general, human psychology, and management practices.

The onslaught has had a very noticeable impact on the traffic in comp.object. People attempting to have discussions on non-XP traffic, for example, sometimes find their threads taken over by XP issues. Others simply refuse to post, lest they get caught up in the fanaitical vortex. (I have received several pieces of private e-mail on this topic.)

There has always been fanaticism and zealotry in comp.object. Booch advocates slugging it out with Rumbaugh (OMT) advocates and Shlear-Mellor advocates, for example. Language wars were also common, e.g., Eiffel, C++, Smalltalk, Java, and Python. Of course, there were the nitty-gritty detailed threads on the real definitions/values of such things as covariance, polymorphism, and single versus multiple inheritance.

The pre-XP mix of people in comp.object included good numbers of university professors, grad students, methodologists, and gurus of all denominations. People doing real object-oriented projects, who had an immediate need for information, made up a significant portion of the posters.

To be fair, there are at least 2 other major factors that have colored what we see in comp.object:

1. The Internet itself. Until the early 1990s, the Internet was primarily the domain of people who understood technical matters, and who also had the resources to exploit it. These people included universities, people working on government contracts, large telecommunication companies, and technical consultants for hire, for example.

Nowadays, in the "dot.com world," the Internet is a model of cultural diversity. This diversity is wildly amplified by all the new avenues (and cross avenues) for communication. One no longer has to understand the differences between newsgroups, mailing lists, bulletin boards, and chat rooms. You can tap into just about any on-going conversation, from just about anywhere.

Our enriched discussions make it very clear that we "Internet old-timers" can no longer make the same assumptions about our fellow participants as we once did.

2. A branch of sociology called "diffusion research," investigates how a new idea (an innovation) moves through a society. Diffusion researchers study how new ideas are communicated, how they change the society, and how the diffusion (communication) changes the new idea itself.

Diffusion researchers identify the phases of an idea's acceptance, and the types of people involved in those phases. For example, the first on the scene are referred to as "innovators," while the last on the scene (if they ever make it) are referred to as "laggards."

You can get some idea of the phases of an innovation's acceptance by looking at the labels given to the people involved, e.g., innovators, early adopters, early majority, late majority, late adopters, and laggards.

The innovators for object-oriented technology were involved prior to 1970. The early adopters showed up in the first half of the 1980s. The early majority started showing up in the second half of the 1980s. Somewhere towards the end of the 1990s, the late majority began arriving.

Those people coming late to a technology have a distinctively different outlook than those arriving early.

Different Playing Fields, Different Rules

In the not-too-distant past, discussions in comp.object were governed by certain "rules" (conventions really). For example, it was perfectly permissible (and, in many cases, encouraged) to cite literature references to back up your point. This process was an interesting cross between the scientific method and legal arguments (i.e., using precedence to argue the meaning of a point).

Real case studies, as well as data and stories from real projects, were especially prized.

One of the more commonly-invoked rules was "some data is better than no data." This could be trumped using the "more data is better than less data rule." Of course, there were endless discussions involving statistical significance, the context of the data, and what one could and could not generalize from the information at hand.

Another, very important, rule was that you had to work within the confines of nature as defined by other disciplines. For example, we have to accept the limitations of the human mind to learn new things. We have to understand that not everybody can do everything equally well. "Too many cooks" can spoil the "broth" regardless of the context.

It is also important to realize the value of particular activities, tools, and concepts. E.W. Dijkstra and Harlan D. Mills both advised us that designing programs correctly was far better than testing and debugging a program into correctnes. This lead to another rule: don't be sloppy now hoping that your sloppiness will be caught later.

Dilbert's boss ("Mr. Twin Peaks") gave us an "anti-rule" (a rule in reverse) when he said "Anything that I don't understand has to be easy." This told us again that we had to respect existing bodies of knowledge and experience. If we were not willing to investigate a methodology, or a discipline, for example, we should be very careful in our assumptions about that item.

Sir Isaac Newton (1642-1727), in a letter to Robert Hooke, reminded us that "If I have seen further (than you and Descartes) it is by standing on the ShouldersOfGiants." This taught us to first investigate what is already known or in place, before setting out in a "new" direction. Doing this would either give us a "head start," or show us that we did not need to reinvent the wheel.

The abundant diversity in the object-oriented world also made us aware that the "playing fields" were not all the same. For example, depending on who your were talking to, a "class" could be a simple template, a template plus a creation mechanism, or a set of items constructed using a particular template. Some people talked about "classes and inheritance," while others sought out "exemplars and delegation." Types and classes are the same thing for one group, while being fundamentally different for another.

One of the more interesting and important playing field distinctions, with regard to object-oriented software engineering, is that among the artificial intelligence sector, the Smalltalk sector, and, for lack of a better label, the C++ sector. (Eiffel, Ada, Java, and other programming languages are all used by people who could be said to fall into this sector.)

"Software Engineering" in its most exacting form lives in the C++ sector. The denizens of this sector typically value metrics, methodologies, engineering knowledge and disciplines, technical literature, and formalized approaches in general. They strongly tend to value science and engineering over "magic." For these people, it is not enough to do something well. You have to know that you did it well.

Lord Kelvin (William Thomson), who lived from 1824 to 1907, speaks for the C++ sector when he says

"When you can measure what you are speaking about, and express it in numbers, you know something about it; but when you cannot measure it, when you cannot express it in numbers, your knowledge is of a meager and unsatisfactory kind: it may be the beginning of knowledge, but you have scarcely, in your thoughts, advanced to the stage of science."

When he came up with FLEX, Alan Kay was looking for a computer for the rest of us, i.e., a computer for non-programmers. He carried this idea forward to Dynabook ("Dynamic Book"), and then to Smalltalk. Ideas like a simple language syntax and semantics, a graphical user interface, and the mouse could all be combined to bring the computer closer to the masses.

It is no accident that many Smalltalk advocates see little value in formalized methodologies, metrics, and other artifacts of the C++ sector. Since it is so easy to quickly create and modify applications why would anyone bother with some rote process? As knowledge of object-oriented software engineering became more widespread, Smalltalkers were mercilessly harassed by people asking them "what methodology to you use?"

When Rebecca Wirfs-Brock and her cohorts came up with "responsibility driven design" (RDD) in the late 1980s, she provided some relief. Now, when asked about one's choice of methodologies, Smalltalkers could (and did) say "Wirfs-Brock." Kent Beck, Ward Cunningham, and others contributed items such as class-responsibility-collaboration cards to the Smalltalk arsenal.

To be sure, those in the Smalltalk sector can become very nitty-gritty technical when discussing such things as typing, the mathematics of inheritance, and reflective systems.

The terminology, concepts, and approaches found in the artificially intelligence sector differ significantly from those in the other two sectors. The Common Lisp Object System (CLOS) boasts full-blown metaclasses (classes whose instances themselves are classes), for example. The word "class" also takes on a different connotation, meaning the set of all instances of a pattern.

The artificial intelligence sector is more free-form than the Smalltalk sector. Progress is usually made in a heuristic (experimentation, evaluation, or trial-and-error) manner. Methodologies and metrics are remotely important, if at all.

The Arrival of the XP Contingent

Extreme Programming came from the Smalltalk sector. Like RDD ten years earlier, it is an approach that comes from the programming language level upwards -- as opposed to many methodologies that arise in the C++ sector which, almost exclusively, work from high-level concepts downward. XP is unapologetically code-centric.

Rather than evolving it from a more complex methodology (e.g., the Rational Unified Process), Kent Beck took what he knew from his decades of software development experience and crafted an approach that would greatly amplify what he considered good programming practices. If you understand the Smalltalk sector of the object-oriented community, many facets of XP will look and sound familiar.

Kent is very concerned with the human element in software development. So, there is a huge dollop of people-related issues in XP. Some of these people issues are fairly tame. For example, XP advocates keeping the number of hours worked to a reasonable level, e.g., 40 hours per week.

Some of the XP people issues do not initially make much sense to people in from the C++ sector. Specialization of skills to the point of creating a new job category is not allowed. There is also a lot of talk about intelligence and job descriptions.

About the only way I can begin to describe these (to me) very strange ideas is this. Somewhere in the past, somebody came by and disparaged coders, i.e., people whose primary job function was writing code. In a reaction to this abuse, several things happened:

* Non-coding activities became trivialized, and were considered less important than coding. So, for example, analysis and design activities that had non-code components pretty much disappeared. The products of these processes were mocked as "pretty pictures." Software testing was something that anyone could learn, and anyone could do well.

* Horror stories, not always untrue, of documentation for documentation's sake became almost a mantra. Another commonly- recited point was that non-code documentation always gets out of date, so there is no sense in producing it (or more than a very small amount of it) at all.

(Apparently, one is not supposed to point out the differences between a bad idea and a bad implementation of a good idea.)

* Many (most?) aspects of one's job description became intelligence issues. Attempting to take away any activity from a programmer was viewed as saying the programmer was not intelligent enough to perform the activity. Issues such as having too many different things to do, lack of demonstrated talent or desire, or mindset conflicts were all ignored.

* Classic software project management became daemonized. Self management (or emergent management) was the new norm.

You could say that very few, if any, restrictions were placed upon programmers.

Another confusing (to me) situation involves XP advocates using terms found in the C++ sector, but with different definitions, and without any metrics. For example, there is a lot of talk about "quality." However, when asked for definitions or metrics to support the quality claims, XP advocates are not very forthcoming.

There are other points of contention that I have with XP. However, in my discussions, I have become acutely aware that we are playing by two very different sets of rules, on two very different playing fields.

It is not that my values and my perspectives are right, and XP's are wrong. It is just that we are so far apart on the issues that it would take forever to get anywhere close, and that is assuming that that is a desirable goal.

Adopting XP in its current form, goes against most of my training, research, and experience. It also conflicts violently with my training as a scientist. Further, XP can evolve and improve on its own without any "assistance" from Ed Berard. Thank you.

People like Topmind and Elliot see themselves as crusaders, fighting for a cause. While I do crusade, I have no vested interest in, nor desire to see, XP go away. In addition, commenting (positively or negatively) on XP does not further my career, nor does it put any money in my pocket.

Kent Beck is very right. We are talking across a paradigm gulf.

I am not interested in any more "XP battles." If you want to say that XP is righteous has defeated the evil Ed, be my guest. Now, and in the future, I will make every attempt to duck XP questions and any references to XP technology.

I will be glad to discuss my views on software engineering in general, and on object-oriented software engineering in particular.

Let the celebration begin!

-- Ed

In 2000 he posted this to news:comp.object. After you read several pages of misdirection and insinuation disguised as erudition, you will read 'I am not interested in any more "XP battles"'. So he seems to have reversed his stance, with his recent "AgileZealots?" career shift.

That's interesting background on him, thanks. If I'm not mistaken, he started news:comp.object (?). I don't monitor it ... has XP continued to dominate over the past 3 years, and has Ed resumed commenting on XP there?

For those who haven't yet read the paper, it covers misconceptions about agile advocates ("anti-agile misconceptions") as well as misconceptions by agile advocates ("agile zealot misconceptions"). The stated goal of the paper is "not to discredit agile methods, or their advocates. Rather, the goal is to recalibrate the on-going exchange of technical ideas." I read his paper in that light, all 110 pages, and found it thought-provoking. (About half of the paper is references.) YMMV.

Kent Beck is very right. We are talking across a paradigm gulf.

In the past year, I have found a lot of presentations and papers by people (even from the SoftwareEngineeringInstitute) looking harder for compatibility than for incompatibility. I don't consider myself an expert on the same level as KentBeck or EdwardBerard; I'll readily admit that I still have a lot to learn and experience, and I definitely expect my opinions on these matters to evolve over time as a result. Right now I'm personally not convinced that the paradigm gulf between agile and everything else is so large as to be unspannable. I visualize the bridge as a spectrum, shades of gray as opposed to black & white. (Or rainbow, like the CrystalMethodologies?) I'd love to hear others' thoughts on this.

There's probably a more appropriate [existing?] Wiki page than this for a general discussion of the ParadigmGulf? [ComparingParadigms, ExtremeParadigmShift, MixingParadigms, ParadigmShift, SoftwareDevelopmentImprovementParadigmShift, TimeForaParadigmShift ?]. Anyone have a recommendation, and/or care to refactor?

-- KarenSmiley, Sept. 19, 2003