To Ayoung Extremist

KentBeck sent this note to a young extremist. She asked about the coding and partnering experience and how to communicate to her partner what she foresaw:

I have this vision of how I want the objects to be and what they need to do - and I don't seem to be communicating it very well.

Here's Kent's reply:

Perhaps the vision is the problem, not the communication. I will explain by analogy.

When you sit ZaZen (the Zen mediation technique (I don't know how much you know about stuff like this, so I'll make little explanatory notes until you tell me to stop)), I am told that bizarre things can happen to you. You can get sudden bursts of psychic powers- precognition, far-seeing, telepathy, etc. Lots of people would think that was cool. They would hold onto these powers (if they could).

Zen teaches exactly the opposite. While it is true that the mind is capable of many things that we don't understand, it is far more important to get behind all of these experiences to the true ground of existence than to play with a few more mind toys. If you focus on the visions, you will lose track of the search for the essence.

By analogy, we are embarked on a quest to find the simplicity and commonality of experience. Because we want to get paid, we generally restrict ourselves to finding the simplicity and commonality of commerce. Fortunately, commerce is a close enough reflection of humanity that it is still a fertile and challenging field.

When we program, we are always engaged in a struggle to truly see. If we see clearly- what the users want, need, and fear; what our co-workers want, need, and fear; what we ourselves want, need, and fear; and finally what the system wants and needs (damn good thing computers can't fear!)- we work well and truly. Our systems are simple. Our thoughts are simple. We work without stress, without fear, with joy and love for our task and ourselves.

Visions of objects screw this entire process into the ground. When you already see the illusion of objects, you stop seeing everything that could tell you that you were wrong. The cases where our visions came true only exacerbate the problem, as we make the vision the goal, not the system.

Sorry to interrupt interesting stuff but on first reading its not clear... screw this entire process into the ground could be a bad thing, as in screw it up, or a good thing, as inprovide it with a foundation. What about substituting something like pervert this entire process'' ? -- BillWeston

Because envisioning feels good. It brings many of the good feelings that really programming brings, but it can't crash. So people pursue visions instead of code (lots of design before you code), and visions of visions instead of code (lots of object oriented analysis), and the worst of all are those who pursue visions of visions of visions instead of code (the methodologists).

So - congratulations on having gained the ability to envision objects before you program. Take a moment to enjoy the feeling when it comes. Then knock it the hell off. Find the one piece of the vision that seems most compelling and do the least possible amount of that. Then bless and release the vision and get back to listening- to your code, your user, your partner, and yourself.

What does Zen have to do with software development?

Even half-baked ideas seem whole when they first come to you. It's hard not to be proud of them. But resist your impulse to show them off too quickly. Recall instead what led you to the idea and share that with your partner. Listen to what comes back. If you are lucky, the response will support or complete your idea. It could also eclipse your idea, leaving your partner with the impression that it was his own. Learn to accept this as it makes very effective communication. -- WardCunningham

Couple of comments. This is all about accurate perception. TomDeMarco in TheDeadline said "It's not what you don't know that kills you,... it's what you know that isn't so." He was talking about project management, but it is true in all of life. It is very easy to forget to look, or to look once and never look again. Worse still, to be lulled into never considering changing something when it is best that it is changed. One's notion of the world gets in the way of the world.

I am concerned that Kent's note throws away design. I agree with the note, but I think that it will cause a perception problem. While it will be possible to get programmers to drop the notion of design and deal with the code in a very interactive flow, I am certain that it will be increasingly difficult to sell such an idea to managers and worse still people who acredit software engineering programs as states appear ready to make software development a part of engineering.. licensing as PEs everyone who sells software or consults.. making them legally accountable for what? Their designs or their code?

It is extreme to say that you do not design, but I also believe that it is inaccurate. TheSourceCodeIsTheDesign. Sell that idea and you are covered. It crystalizes so much. I can't say that I feel more strongly about anything in software recently, except perhaps that the standardization of software engineering practice is still a bit premature. -- MichaelFeathers

Thanks! This is good advice. I'll have to look to see where we've said we don't design and see how to modify what is put there. (In my copious free time.) I couldn't find much where we come out against design ... modified Kent's words above, since I happen to know he won't kill me. If folks can identify places where we seem anti-design, please point them out. Put 'em on my name page or something. Thanks!

The note itself is to a young ExtremeProgrammer, so I'm not worried about the perception issue there, nor about any negative impact on WikiDenizens?, who are already either infected or inoculated on the XP stuff.

Thanks again. --RonJeffries

Oh, way to go, Ron. If you're gonna get all polite and everything then that means it's mine turn to be the prickly curmudgeon...

I reacted strongly both to Ron posting this, although I suggested some time ago he find a forum for it, and to Michael's comments. The above comes very close to the heart of what I think, which makes it dangerous. If you don't like CommitmentSchedule, well, okay, but if you think listening for the universe's simplicity is a stupid idea, that would be harder for me to take. Be brave, little Piglet...

Re: design. Software design to me is the process and effect of decisions with a certain scale of effect (typically between objects) and a certain set of concerns (typically having to do with communicating intent, reducing duplication, and technical elegance).

Now, how could what I wrote above be construed as anti-design? It is certainly against making lots of design decisions all at once. In the style I aspire to, you try to make each decision at each scale with as much context as possible. The reductio ad absurdum is that you make one requirement decision, one analysis decision, one design decision, one testing decision, and then one coding decision. This typically takes 1-10 minutes. You have learned a lot, because you have received the concrete feedback of the code and the tests. Now you have much more context for your next set of decisions- you're smarter, you have the context of running code to keep you from getting far off course, and you have remained focused, not pursuing high-risk, low-payoff phantasms.

I suppose if design is "thinking ahead to avoid problems in the future", I am anti-design. That looks radical to me just sitting there, but it is my honest opinion, so I'll let it sit there. If design is making decisions that span objects, I spend far more time and energy on that now, and I do so much more effectively, with the style I describe above.

(Let me interject here... But this isn't how you behave. As is pointed out at the bottom, lots of things in XP are thinking ahead to avoid problems. What's SpikeSolution. How do you attack WorstThingsFirst without thinking about what's worst. What are the six weeks of exploration at the beginning of C3 if not analysis. It's clearly not the same as what traditional methodologies consider design, and much more focused on getting quick feedback on what you've thought, but there's still definite forethought involved -- AlanKnight)

That doesn't mean you can't design ahead and be extreme at the same time. In the spirit of XP, however, I urge you to pay attention to when your "designing ahead" activities pay off and when they don't, and to experiment with other styles. Within the context of clear communication, simple designs and code, and tests for everything that could break, I discovered that the fewer design decisions I made in a row without feedback, the faster and better I went, and the better I felt.

You have to handle a wild ocean of change. Are you going to build a breakwater, or are you going to learn to surf? --KentBeck (scared of being honest, but there you have it).

Kent, it looks like this is another one of those cases where I agree with you, and I see where you are going, but by playing devil's advocate I've boggled things.

From ExtremeProgramming:

Listening, Testing, Coding, Refactoring. That's all there is to software. Anyone who tells you different is selling something. --KentBeck

You just don't use the word design at all. I work in the medical instruments area. FDA regulated industry. I can imagine people in safety critical areas who believe that unless you diagram first and code second you are doomed. And if your code does not match your diagrams, you've failed to follow process and you've made mistakes. I tend to feel that this is a result of the fallacy that TheSourceCodeIsTheDesign recognizes, and that XP does away with even if it does not say so explicitly. So, I agree with you totally, I'm just presenting this other perspective which XP may have to deal with in some milieus.

We start with waterfall, then we go to spiral, and then iterative incremental, and now XP. What is the tendency? Has the process for design and constructing bridges followed this sequence over the years? In software, we can say that it is L, T, C, and R (as above) and design is what goes on between the grey matter and the fingers on the keyboard with no reviews or signoffs, or we can say it is all part of the DesignPhase of software and TheSourceCodeIsTheDesign.

What I am saying is that in some areas people will not look twice at a process with no design phase. They will call it hacking and freak out. I just see the whole thing L,T,C,R as the design phase. Everything before you distribute the software. -- MichaelFeathers

Michael- I agree that we agree. The question is how do we break the news. Early in the life of an idea it can be worthwhile to be outrageous. I have certainly used this tactic at times.

However, as time goes on, I find I am getting more outrageous, not less. I was interviewing a project for a possible coaching position. They said, "We like to do two months of analysis and design for a six month project." I said, "I don't see how that could possibly work. I can't imagine doing less than six months of analysis. And six months of design. And six months of coding. And six months of testing. Fortunately, though, we can do them all at the same time." We'll see how that works... --KentBeck

Oh, I like that last bit! --RonJeffries

I like this page a great deal and hesitate to comment ... but.

Hmm, where to start after that but. How about this:

But I like to see the far side of the lake before I commit to cross it. I like to see how high the mountain before I commit to climb it. And I like to see how deep the problem domain before I commit to solve it.

At what point does XP scope the problem domain? At the end of the project? Isn't scoping the problem domain necessary to adequately resource the project? Analysis, for me, is more than just input to a design process; it's what gives me an idea of the mountain's height or the lake's width so I can adequately resource my commitment.

I'm going to go out on a limb here: You are asking to predict the future. You are asking questions that can only be answered once the project is completed (ie how many resources did it take? What objects are necessary? How do they interact?). You are asking to scope the project in such a way that you will know more than you would by gathering requirements and making a CommitmentSchedule....which I would call analysis, design and scoping if pressed (by management) to call it more than it really is.

Basically, as in RealLife, in these things there are some inherent risks. The best you can do is (1) understand what is being asked of you, and (2) perform to the best of your abilities. There's no safety net. --AnthonyLander

Nicely put, Anthony. --rj

True. As far as scheduling and scope go there is no safety net, but in the nuts and bolts of development, TestingIsTheUltimateSafetyNet?. There is no excuse today for a project which pushes all the bugs to the final month of development. Developing without testing as you go is like climbing to higher and higher tightropes without a safety net. The chances that you will splatter almost approach certainty.

This is the nub of XpChallengeCarteBlanche. I'd take XpChallengeCarteBlanche as an appeal for XP to adopt an extra value, which, for want of a better word, I'd call Vision, and an attendant activity, which. for want of a better word, I'd call Scoping. --PeterMerel

The letter above is about how to code, and how to partner. I didn't make that clear.

Overall, XP is about seeing the far side of the lake, how high the mountain is, as clearly as necessary, not by introspection, but by production of actual artifacts.

UserStory cards, written at the beginning of the project, delineate its overall size;
SpikeSolution (ArchitecturalPrototype?) tests the solution from the very beginning;
CommitmentSchedule, done after the first SpikeSolution, assigns risk to each story and ensures that they'll be attacked WorstThingsFirst.