Test Driven Design And Patterns

[Swiped from the XP mailing list; see http://groups.yahoo.com/group/extremeprogramming/message/51730 for the original.]

Edited to expand abbreviations and join up some terms so that they link to corresponding wiki pages. I hope noone minds. --KarlKnechtel

Subject: TDD versus Patterns

The title shows a fundamental misunderstanding. TestDrivenDesign is not opposed to patterns. All programmers use patterns. There are patterns in the tests you write, in the way you name methods, and in the way you format the code. When we are discussing the design before we code we'll say things like "we will need to parse this string, but let's not write a parser first because we can probably put it off for a few days and just have a few simple methods in same class that reads the file." This assumes people know about the patterns of parsers. Programming is a knowledge intensive activity, and good programmers know a lot. It stands to reason that knowing more patterns will make you a better programmer. It does NOT mean that reading a book on patterns will make you a better programmer.

When people say "versus", they are probably referring to the book DesignPatterns and the fact that a lot of people use patterns in their up-front designs. People who make up-front designs are attracted to patterns, because they think that patterns will help them to have designs with less of the flaws of their current designs. Patterns might help a little, but the real problem is lack of feedback and lack of experience, and patterns do not directly address these problems. If people are both designing and implementing systems, then when they use a pattern, they will quickly learn whether it works and whether it makes their system better. If they are doing paper designs then they will not, and so are likely to use the pattern inappropriately.

I'm one of the authors of DesignPatterns. I've taught them to lots of people. In my experience, you can't learn a pattern just from a book. Often, people know a pattern already, and reading the book will crystalize it in their mind. Sometimes they are working on a system that needs a particular pattern, and it is easy for them to see how to apply it. But it is only after they apply it a few times that they really know the pattern.

No expert designer will design a system by looking at the problem, running down a list of patterns (in his head or in a book), and selecting the ones that fit. A novice might do this, and I certainly force students to do it in class, but that is not the way I design anything. Normally I look at a problem and all sorts of ideas pop into my mind. I'm not consciously running down a list. Instead, I'm using my brain's natural "pattern matching" ability. Humans are good at recognizing things they know.

Similarly, I usually don't implement a pattern by reading a book and following instructions. I might do this the first time I use a pattern, but I expect lousy results. No book can completely describe a pattern. It will never describe perfectly when the pattern is applicable, it will never tell you all you need to know to implement it, and it will never tell you all the consequences of using the pattern. But having a book is better than not having a book. The book will tell you enough to get started, and after you have used the patterns a few times, you'll understand it. So, the first few times you use a pattern are learning experiences that might not produce a good design, but will certainly add to your mental toolbox.

Lots of people who use patterns overdesign. The result is systems that are complex and hard to change. They are hard to change in spite of the fact that it is theoretically possible to change just about any aspect of the system. But if you can't understand the system, it will be hard to change. Most patterns make your system a little bit more complex. Using lots of unnecessary patterns makes your design so complex that it becomes unmanageable. But it is not the fault of the patterns. Those same patterns are very valuable in a different system. It is the fault of the designer who used them inappropriately.

Overly complex designs are a natural result of inexperienced designers. When people learn something new (ObjectOrientation, DesignPatterns, Swing, C#) they will make lots of mistakes. This is inevitable, and isn't fault of the technology. It is just a fact of life. The problem is managers who don't realize it.

When people learn some new technology, they usually want to play with it for awhile. Ideally, they can play (and learn) on something that is not essential. However, often people are forced to play on production systems. This isn't that big a problem if you are keeping your system simple and are prepared to refactor out bad design choices. If you think a pattern will help, you try it, and back it out unless it makes things look better. But if it takes a month for a design change to become reality, changes are expensive and so an inappropriate pattern will tend to stay. So, the problem with complex designs with too many patterns is a natural consequence of BigDesignUpFront.

I use different patterns from DesignPatterns in different ways. Some happen entirely during refactoring, some can be planned before you start coding, and some are both.

TemplateMethod happens during coding and refactoring, and I would almost never discuss it during a design meeting. On the other hand, I might create CompositePattern while I am refactoring, or I might talk about it with others during a design meeting. Sometimes there is a particular feature that cries out for Composite, so I would know I was going to implement Composite as soon as I saw the feature. Thus, I might know I was going to use Composite before I had written my first test case. In contrast, I only use VisitorPattern after I have been struggling with a design for some time. It never comes from a single story, but comes by refactoring a system once enough stories have been implemented. Visitor is a pretty rare pattern, and most of the time that people want to use it, it is a mistake. But it is pretty easy to create by refactoring. So, it is much better to hold off on it and make sure you really need it.

In summary, it is silly to say TestDrivenDesign versus patterns. People using TDD use patterns, though they won't all be the same as the ones used by people using BigDesignUpFront. Patterns from DesignPatterns do not require up-front design, they all are suited for being targets of refactoring. In fact, some of them are almost always the result of refactoring.

--RalphJohnson