Selling Patterns Via Profiling
Selling Patterns Via ProfilingI was having a conversation with a friend a week or so ago. It's a conversation I've had with several people over the past 2 months, as I try to formulate why I think patterns are useful.
The basic problem is this: profiling in the sense of procedural programming is a much less useful tool in the OO world.
IMOandE, profiling works better with objects. Discussion below. --RonJeffries
Brief side-note: Why ? Well, I'm gonna steal from Coplien's recent article [1]. He says:
Under the procedural paradigm, we chunked hierarchies of procedures under some top-level procedure, using structured design techniques. The object paradigm supports hierarchies too (class hierarchies) but now we chunk several procedures at each level of the hierarchy. Also, the abstractions of the procedural paradigm were fairly concrete...
Which is a smack and a dab away from saying "profiling is a useful tool in procedural programming and less so in object-oriented code."
What do you do when you have 37 objects, neatly layered and polymorphed to hell and back. Every thing fits neatly into place and all the methods are nice and tidy but the central question remains: Is It Efficient ?
We've all seen code that looks neat, has superlovely abstractions and runs likes a crippled dog.
And this is one place where patterns can be sold. That is, Design Patterns, in the Gang of Four sense (not "architectural patterns" or any of the other pattern styles).
If your code neatly fits into one of the standard patterns, if you structured everything according to one of the standard approaches (and can justify your choice of approach as somehow grasping the essentials of the situation), then you can be reasonably certain that your code isn't exponentially bad.
From the point of view of "guaranteeing somewhat efficient code," patterns take over a lot of the profiling task. Which is a good thing. Because pattern-level thoughts occur earlier in the design process (when it's much easier to change things) and profiling occurs after all the code has been written.
I don't want to squash your enthusiasm for patterns, but I disagree with the basic premise. I'm a VisualWorks programmer, and I find the profiler essential. I don't know if profiling is easier or harder than in procedural programming, because I only used profilers a few times back when I was a C programmer. It is certainly valuable to Smalltalkers, and I wouldn't dream of building systems without it.
The DesignPatterns have flexibility as their reason for being, not performance. Good design is essential to good performance, but you often have to trade flexibility for performance.
At the risk of repeating myself and at the risk of looking extremely foolish as I completely misunderstand the obvious (repeat: at grave personal risk), I will now try to explain what I might have meant.
The stages of a programmer's life... (a WittgensteinsLadder, if you will --FalkBruegmann)
Spaghetti. In my life, this was 6502 assembler, first learned when I was far to young to understand the value of planning ahead (and far too energetic to care). It's code that is a mess. You have an idea, you want to code it, you can't be bothered, or don't know how, to think it through ("Thinking through the program flow ? Nah. I'll just use a goto when I get stuck.").
Procedural. It's still the way most CS courses are taught; whenever you look at an undergraduate text, the dominant focus is on "here's how to accomplish this task" and that's a procedural statement par excellence. Top down, structured, whatever words you want to use to describe it. It was popular for a long time, many tomes were written to make the deep mysteries even deeper, many programmers still think this way even if they're writing in an OO language.
Object-Oriented. We all know what this is. Cf: Booch or Jacobsen or Rumbaugh's books, grok a bit from the Smalltalk-80 manual, think deeply and, late at night, chant the words "polymorphism" and "information-hiding" to your cat. Your cat will nod knowingly and you will have a strange urge to lick your paws. Maybe you will become enlightened.
Patterns-Oriented. Not for me to define.
And the point is that your code changes along the way. Spaghetti code is a nightmare. My very first program was in 6502 assembler. It had an initialization routine which asked the player ('twas a game, natch) questions and then went and changed its own code based on the answers (I didn't quite understand the concept of variables). I once worked at a company where the lead programmer, a man who was old and grizzled and could remember the days before transistors, used "design by typing" as his principal means of analysis. Spaghetti code is not pleasant.
Then you move into procedural land. Structured code with well thought out and almost linear execution flows. The functions tend to be task-oriented and long (a page or two of code in a single function is not uncommon). And it is here that profiling is amazingly useful. You profile the code and out comes the answer: You're spending too much time in this function which corresponds to that task and so it's time to stare at a fairly small and fairly linear subsection of the code for a bit and, if you don't see any gross errors, grab for the coffee-stained copy of Knuth (DonKnuth) you stole from an ex-girlfriend 5 years ago to find out how a real pro would have accomplished the task.
Then you move into OO, phase 1. Objects, lots of little tiny objects, they clutter your universe. You've got experts in the problem domain, you let their insights structure your code, you create these things that correspond to nouns in their specialized language. You get objects which have very nice properties-- they have methods which correspond to user expectations and the methods tend to be fairly short (most of my methods are under 10 lines). But profiling is less useful here. Because you've focused on the objects, not on their interactions, not on their roles in systems.
Good phase 1 programmers tend to have lots of nice objects, fairly general and fairly pretty, and the performance takes a big hit. And profilers don't really buy you a lot back here. Because that nice clean mapping in procedural code, function <--> task is gone. And the nice linear sense of flow, the knowledge of where the thread's going once it's left a method, is gone as well. You can get profiling information, but it's much harder to interpret.
Moreover, the tasks aren't localized anymore. Procedural code lets you make a large change, locally. Now you have to make little changes in a fairly large number of places. If you can even figure out where the changes should go; lots of times, towards the end of a project, procedural hacks get kneaded into the code, simply because it's hard to figure out what the correct code change would be and the deadlines are looming. People who've seen "manager" classes sprout like dandelions as the ship date nears, they know what I'm talking about.
And then you move on and you discover patterns. And what they contribute, aside from flexibility, aside from an intrinsic grace (insert standard QWAN plug here), is a sense of global flow control. You know how the objects interact with each other and this gives you much of the information that profiling gave you back in the days of procedural programming. With design patterns, you get an understandable, tested-by-trial, chunking of the system architecture into pieces which you can understand and in which you can, even before implementing, spot many of the potential bottlenecks. And this knowledge can come from a completely different type of application--you don't need a lot of specialized knowledge (nothing of the form "I've been writing payroll packages for 20 years now and I just know where the bottlenecks are"). Intuition and experience from completely different projects transfer nicely at the patterns-level.
What's more, you get all this much earlier than you do with profiling. Profiling tends to be a late-in-the-game thing. When the abstractions have already been chosen and the implementations are hardening. Pattern-level thinking occurs much earlier, when it's much easier to change your mind about implementation decisions.
If you've been successful at SellingPatternsViaProfiling, try listing the forces that made it possible. I think that if the idea has merit, you can express it in pattern form and condense the above prose into some fundamental principles that people can buy into. In any case, articulating it at the level of principles might provide a foothold to focus the discussion.
-- JimCoplien
But, but, but... I like my prose.
Here's a first crack at it in pattern form.
Design Patterns As A Key To Code Which Performs Okay
PROBLEM: Late in the project, you realize: The app don't perf. And something must be done.
CONTEXT: Application Development
Frequently, the code will not be assembled into a coherent application until the last month or two of a project's lifecycle (individual components might be unit tested, but that tends to be delayed as well).
Thus, performance problems often go undetected until all of the code has already been written and the shipment date is near.
FORCES:
(1) Design changes late in the product cycle tend to be expensive and cause unexpected side-effects.
(2) Full adoption of the OO paradigm leads to objects which model the user domain but do not explicitly control the thread of execution (unlike "functional decomposition" (footnote: 1) which is a cornerstones of structured programming). Moreover a given task can involve different objects (the effect of polymorphism) at different times (and can involve hundreds of objects).
(3) In large (more than 4 people) development teams, knowledge about calling sequences (and the threads of execution) tends to be distributed-- no one person fully understands how a complex task is accomplished.
(4) The OO goal of "loose coupling" (footnote: 2) and the increasingly prevalent view that a major benefit of encapsulation is that it allows developers to treat other parts of the system as black-boxes (footnote: 3) greatly exacerbates forces 2 and 3.
(5) Complex systems often use either threading or some sort of asynchronous messaging (for example, doing a database commit in the background, trying to "preload" files that may be needed from the disk, or forking a thread to communicate with another process). This means that profiling an executable can be a thankless and futile task.
(6) Coding is an inherently "local" task. When writing an object, developers tend to write ONLY that object and not to consider large-scale issues. Later on, when it comes time to tune the app, they have adopted this low-level view of their objects through habit (and thus, they tend to focus on "local" solutions).
(7) Most programmers have "procedural instincts." When faced with both a bug (or performance problem) and a deadline, they tend to write "manager-style" code that solves the problem but sacrifices flexibility (and tightly couples unrelated abstractions).
(8) Performance is a make or break issue. But not in the way most people think of it. It is not necessary to fine-tune the code to optimal performance (for most cases); it is much better to be "close" to optimal and have a flexible architecture.
SOLUTION:
Use design patterns to explicitly chunk system architectures into well-understood pieces that are, while not optimal in the given task, unlikely to be badly sub-optimal. Do this during DesignPhase.
RATIONALE:
There are many reasons why this is an effective procedure. This is a partial list of them (in no particular order).
(1) Design patterns are a way of understanding the ways in which objects interact. If you know the pattern, you know (roughly) the flow of execution. You thus can transfer knowledge from other projects, in completely different application domains, that used the same patterns. Restated: Patterns give you domain and application independent knowledge of program execution structure. They give you intuition about likely bottlenecks and likely performance issues.
(2) Design patterns are arrived at through repeated experience. This means that they represent general solutions to a common problem. While this has the goal of flexibility, it also means that code which adopts a commonly used pattern probably doesn't have performance problems "in the large." Which takes force (6) and turns it into a positive virtue.
(3) Design patterns represent an understandable system overview.A programming team can have several meetings and wind up with everyone having a reasonably thorough understanding of the program architecture (and how the various parts interact). This enables programmers to structure their sections better, both as resources for the system (since they know how their objects fit into the large-scale design) and as a "client" of the other developer's code.
(4) Many design patterns (Factory, Strategy, Visitor) explicitly isolate potential performance hits in a single object. Since the "single object" view is the one at which most coding takes place, this also tends to make force (6) less important.
SUMMARY:
By viewing design patterns as a tool for efficient coding, and explicitly adopting them as a tool to manage flow of execution, you can spot, in advance, methods that are likely to be performance bottlenecks and code accordingly. Moreover, by taking care of the large scale messaging structure, they reduce the need for tools like profilers.
FOOTNOTES:
1. See "Object-Oriented Analysis" by Coad/Yourdon. In the second edition, look at page 20, section 1.3.1 for an explanation of functional decomposition.
2. "Loose coupling" is discussed, among other places, in Booch's "Object-Oriented Design with Applications." In the first edition, the discussion starts on page 124.
3. See _Working with Objects_ (Reenskaug, 1996), page 38 for a picture of a black box.
Design patterns give a team (even a team of one) a rich design vocabulary from which to assemble systems, and the systems themselves will have performance characteristics, but I'm skeptical that design patterns are reliable predictors. Patterns provide guidance on where to instrument the system to take measurements, but my experience has been that most hotspots end up being in surprising, localized places, involving the usual suspects: doing work that's not necessary (e.g., code that should be lifted out of a loop), or poor choice or use of algorithms or data structures (e.g., feeding sorted data into a binary tree). These seem to me to fall below the radar of design patterns, though they may be covered in part by KentBeck's coding patterns.
--DaveSmith (11/26/96)
A while back, I was involved in performance tuning a large C++ system. We did it by timing the system at the function level, to find out which functions took the most time. I felt that we could have seen even more improvements if we had looked at performance at the object/pattern level. So I tried to (quickly) write a program that would analyze our post-mortem performance dump data to gauge the amount of coupling between objects and subsystems. (I got this idea from the ChoicesObjectOrientedOperatingSystem). I never had the time to finish it.
This pattern sounds like it addresses the feelings I had about performance analyzing that system: that performance analysis should also be done at the object level. I bet there are (mostly unwritten?) patterns and pattern languages for designing with an eye to performance and for performance analyzing. This pattern seems to tell us to use those patterns. I buy that, but where are they?
Some comments on the pattern itself:
The solution doesn't "fix" the problem as much as it tells how to avoid the problem. Which is fine, but how do I fix it? Also, the Forces section seems to list a bunch of related problems... how do I fix them?
I think Rationale 3 states best the reason for using design patterns to design for performance.
Rationale 4 might be better stated as something like: "... design patterns can show the performance/flexibility tradeoffs to the designer..." I say this because that is what the Strategy pattern does.
-- DavidHooker
With all due respect, I really don't get this. I see patterns, including the GoF patterns, as generic techniques. To this extent I lump them in with other generics like the various tools and techniques of, say, STL or CPAN.
Now the most common mode of code optimization, in my experience, is to code using groovy generics first, then profile (if you're using C++, Quantify is the profiler of choice imho), find out which member functions contain the bottlenecks, and attack those bottlenecks by replacing the generics with special purpose code. For example, you might find that a map lookup is slow, so you spend some time crufting a more problem-specific hash algorithm.
What you don't do, if you're concerned about cost-effectiveness, is try to design or code for optimal performance in the first place. The reason you don't is that optimizing outside of tight loops and frequent usage is just not going to make a difference, and it's practically impossible to say just where your bottlenecks really lie until you've got something running to play with.
So I think patterns are dandy for explication and documentation of your design, and I'll even grant that some of the sample implementations in the books probably won't yield the worst possible performance, but I really can't see that they have much to do with profiling and optimization. They're about common relationships between classes, not analysis of the algorithms inside oft-used member functions. I think you'd be much better served to familiarize yourself with Knuth (DonKnuth), Sedgewick or one of the other famous algorithms tomes if performance is your concern.
--PeterMerel.
Ah, but you're selling patterns as a whole short. DESIGN PATTERNS are in fact about building flexible systems, and are (as you say) dandy documentation and explanation tools, but you can't paint patterns as a whole with that broad brush. For instance, KentBeck and KenAuer wrote a nifty little pattern language about Optimization in the second PatternLanguagesOfProgramDesign book. I've also found that many of Kent's coding patterns tend to lead to faster code as well. I heartily recommend that you go read their pattern language -- you'll find that it supports your point of view about optimization. As Ken and Kent have often said (to me at least)
Profiling, in my experience, works incredibly well with objects. Refactoring leads (emergently, I know no one is this good every day) to every function being performed in just one object. Profiling picks up these hot spots readily. Often, you can just fix that method, unrolling a loop or something. Less commonly you have to figure out why there are so many calls to Dictionary>>at:put: and fix the structural mistake.
In profiling procedural code, especially assembler, the same or similar code will appear all over and there's no way to discover that the problem that the similar code is (a) all bad or (b) being used too much.
My conclusion would be that OO Patterns lead to good designs, to rapid development, but that they aren't (generally) aimed at performance. --RonJeffries
EditText of this page
(last edited May 6, 2008)
or FindPage with title or text search