Big Models Are Useless

Context:

Analysis, design, functional decomposition, scenarios, deployment or any of the other hydra-headed UML diagrams. Or any other modelling technique.

Forces:

There's more information in what's not drawn on a small, concrete diagram than in what's drawn. Classes that don't interact tell us all about context. When we integrate these small diagrams into the big model that we vainly hope means more, this information is lost.
Big models are much less accessible and comprehensible than small models. A small model is something you can throw at a developer and expect them to understand by themselves; a big model can only be interpreted in meetings where everyone comes away with a different version in their head, resulting in friction, more meetings, and refactoring ...
Refactoring and maintaining an intricately detailed model is about ten times more work than refactoring deliverables.
Each named model entity obscures the problem space immediately around it. A named entity is a place where we stop thinking. The bigger the model, the more the problem space is obscured.
Where we stopped thinking two months ago bears little relation to where we start today. With a big model we spend more time trying to reconcile the two than we do getting closer to deliverables.
Laying a big model out so it's coherent is time-consuming and, beyond its presentation in a meeting, that time is totally wasted.
AnalysisParalysis!

Therefore,

If a model won't fit on an A-sized sheet, it's useless. Well, so are many A-sized models too, but at least they can be thrown away without much lost time or failed understanding. Don't make big models, and don't let any of your team waste time with them if you can avoid it. -- PeterMerel

See Also: AntiPattern

I'm unconvinced.

I could see that a fully-detailed visual diagram is only useful on a small scale, but that's a far cry from the claim that a large visual diagram is a bad thing.

On one project I worked on, we only had the big visual diagram. The app ran a thousand or so classes, the big visual diagram mentioned about 100 of them. But it had the facades, and the key service providers, and lots of boxes around groups of objects along with annotations. It enabled us to easily discuss things like:

Who are the main clients of this subsystem?
If I am using a facade and getting an incorrect response from a subsytem, where is the problem likely to be located?
What are the main ways in which messages flow?
What paths does the information flow along?

Why couldn't these questions have been answered better with A4-sized models?

Because it's hard to draw 100 or so objects plus groupings plus annotations on a single sheet of paper and the key point of the big diagram was for people to get a feel for how the whole shebang hung together.

So ... don't draw them on a single sheet of paper. Draw them on a number of single sheets of paper, each well defined and complete in its limited context. That will reflect what is actually understood, rather than feed the IllusionOfControl.

I understand the desire to see the whole shebang. I have often seen developers pine for such a big model, but when it is achieved there's always this unanimous anti-climax. It's as if they expected that having the whole model on the one sheet would make them understand it better, where in fact it makes them understand it worse for the reasons numbered at the start of this page.

So I feel that how the whole shebang hangs together is best done with a number of small models, each at an appropriate level of abstraction. By all means have a top level one. By all means have an architectural one describing a generic way to glue the next level together. By all means have ones that describe the contract between two others. But keep each of these to a dozen entities or less.

Now when they say, "that's lovely, can you just put it all on one sheet so we can see it all?" I've learned to say, "It's not necessary or useful but I understand why you want it; it'll make us all feel good. Let's give that to so-and-so-who-needs-some-busy-work. It'll be good experience for them, a nice way for us to spot holes, and it'll give the rest of us do some prototypes of these concrete small models."

When, after much time and great effort, so-and-so unveils the mighty E-sized sheets, I say, "Oh, that's grand. What a good job! But our prototyping work has changed all that quite a lot, and as I said it's really not necessary now." When they insist it be maintained I say, "well, if you think so. But let's make certain it reflects what's going on. Have so-and-so update the small models to match our prototyping, and then produce another one of these big things based on that. That way we won't slow down these fine deliverables we would like to demonstrate for you over here ..."

A couple of iterations of this and our poor so-and-so is ready to kill and kill again. So-and-so spits the dummy. No one else is willing to take on their utterly pointless job. The big model is tidily filed away, so-and-so gets kudos as the CASE guru, and AnalysisParalysis is neatly averted.

When developers understand the big picture, it becomes easier for them to design and build the small pieces.

Would you believe YouArentGonnaNeedIt?

In short, it enabled all the developers to get a feel for the entire application without bogging them down in the details of all the pieces. And it didn't really need a lot of updating (since updates and design changes primarily occurred in the objects that weren't mentioned).

Was it maintained over the life of the project, or was it quietly discarded once deliverables began to appear? I'm curious about how you'd make use of the big model beyond initial conceptualization and planning stages - in all too many instances I have never seen such a use.

We didn't bother with small scale visual diagrams because, by and large, they're superfluous to the code (implicit, quasi-extreme claim: any detailed visual diagram that can fit on a sheet of A4 paper ought to be obvious from the code).

What a developer can absorb in five minutes from a good small diagram may take him ten times as long to absorb from the code. Especially if there's concurrency or observer patterns involved. This is because there's more information in what's NOT on a diagram than there is in what's on it.

Which leads me to believe that the claim ought to be that large-scale fully-specified UML diagrams are a waste of time.

Which is a statement that can be generalized.

Then how come it couldn't be generalized to BigModelsAreUseless?

I'd suspect that the correct generalization lies somewhere around UMLIsAGodawfulMonstrosity. But that's simply my prejudices shining forth (every useful UML diagram I've ever seen had fewer than 20 or so classes OR used only a tiny subset of UML [typically, a subset of what was available in OMT]). -- WilliamGrosso

As I said, I'm unconvinced that you can replace the large-scale picture with a set of appropriately drawn small-scale ones. It seems an awful lot like replacing a wall with a set of individual bricks.

I see it more as replacing religion with science. Religious books provide grand schemes within which every observable finds its place. The Earth is flat, the stars fixed in a great crystal sphere that orbits around it, the whole lies on the back of a pile of turtles ... that sort of thing. Science comes along and says, okay, let's forget that big model for a moment. Let's just talk about stuff we can understand well enough to build and test. The result is unsatisfactory and unproven - just a network of little models with no grand scheme to bind them up. But it provides a basis for engineering that religion never can.

More importantly, I find Would you believe YouArentGonnaNeedIt? as a reply to When developers understand the big picture, it becomes easier for them to design and build the small pieces. rather confusing. And I suspect the confusion lies around the question Was it [the big diagram] maintained over the life of the project, or was it quietly discarded once deliverables began to appear?

The answer is Of course it was discarded. But that doesn't mean it wasn't a useful thing to have had for quite some time. It facilitated communication;

Did it? How do you know? More importantly, how do you know that the communication actually contributed to producing deliverables?

It helped people understand where we were going;

If that's true, why was it discarded before you got there? I've certainly seen these things used to fend off management - blind them with science and so on. I've seen these big models provide a lovely IllusionOfControl ... until real development forced them to be discarded. But I've never seen them promote real understanding - the sort that brings projects better quality, greater speed, or reduced expense. Are you sure about this? How do you know?

It led to more, and better, conversations about overall application structure;

I have no doubt about that. The question is whether these conversations were useful. In my experience, words are a lot less efficient transmitters of information than dendrites. In my experience, BigDesignUpFront piles bad assumption on bad assumption. In my experience, design by committee invariably produces inconsistent, inelegant, costly, and downright confused models. But perhaps your experience has been very different to mine?

It helped people think about their pieces of the project et cetera.

Because they understood their context? Then why wouldn't more concrete and tighter models - small models - have done this better? This returns to the forces described at the top of the page - do you feel these are invalid?

Just because something is discarded doesn't mean it was of no value. Prototypes are often discarded; they're nonetheless useful.

Perhaps we're using different shades of "discarded". I think prototypes aren't discarded in the sense of "junked". They're superseded - improved upon and replaced. When I say that the Big Model is discarded, I mean "junked". No one ever refers to it again, it does not contribute to constructing deliverables, and in general it is as if it had never been built at all.

Another nit: No argument from me about UmlTotalFailure.

That page is different; it's a complaint about the expressivity of UML that goes in the opposite direction. My complaint is that UML is far too expressive and that attempting to use the expressivity of UML is, in every situation I've encountered, an absolute waste of time. Consider a sheet of A4 paper on which you've laid out 12 classes. You're hoping to communicate the design to someone else (let's do that first, then worry about compiling the picture into code). What do you need to communicate? A reasonable cut for each class is:

Class name and purpose. A mission statement for the class.
The important associations. Who does the class, as a class, know about? The class may message other classes, it may have to know about a ValueObject or two, but there is a distinction between the important relationships and the secondary ones.
The sorts of operations the class supports. Both the ones the class performs well and the ones that the class can do (but not well).

And then, there's sheet-of-paper level information. Stuff like:

The reason for the sheet of paper (what these objects, acting in concert, accomplish).
The overall flow of control (who messages whom, under what contexts).
The roles the objects play.
Patterns embodied in the design.

If you could get the above into your diagram, you've probably got an excellent diagram that will help considerably in communicating the design. But a lot of people using UML don't really work towards the above, and instead focus on expressing the low-level details in an arcane symbolism. And the fault, I think, lies with UML. -- WilliamGrosso

Again no argument, and I agree this isn't quite what's on UmlTotalFailure. Perhaps we should move this bit to there? -- PeterMerel

As I read all the discussion here, I hear a different message from what anyone seems to intend. I hear that diagrams, done as well as anyone here has seen them done, are not serving their purpose very well. (I also hear that we don't have a better idea, but that's another question.) Am I missing something? -- RonJeffries

Yes and no?

Diagrams do two things for me. They help me design software. I draw a bunch of rectangles and some lines and say things (to myself) like "This guy sends a message to that guy to tell him about the other guy." I use almost none of the UML style details for that - I use no details that wouldn't be in a CRC card for the class. It's just that I'm primarily a visual person and seeing the rectangles helps my brain work.

And diagrams help me communicate a design (as in the above discussion).

CASE tools, on the other hand, seem to focus on diagrams as a detailed design tool. Which nobody I know (admittedly an idiosyncratic cross-section of the industry) has ever found useful. I spent a year or so trying to believe that CASE tools are a good thing (even took Rational's course in using theirs) but they just don't work for me, in the work I do (and in the way that I work).

-- WilliamGrosso

I think small diagrams usually can achieve their purpose very well. I think William thinks both small and large diagrams can, but that some modelling languages are better than others and some CASE tools encourage unproductive activities. I know XP dispenses with all diagrams it can, but I do find the small ones very useful - even when very detailed. -- PeterMerel

Maybe this is just horses for courses: in some situations a large diagram is required so that developers get a quick grasp of the shape of a system, and that illusion of control. For example, as a developer entering an existing project it is good to get an overview of the whole system, and to have confidence that the team has a grasp of the problem. Same for management and clients, where it can be a useful tool to manage perception and expectation.

If the system really needs to show 100 top-level components to describe the problem and system usefully then that is going to need a large diagram. And large static diagrams are more useful for stable systems under maintenance than developing systems; as a point-in-time snapshot of the system they won't need maintenance if the top-level components don't change.

Or maybe I haven't sold XP to my management well enough. -- WillChamberlain

When I'm working up a new design on my own, I often prefer to draw a little picture - sometimes it'll be a kind of micro UML pic, a couple of rectangles with lines - sometimes it'll be something completely different, like an OakTreeDiagram. I find that when I'm alone, CRC cards don't usually help me, though I have had some luck using them solo.

When working with others, however, the dynamic character of CRC really pays off. The other person can pick up a card, remove it, move it, destroy it and replace it with another. This makes for very effective communication, real give and take that doesn't happen when one guy has the pencil and the other is more audience than participant.

This was news to me when I started doing XP, because I'm very visual and always thought diagrams were best. I was, well, not as right as I might have been. -- RonJeffries

Could it be that CRC are limited by the time it takes to pick up and move the cards when working alone? In a group the motion conveys intent, and gives other people time (1/2 second or so) to see what is happening. When working alone you can think faster than you can move the cards. You already know the intent of the relationship when you start to draw the lines. The drawing 'just' gives you a visualization, and a temporary record which allows more objective definition of the problem domain, and gives you a focus and time to think before your internal dialogue swamps the detail. -- WillChamberlain

XpCrc questions moved to CrcDescribesScenarios

I disagree. Look at the models produced by standards bodies. There are numerous examples of large models that have been successfully implemented many times. Look at the TMF or DMTF models: Huge, complex, and implemented to various degrees of success by many large corporations. Big models are useless if you make them up yourself, is more like it. Good, coherent, practical models of a domain (necessarily and inevitably large) can be put together, but not in isolation. Collaborative domain-specific forums working to agree industry-standard models can do this very well. Slowly and often painfully, but ultimately well.