Room Full Of Meccano

While co-teaching with someone who has a radically different approach to design methodologies than I do, and slowly drifting off, I had this little vision or dream. Like all analogies or metaphors, it will break down eventually, but it captured quite nicely the difference between our two schools of design. Where do you live with respect to these two?

I imagined a big room full of building toys, like Meccano or Lego or something else, with sticks and tubes and joints and connectors. The room was partitioned into two areas, a team of people and a pile of parts in each. KentBeck, WardCunningham and I, and others of our school, were over on the right side, my coteacher, the UML, Syntropy, Fusion, Catalysis and formalists over on the other side (I may be maligning someone here, so let me know if you need me to genericize this more - but that was who was in my dream).

The task was to build this big hairy construction with a multi-way arch and lots of little cubby holes for different shaped things, tunnels and so on.

The informalists immediately started picking up pieces and putting them together in various ways. Someone said, "Hey, look at how these go together," and showed a stable support structure. Immediately he and several others started using that to make a bigger structure. This repeated itself several times. After some time, they had their arch and tunnels, a horrificly complex combination of pieces that worked. When asked to describe their design, they said, "What describe? There it is! Look at it!"

On the other side of the room, the notationists had laid out several drafting tables and were arguing over notation and perspectives. They settled that and started planning and drawing. Days went by, and they were tearing their hair out at the complexity of the 3D spatial angles and interconnectivity they were trying to show. After several months, they produced a stack of paper and announced, "There it is. It is all in there." And the sticks were still all on the floor.

If you hadn't had to get up and do your bit, you might have had time to dream on to the second phase, where the notationists tried to build the thing and slowly found all the errors in their paper, while the informalists refactored their reality and made it even better. ;-> --RonJeffries

This image of the two groups gave me a visceral appreciation for the two approaches. (Even poking a little fun at both sides, I guess you can tell where I put my money...)

--- AlistairCockburn

Sounds like the difference between building a medieval cathedral and employing an architect. See MedievalArchitecture.

--DickBotting

I think the tension that pulls these two perspectives apart has to do with the way our development process interacts with the world around us. Let me see if I can stay with your metaphor for a moment...suppose the party asking them to "describe" their solution is, for instance, the legal beagles who have to demonstrate that it complies with the local zoning ordinances that govern where it will be built?

The iterative follow-our-nose process that all of us long-time Smalltalkers love is very satisfying, and its great for exploratory kinds of problems.

But really guys...what *DO* we do when, instead, our task is to replace the existing call center? When we have to build brick and mortar to house our installations, when we have to lay fiberoptic cable to connect our switches, we need *some* formalism for deciding how many servers, how many cables, and why. When the lead time on printing the operator training manuals is 12 months, and the call center has to be ON LINE for the 1998 holiday season, it doesn't really work to just iterate our way towards Smalltalk nirvana.

When we're talking about $60M projects, 3 years, including building buildings, buying real estate, laying cable, launching satellites, that sort of thing...I can't really see myself saying "What describe? There it is! Look at it!". I bet even an "informalist" like yourself would have a hard time proposing to walk away from $50M+ engagement because the client has the chutzpah to ask for a written design!

I'm trying to figure out how to make this OO stuff WORK for "enterprise scale" projects. I need something more structured than "just build it"...so can we find something between the "informalists" and the "notationists"?

--TomStambaugh

XP says right up front (or it ought to; I'm pretty sure it's in ExtremeProgrammingExplainedEmbraceChange) that it is for small teams. (I would add "of high-quality people", but that's a separate point). I don't think it's suitable for large teams (large projects, perhaps, but only if the customer is willing to have them done slowly and piecemeal by a small team). -- RobertEikel

Agreed. Although, I have known people of the informalist camp to walk away from 50M$ contracts for the exact reason that they don't feel they can either describe or construct such a system.

A few years ago (1994) I said that I thought the state of our (OO) business was building the equivalent of 7-story buildings. It is not rocket science, doesn't call for extensive math or simulation, but you can't mess around or thing will fall down. I said at that time that I would not even entertain the notion to build anything resembling a scyscraper, since our little industry is nowhere near that scale or reliability of work.

I am older now, so either I am braver, smarter, more foolish, more forgetful or simply senile, but now I would be willing to entertain the notion for at least a short while, even given that we haven't moved forward much as an industry.

--- AlistairCockburn

I like this metaphor a lot, but I see the two approaches as complementary. It's where Alistair says "a big hairy structure ..." I see the value in the formalism. I want the formalists to produce requirements in terms of domain analysis, conceptual architecture, test harnesses and QA requirements - goals for the informalists to meet. But I don't want the formalists to do any structural architecture or design work - I think that's where things get tense and wasteful. In short, I want the formalists to define the problem's form, and the informalists to fill it with their solution's function. If this is done well, eventually FormAndFunctionAreOne.

--PeterMerel

I don't want the domain analysis, conceptual architecture, etc (the testing, sure). I want to know why someone is willing to pay us to build this. I want the team to pick a metaphor that covers what we think we need to do. Aside from that, the users and the programs, properly encouraged and listened to, will tell us everything we need to know.

When someone says "big and hairy", that tells me that someone, or everyone, is afraid. Better to address the fear directly rather than simply heap layers of ritual and superstition atop it. --KentBeck

Given my druthers and a clean slate, I'm in the corner attaching nuts to bolts with the other kids. But when I've got a formalist to work with I try to get the best mileage from them I can. Telling them they're barking-mad is not usually a good way to do that. I've also encountered procedural specs that ran to many pulp-feet of oxymoronic procedural requirements - big and hairy - and appreciated having a formalist to turn it into something wieldy for me to work from. There's a time and a place for them is what I'm saying. -- PeterMerel.

There is a middle ground here. I admire formalists, but I never have the patience or time to adhere fully to their approaches. Am I glad that I've looked into Catalysis, Syntropy, et al? Absolutely. It all comes down to the old adage: we learn the rules so that when we break them it is consciously and without serious repercussions. I look at the work of the formalists the same way that I look at patterns: "Here is an approach.. It is not universally applicable, but if I understand it, I may someday find a need for it or its supporting ideas in some future application." Lifelong learning combined with vicious pragmatism. The pragmatism comes from the recognition that you do need to get something done within whatever constraints are presented. Religion is a hangup when you need to get a job done.

I just observe that it is better to have more tools than fewer, and a pragmatic attitude should never keep us from looking at the approaches of people with diametrically opposed philosophies, because no one is ever completely wrong about everything. View formal methods as patterns:

Large Pattern Knowledge + Good Judgement = Programming Effectiveness.

-- MichaelFeathers

Isn't it more like

  (Large Pattern Knowledge + Good Judgement) e ^ ( quality investment * time) = Improved Programming Effectiveness?

-- JeffBay

Yes, yes. Far better to have a tool and never use it than to hear of it, malign it, and never really grok what it might have been good for. When actually building the thing, it does take some will power not to dig way deep into the tool bag and bring up some amazing hunk of machinery ... but it's nice to know one could. --RonJeffries

Be careful here. XP is great for software development but I suggest not trying to extreme a house (extreming house /blueprints/ is a little better, but not much). Make sure the domain you are applying XP to fits Beck's preconditions for XP. -- FelixTan?

ChristopherAlexander has quite a lot of ideas for how to create buildings with a more informal approach.

The problem with the formalists is that their formalisms are not based on experience. The problem with the informalists is that they never explicitly codify the experience they get.

A formalism should be based on abundant evidence, not on whimsical fantasies -- and the formalism should be clear on where it is applicable. Architects, for example, know the properties of their building materials and can tell in advance whether a building is going to stand or not. Architects can also tell when their knowledge is inadequate. We programmers don't entirely have that yet.

What we often have are planning tools that encourage us to guess a bunch of numbers and then average them together -- as if ignorance, filtered through equations, can produce knowledge.

In the absence of a useful formalism, the informalist approach is the only one possible -- but it should be done with an eye toward developing a formalism.

The purpose of a formalism should be to prevent programmers from having to rediscover things that have already been discovered, and to prevent them from making mistakes that have already been made. A formalism should be distilled experience.

Using a formalism should be quick, simple, and painless -- taking much less time than the actual construction of the program -- and you should be grateful that you did it, because of all the errors you'll have avoided. A formalism that gives you anything less than that is a bad formalism.

-- EdwardKiser

It strikes me upon reading this that Design Patterns have been a key formalism, and that Extreme Programming has had quite a positive impact on the informalist. It surprises me that this page focuses on the duality of the situation, given that wiki and the things people talk about here seem to be a good example of the fusion of the informalists working with formality and extending it where possible by distilling experience.

Neh? -- JeffBay

uh, where on wiki do you see "a good example of the fusion of the informalists working with formality"?

Interesting that you have picked meccano: ie a simple set of objects with a known, definable and predetermined set of interfaces linked with a known and predetermined set of tools and joining functions (bolts etc). This is good because it is so simple Everyone can understand it.

For a big piece of meccano-creation there is a requirement for re-use, for increased complication as more and more things have to be supported by it. This is where formalism is good, to define the shape and interfaces of the new objects that are needed and to allow those coming to see the project an understanding of the objects they see.

A comment, if I may, regarding the architecture analogy further up the page. It seems to me that these days, when an architectural firm sets out to construct a huge building, they don't just develop a design through a formalistic process, hem and haw over it, sweat the details until the design looks perfect on paper, then run off and build it - there's this intermediate, iterative step where they take the design, and model it using this thing called a "computer", and simulate stresses, loads, and shocks, and model traffic flow through it, and feed this information back into the design, and subsequently refine the model until they're satisfied with it - and then they go and build the thing, all at once, in accordance with the formal model, trusting that, if their models were accurate enough and thoroughly tested enough, the reality should reflect the design. Why can't we do that in the software business?

Well, that's a bit facetious, I suppose - there are indeed software projects where that's exactly what is done, of course - such things as military avionics systems, medical equipment, etc. The thing is, such projects tend to be extremely mission-critical, and extremely expensive, as with the 300- and 400- (and soon 500-) meter skyscrapers and 2000- and 3000+-meter bridges, etc. that we're comparing them to. Perhaps we simply need to accept that for most software projects, that don't fit this category, it simply isn't cost-effective to expect such practices to be followed? Or is there a way, given the differences between software engineering and structural/civil/mechanical engineering, to make these practices usable on software projects with smaller budgets and expectations?

-- MikeSmith

My experience (caveat: with software projects that exceed two hundred developer years effort) suggests it is a matter of applying YouAreGonnaNeedIt and DoTheSimplestThingThatCouldPossiblyWork. You _are_ going to need more documentation and modeling than on smaller projects. And if it is true that performing less preparatory/upfront planning/design work could not lead to a successful project, you _are_ executing the work as simply as possible.

-- NormanECarpenter