Down Toa Science

(I originally wrote this in private email to another subscriber to the otug mailing list. It strikes me as the sort of half-baked, provocative idea that might benefit from being wrangled over on Wiki. --GlennVanderburg)

I often hear people say they wish Computer Science were really more of a "science" and less of a "black art". But what do they really mean when they say that?

Most people who say that seem (to me, at least, reading between their lines) to be using the word "science" in the sense of "He's got that down to a science." But that phrase is based on a popular misunderstanding of what science is. The phrase means that something has been defined and understood so precisely that there's no longer any uncertainty.

But that's not science; that's the goal of science. (And it's an idealized, unrealistic goal at that.) The process of science is itself messy, unreliable, creative, full of false-starts and dead-ends ...

Hmmm ... sounds a lot like software development!

I'm sure many scientists wish that science were more of a science. I bet it would be great to be able to say, "OK, we'll have cures for cancer and acne plus the Grand Unified Theory module ready by 2002, and it'll cost this much ... oh, you want practical antigrav? That'll take another six months ..." But that ain't gonna happen, of course. And there's another funny thing: I suspect that few of the really good scientists would like that situation at all, because that would take all of the fun out of it.

Maybe, instead of all these methodologies, what we should be looking for is a sort of "ScientificMethod" for software development --- not for testing theories about development methodologies, but to be the development methodology. This "SoftwareMethod?" would be a framework or ethic, if you will, that brings some discipline to all of the guesswork and creativity, and helps us to be realistic about what we really know and what we can really promise at any given time. Just as the scientific method doesn't prescribe particular steps for observation, constructing a hypothesis, or designing experiments.

I wonder what such a thing would look like?

Glenn, I'm not going to be much use in wrangling over this as I agree with most of it too much and I'm simply not clever enough to answer the final question. But this might at least provide an opportunity to put the record straight on what I was trying to say early on in RespectedSoftwareExperts.

The use of analogies there is to suggest that software engineering (which in my view is about designing and building machines that have a positive impact on the real world, unlike pure maths or pure science) is immature compared to other disciplines or professions, and this partly manifested in a lack of consensus on who is a software expert.

I agree. I just think that software development really is fundamentally different from the other engineering disciplines in some important ways, and that it's a mistake to look to them too closely in trying to achieve our own maturity.

Note that the problem is not necessarily a lack of supporting theory - as FreemanDyson has said, no theory helped to build a successful bicycle and once we had one, there's not much theory in explaining why it works. But in my view there certainly is a problem in the predictability of software engineeering's results as told to and then experienced by outsiders.

Yeah. Over time, that might be solved by making things more predictable, but right now I think it must be solved by acknowledging what we can't really predict, and refusing to predict it. This honesty is one of the things I like about XP.

That's why the analogy with physics above is the weakest link in the argument, in my view. We're not building cures for cancer or anti-gravity machines (or am I missing the really interesting projects around here?) Even so, the analogy is still in the right direction compared to rigid PhaseIst thinking.

But I think the analogy applies at less exaggerated scales. Even smaller, commercial science projects (such as, say, developing a new asthma drug) have similar concerns, even though they are similar in many ways to most software projects. They are subject to some amount of long-range planning, and they're expected to either produce a marketable result or, if that's not possible, to find out quickly and at low cost.

XP is going to make some of this much better. So, in a broader sense, are stuff like your speculations above. Thanks.

My theory is that there's a close relationship between the two ...

--RichardDrake

ComputerScience