Xp In Common Lisp

Much of the ExtremeProgramming discussion here is using SmalltalkLanguage or JavaLanguage for illustration: while the tenets of XP are fundamentally language-independent, there's still a degree to which the choice of programming language can make ExtremeProgrammingPractices more or less easy to achieve.

This page examines the fit between XP and CommonLisp. It's not intended to be a general discussion of Lisp's worth: general complaints that CAR and CDR are silly names, or that XML is a bad reinvention of sexps, should be directed elsewhere.

Pro:

Lisp as a language is easily extendable in more directions than most other languages. If you're programming Lisp "right" (see e.g. OnLisp) you end up with a domain-specific programming language in which to describe your problem. Truly TheSourceCodeIsTheDesign.
Because the system is typically interactive, the usual mode of development is to alternate between creating new methods and typing in expressions at the top-level to see if they work (PiecemealGrowth, AskTheCode). These toplevel expressions can often be formalised into extra UnitTests for little expense beyond that of CutAndPaste.
Low cost of refactoring: very few changes are actually going to need the system to be rebuilt (unlike CeePlusPlus). Class names are not tied to file names (unlike Java). Decent CL environments (including the free ones like Emacs' IlispMode) have "find all callers" functionality.
DynamicTyping makes a lot of sense when covered by thorough UnitTests. StaticTyping can only ensure syntactical coherency, while UnitTests can ensure semantic coherency. See: DavidThomasOnTheBenefitsOfDynamicTyping.
CL's package system makes it far easier than in, say, C to have the test provide fake implementations of other modules, for putting the "unit" in UnitTest.
Macros allow duplication one to factor out duplication that would otherwise be inherent to the language, facilitating OnceAndOnlyOnce

Cons? Some people might argue that

The ability to evaluate arbitrary code and redefine functions at runtime is going to make it actually quite difficult to find all callers
The language provides no particularly cast-iron guarantees of encapsulation (so you can get at other packages' private symbols with little extra effort) making it easy to bypass public interfaces and increase the risk that refactoring breaks something. This is probably not a big problem: PairProgramming will stop us doing that and UnitTests will catch us if we do. PythonLanguage and other similar languages have the same attitude, and it doesn't seem to impede them.

Given more pages like this for other languages (see also RefactoringWithCeePlusPlus) I wonder if we could factor out a general list of desirable attributes for an XP-friendly programming language/environment. -- DanBarlow, WikiGnomes

For what it's worth, I'm starting a (commercial, paid-for) project where I most certainly intend to do XpInCommonLisp.

Well, we, at Memetrics (http://www.memetrics.com) have been doing XP (well, as XP as we can) for about 6 months now. Specifically, we do:

Lisp seems not to help or hinder either was as far as XP is concerned. (Of course, it's the best language in the world, for other reasons. :-) So far, I think programming is just plain hard, and while XP helps, and provides the most pleasant programming experience I know of, its no SilverBullet and we still have to stay on our toes.

The design and constant refactoring aspects, in particular, are painful, but, that being said, our project would probably have failed massively by now in a WaterfallModel. On the whole, we like it, and will keep doing it. -- AlainPicard

Update Our product is shipping! XOS (see Memetrics's web site) is now available. The back end is written in CL and was developed in a (mostly) XP fashion. I keep meaning to write up a report of our experiences. Would there be any interest? --AlainPicard

Sure! First-hand experience-sharing is one of the best things done on this Wiki.

See CommonLispUnit, CommonLispXpTest, also SchemeUnit

Management Testimonial

A bit more on XP and Lisp at Memetrics.

We have been doing XP for close to a year now with excellent results. New people are writing code the first week which is the highest value to the company. I have been doing commercial software development for nearly 20 years and this is the highest quality code I have seen produced. The expense of constant refactoring is very low as the overall development cost is much less. The value of rapid feature evolution to meet changing demands is a large strategic advantage.

We are doing 24/7 client/server code, which means that the code can't afford to fail. Hey, if the code can fail, you can do it in any language. Using CommonLisp means that we can change server code on the fly without having to bring down the server. If you want processes to live for years, you basically need to use a dynamic language. For commercial multi-platform delivery the leading choices are really Lisp or Smalltalk. (AndrewAppel has a paper advocating MlLanguage for this purpose, however; see his homepage.)

We are very pleased with both XP and Lisp as development choices. --KenDickey [CTO, Memetrics]

Is it possible that XP is the juice that lisp was missing when it tried to find a home outside of the ivory towers?

Dunno. I think Lisp and XP are orthogonal. Lisp has many problems, but every one of them is social, not technical. --AlainPicard

That might make them less orthogonal [sic] than one might think. Xp primarily addresses the social problems that this activity we call development is prone to.

It seems to me that the things that makes the Lisps great overlap heavily with the things that make Smalltalk great, and since Xp comes out of the Smalltalk world, we might expect XP and Lisp to be a good fit. But just that, a good fit. Given the way that XP developers want to interact with their code Lisp should neither help nor hinder XP, but rather just let you do your development the XP way without distractions from the language.

A language with, oh I dunno, unit tests and refactorings as first class entities might aid XP, Intercal would probably hinder it. I'm tempted to add that Intercal would hinder anything, but actually I suspect that using a language that clunky on any kind of large scale would probably demand some big design up front and so forth, since the code won't be telling you anything it'd help you to hear. --KeithBraithwaite