Extreme Programming With Types
Extreme Programming With TypesThe following conversation was trying to go on in XpCritique:
The only large outstanding question that I have about XP now is whether it can work well with C++ or typed languages in general. I know that we've gone through this on other pages, and I've done some C++ XP-in-the-small, but I still want to find out how crucial rapid feedback and malleability are in XP. My gut tells me that XP would work well. If anything, we with compilers would have an advantage in being able to search for type names across a project and change them all at once. Our true downside is compilation time and the thought that has to go into physical architecture to keep it down. Although with good factoring much of it can happen automatically.
My feeling is that a typed language, even a high-level one like Java, interferes with ExtremeProgramming techniques. On a project I was working on (I will describe it here Real Soon Now, Honest), I found one of the most time-consuming parts of refactoring was dealing with type changes. If an int had to be expanded to a long, I would end up making the change, compiling it, seeing where else I now needed a long instead of an int, changing those, recompiling, finding more that needed changing....
That, I think, was part of why the ExtremeAdaExperiment talked about having the code broken for days whereas RonJeffries suggested that code should only be broken for minutes. The ChryslerComprehensiveCompensation had a refactoring that took over a day -- I would imagine that Ada would be ever worse than Java with regard to changing types. And ChryslerComprehensiveCompensation has a RefactoringBrowser....
-- BillTrost
There are two reasons for refactorings taking longer than is comfortable. One reason is that we undertake the refactoring in too large a bite. And the other one is, um, the other one is, er, ... never mind. There is no second reason. --RonJeffries
BillTrost's example of a refactoring "If an int had to be expanded to a long" is a bit of an anti-pattern. Instead of changing int to long, you should create a tiny type just for that "int" or "long", so that the internal representation is "hidden". "Hidden" is quoted because you should fully inline such tiny types so that there is no performance impact (never mind that you can't inline in Java, you can't use normal arithmetic operators on such values, and we were performance-pinched as it was -- BT).
Another advantage of such tiny types wrapped around simple values: they can also be used to prevent the sort of error that is caused by mixing units of measurement, say, English and metric (are you listening, NASA?).
-- AndyGlew
I would think the effort involved in changing an int to a long is independent of using ExtremeProgramming. I also would suggest that by really following refactoring, the problem is not as difficult. I would suggest the following steps:
-- WayneMack
That wasn't the problem. The problem is that I discovered that a core value in a data structure needed to be a long. Every place where I used that now-long value as a parameter instantly became an illegal downcast to a narrower type (int), so I had to modify all those functions to take the new long. This, of course, cascaded through to wherever the value was used. -- BillTrost
I don't want to get into a semantic battle, but this sounds like a "C" problem, not a "C++" problem. I have my own set of war stories on changing declarations within C data structures. This was one of the problems solved by classes and data encapsulation in C++. These changes become very localized in C++ and the typing system helps to tracking down external calls, so you don't have to rely on naming conventions to find type declarations scattered throughout the code. -- WayneMack.
The problem with types is that they force you to say things that you don't know are true, just to get the compiler to bless your program. "Here is the interface between these two objects" is a good example of the kind of thing you are forced to say before you know it's true. The interface between two objects invariably changes. I like not having to specify it when it is sure to change. Instead, I can say "This object needs an object like that in order to work". I know this is true (for now) because the code demands it. When the code changes and the interface changes, I change the code (which I know needs to change), but I don't have to change anything else. -- KentBeck
One thing I've learned from ExtremeProgramming is to adopt processes where you only need to specify what you know now, rather than having to make decisions on other things to get the process--in this case the Java compiler--to "bless" what you've created, as KentBeck says. This also applies to using StoryCards instead of writing heavy UseCase documents, for example, or CrcCards instead of UmlDiagrams in diagramming software.
A process that forces you to specify everything might lead you to not do anything at all, to avoid the hassle. In BillTrost's example above, he might decide refactoring to change an int to a long isn't worth it. And less-refactored code is worse code. Or you might decide not to write any code until you know enough about the requirements to write a completely-specified UseCase document, which is basically impossible. Or not create an object model until you know enough about it and have the time to produce a desktop-published UmlDiagrams.
This "looseness of specification" or "deferred specification" is part of what makes using loosely typed languages, StoryCards, or CrcCards (or IndexCards as a ToDoList) so attractive and freeing. -- ApoorvaMuralidhara
[Referring to KentBeck's words above:] True: the separation of interface and implementation is not a good idea in lightweight processes. Many of us coding in strongly typed languages have worked around it, e.g. by providing code generators that extract a .h header file interface specification from the implementation code. GNU C++ supports this approach, reducing the redundancy between interface and implementation via its #pragma interface and #pragma implementation.
By the way, it feels very good to see Smart People like Kent saying this sort of thing - "Separate Interface and Implementation are Not Necessarily Good".
-- AndyGlew
Another interesting and related topic is the process of transformation in the context of an XP project. For example, let's say you take a snapshot of the current state of your project. Then, you suddenly realize you need to change a type. Once you've completed that change through all your code, and you test your change to validate that you were successful, you take another snapshot. What changed?
The difference between the first and second project snapshots can be thought of as a transformation. There are a whole bunch of different kinds of transformations you can go through. And there are many different motivations. (Too many to mention here.) I can imagine that there are many recurring themes in doing it that can be (or has been) documented as patterns and idioms.
If ExtremeProgramming is based on the ExtremeValues -- Simplicity, Testing, Communication, and Aggressiveness, then why wouldn't it work with a typed language? The language is an artifact of technology, not the core of the process or system.
I have a much smaller system (150 source files, roughly the same number of classes, 21 KLOC at present) written in Java. I'm a solo programmer, the only one in the company in fact, but the project has had Simplicity, Testing, and Aggressiveness as its philosophy.
The size helps, somewhat, because it lets me completely rebuild the system and catch any typing problems. During active development, I do a complete rebuild about every two or three hours. If the project were larger, or there were other developers, the merge-and-build cycle would probably have to be daily.
I think ExtremeProgramming is more a mindset -- reflected in the name -- that doing the wrong thing and fixing it is better than sitting around trying to figure out what the right thing is with half the information.
-- RobCrawford
I do not believe Michael or I would say that you can't do ExtremeProgramming in a typed language, and I (at least) agree with your argument. Both of us have done what we consider ExtremeProgrammingForOne in typed languages. I just wanted to point out that refactoring is more time-consuming in a typed language. It's still extreme, it just takes a bit longer.
So, with Smalltalk you can turn all the XP knobs up to eleven, but with Java the refactoring knob only goes up to eight? I wonder what the knob ranges are for other languages.
The knob ranges can presumably be determined by a LanguagePissingMatch.
Manifestly typed languages (see ManifestTyping) have more redundancy. They use that redundancy for error checking, which is fine, but we still have to maintain it. It goes against OnceAndOnlyOnce. But UnitTests are also a form of redundancy
I think XP would work in Java, but scaled down a little. It's much harder to get short methods when what in Smalltalk would be a 1-line loop takes 4 lines minimum. It's harder to get small refactoring steps when you have to update a dozen type declarations.
There's been a suggestion on these pages somewhere that XP should use just a subset of Java. For example, declare all methods public so that they can be got at without introducing an extra refactoring step. Maybe if you are spending time on type declarations and assertions, you need spend less time on UnitTests.
"Typed languages in general" is another matter. I expect XP would do well in a language with a good TypeInference system, like the HaskellLanguage. Haskell is a FunctionalProgrammingLanguage, about as expressive as Smalltalk but statically typed. -- DaveHarris
That is an interesting thought. I used to be pretty good with SML/NJ, but it has been too long, and I am no longer convinced I could do anything useful with the language. -- BT
I have been waiting for this discussion . . . I have been experimenting with XP on a "real, paid for" project and on my own time (minus PairProgramming). Both have been using Java.
The "real" project's tools include JDK1.1.4 and vi and SCCS and, well, that's it. It is a real pain to refactor, especially if it includes changing interfaces. It goes something like this: All tests work, change tests to reflect desired interface changes, build, build fails, refactor to fix build problem(s), build, build fails somewhere else, refactor to fix build problem(s), . . ., build, build works, test fails, fix code, build, test fails, fix code, build, test works, done. Keep in mind that this also includes navigating from file to file, editor to command line and back, over and over.
Now compare this to the project I'm working on using only VisualAge for Java 2.0: all tests work, change tests to reflect desired interface changes, refactor to fix build problem(s), refactor to fix build problem(s), test fails, fix code, test fails, fix code, test fails, fix code, test works, done. Notice all the build and interpret build problem(s) (build, build fails) steps are gone. VAJ does this for you automatically. Every time a method changes an incremental build is done. If there are build errors they instantly appear in the editor. Notice, this also means I do not need to navigate from file to file, editor to editor. It all happens in VAJ. When I do navigate it is all inside VAJ. Plus, because VAJ automatically versions every method change I can quickly go back. This is much more difficult and time consuming when using "traditional" source code management tools (e.g., RCS, SCCS, PVCS, ClearCase). (When I was learning to love VAJ, SteveMetsker told me that I now understand what SmallTalkers? have had for years. It will be hard to ever go back to the old way (PoorCppProgrammers).)
I would say both are extreme. One is just a lot faster than the other. Also, VAJ, IMHO, lets me be more aggressive (hence, more extreme) by keeping every change to every method and class. I now have no fear that I can easily undo any knot I put myself in. So, in summary, I would say that tools can help you be more extreme by letting you be more aggressive no matter the language.
-- HankRoark
In discussions with WardAndKent, a key basis for XP keeps showing up, namely that ChangingInterfacesIsExpensive, according to the normal view of things. If ChangingInterfacesIsExpensive, then of course you set up your design process to avoid changing interfaces. WardAndKent start out questioning this assumption - suppose we could make it not expensive to change interfaces, then what benefits would accrue? Answer: all those things Kent keeps talking about, having to do with learning and improving and reducing design entropy. Well, if it hurts to work in a way that ChangingInterfacesIsExpensive, and it is beneficial to work in a way that makes it not expensive, let's work in a way that makes it not expensive... presto, XP with RefactorMercilessly.
But if ChangingInterfacesIsExpensive, in real life, then the basis for RefactorMercilessly suffers (and so does Hank). That doesn't touch some parts of XP, such as PairProgramming, communication, listening, UnitTests, etc. -- AlistairCockburn
I've found that I can RefactorMercilessly when doing COM programming (see RefactoringCom) all the way up to the point where I deploy to an outside party. So during development -- where design and refactoring is an on-going theme -- it doesn't hurt nor is it expensive for me to change interfaces.
I don't know what else to say ... perhaps WardAndKent or others who have experienced this pain or cost can elaborate with some specific examples? -- PhilipEskelin
If at the end of the twentieth century anyone is building any application that doesn't begin with a weakly typed language - Perl, Python, Smalltalk, Prolog or VB - then they need to get their head read. AlternateHardAndSoftLayers. Start in a RAD language/component paradigm, then profile, then recode the bottlenecks in C++ or whatever runs fastest. XP may work okay with strong typing, but there's no good reason it should need to. The days of the big rigid app are over. -- PeterMerel
That sounds like a case of DesktopApplicationProgrammer? syndrome. Your toaster doesn't have the memory space for garbage collection, and your digital cable box doesn't have the megagruntz (decoding MPEG is hard, and those vendors are real penny-pinchers). And embedded is what really matters -- PC programming is for weenies. (-: -- BillTrost
That's a kickass statement you've made ;-) but it could very well be largely true. Why do they need to be weakly-typed? What about strong-typed languages -- what about Haskell? It's only mentioned in a few places on wiki, and MikeBeedle recently spurned discussion on the topic in PatternsDiscussion?, and this has caused me to look further into it and I'm pretty impressed.
With ComponentBasedDevelopment one of the big benefits is language independence through interface separation from implementation. I like your idea of programming rapidly to fit the situation. Early in the project, the situation is that you are trying to get your hands around the problem and lots of refactoring occurs. In CBD I see that a refactoring that consists of "porting" a component to a compiled language like C++ is more likely.
I recently did this with an Excel add-in. I built an XLA to try out various things and get our hands around the problem quicker. I assumed up front that certain portions of the add-in would become an XLL for various reasons (e.g., performance). This did happen ... in fact most of my extension to Excel went into the XLL including functions, dialog boxes, and menus. The XLL was built in C++ and it used raw COM via the helper classes (i.e., _bstr_t, _variant_t, _comptr_t) and #import to get at components the XLL interacted with to get its job done.
At the end of the day, only a little bit of code existed in the XLA -- in fact, it used the ThirdPartyBinding pattern to instantiate components and bind them together, and it loaded the XLL. The XLL, acting as an adapter, did all the processing and behavior management between Excel and our lower-level components.
The InterLanguageUnification (ILU) people do this with Python. ILU is an ORB, so they develop in Python, define interface types in ILU's interface language, and then rewrite the performance bits in C. This works nicely because ILU is one of the few ORB's that supports multiple languages within an address space.
Just for the record, I am following Xp as nearly as I can using C++. (Even PairProgramming looks like its gonna fly here.) I have no difficulty refactoring. That doesn't mean I don't wish it would go faster: zip zip zip would be nice. But to whatever extent C++'s strong typing slows refactorization, I still am completely delighted with my 1) increased development speed and 2) vastly improved quality of code and 3) my enormous confidence in and knowledge of that code. [Holds product near face:] I unabashedly embrace XP, even for statically typed languages.
Now if only I could find better UnitTests for Win32 GUI work. -- MichaelHill
It seems as though the experienced Smalltalk developers agree that XP is slower in Java. What about people who first became very (relatively) experienced in Java and then moved to Smalltalk? Would they agree? I have been programming in Java for a bit over three years and have only dabbled with Smalltalk. When I started learning refactoring, my initial impression was that Java's strong types actually made things simpler, as I could let the compiler tell me when I forgot to change all of the implementers of a method where relevant. I thought it would be harder in Smalltalk because there you might only find some of these errors at run-time. I suspect that the style one uses in a strongly-typed language is just different from that used in a weakly-typed one, and the habits developed in one simply aren't as effective in the other. -- RussellGold
DoesUnitTestingMakeStaticTypingLessUseful?
After trying out XP for a while, I find myself growing annoyed with Java's type system, and wishing I were using Smalltalk. I'm drawing no inferences. It might just be the effective rhetoric of some Smalltalkers whose writings extolling the virtues of the language, especially in an XP context, I occasionally come across. -- LaurentBossavit
CompilerErrorsAreYourFriends. NuffSaid.
I used to be a C++ bigot; and then I found Smalltalk. WardAndKent aren't lying to you; Smalltalk is better. I used to think that types were awesome and made a real program more "real"; and then I found Smalltalk. Now I use Java for my "money-work" and find it is really about half as powerful and expressive as Smalltalk.
Smalltalk seems to fall into the ScriptingLanguage family, and it has been shown time and time again that people are much more efficient and productive in scripting languages than typed/compiled languages.
At least, I know I am. -- JeffPanici
The problem with types is that they force you to say things that you don't know are true, just to get the compiler to bless your program.
Yet you create a boat load of tests that have to change when you make a change. A blessing occurs.
That's an inappropriate analogue. The very essence of tests is that we know that what we're asking in the test is the desired behavior, by definition. Sending a message to an object doesn't necessarily need to know that the object is of a given class- just that it responds to the method. When we do a typecast to shut up the compiler's frivolous warning or error, there is that syntactic noise(THIS IS A FOO), utterly unimportant, that distracts from the intent(send message to object). When we have to change the tests, that's because the requirements changed. The truth is not what it used to be. The tests become false, so we have to make them true.
If ChangingInterfacesIsExpensive, then of course you set up your design process to avoid changing interfaces.
If you make a change then all the dependent code must change and then any related tests must change. How do interfaces make this harder? It's easier in fact because the compiler can tell you where things are broken.
It also tells you where things aren't broken, and then requires you to change broken and unbroken alike.
These things are broken. If you change the return value of a method from being an int to being a string, is there anything imaginable that still works? Other than passing that datum through I think not. Generic functions or type inference solve this (and prove JavaLanguage a low-level language). Changing something from int to long is not really a change, the interface stays the same. The correct type would probably be a whole class of types, BigInteger or something like that would be a good approximation; and again, JavaLanguage is too low-level to make that practical.
As you say, changing something from int to long is not really a change. But the type system of Java makes that change expensive. The data type changes, and therefore the manifest types in the message interfaces need to change. This is what is expensive.
''Besides, as few methods exercise the entire interface of an object, it's not impossible to imagine a situation in which you swap one type of object for another that shares the requisite subset of the first one's interface. But then you'd have to change signatures... and these objects might not be in the same hierarchy, for instance. I think it boils down to something like ManifestTypingDefeatsPolymorphism?.''
Not necessarily. HaskellLanguage for instance supports manifest typing but you can also leave type inference to the compiler. Even when manifestly typing you still retain polymorphism. Given a function that could be applied at the equivalent of int or long (function :: Int -> Int or function :: Long -> Long) you leave the type polymorphic and only constrain it as necessary (function :: Num a => a -> a), later apply the function at whatever type you have. This is also the type the compiler would have inferred on its own. To do this, int and long do not need to be in an inheritance hierarchy. The same style is possible in CeePlusPlus using templates, but not in CeeLanguage, FortranLanguage or JavaLanguage. So I'd say a monomorphic type system defeats polymorphism.
That sounds fair. I wasn't thinking in terms of other languages. I think templates are kind of a poor solution to this problem, though. HaskellLanguage's type behavior seems pretty neat. The idea that the compiler infers types is interesting to me; I'm used to 'types' as the sort of thing that happens at runtime. I suppose I should have worded my page name differently.
ObjectiveCee allows manifest typing too, but for object types you can use the generic 'id'. I rarely use anything but 'id', however; the behavior has leaked through from my 'Fun Programming' which is in Smalltalk. ObJC is great, and if it had support for blocks and a bit more dynamism I'd consider it a perfectly valid replacement for Smalltalk, at least for me. Feel the proselytization... (: I'll lay off. MonomorphicTypingInhibitsPolymorphism?. (: That sound good? Or is "defeats" better? If anyone were planning on making the page, that is...
No page needed, it is somewhat self-evident, isn't it? What's probably needed is JavaIsNotThePinnacleOfStaticTyping?.
Answering Wrong Question
To me it looks like the wrong questions are being answered on this page. The question should not be "Do typed languages make Extreme Programming more difficult?" rather the question should be "Does Extreme Programming make development in typed languages easier?"
The only parts of Extreme Programming that interact with a typed language would be Refactoring and ProgrammerTests.
Changing types is a non-trivial task. Encapsulation can help, but it has its limits. If, however, one must change a data type, it is much easier to use a structured approach like Refactoring than the more common grep and guess approach.
Data typing does force a higher level of coupling between test code and development code and between calling code and called code in general. This can lead to some initial thrashing when defining the method and all necessary parameters. Nevertheless, one is better off using TestDrivenDevelopment with a typed language than to use upfront paper design and after the fact testing.
The conclusion I draw from this is that Extreme Programming is beneficial when doing development with typed language.
Seems to me that they're both good questions. If you like XP and you're wondering what language to use, the first question is important.
[Has anyone ever been in that situation? Are there people who like XP and base their programming language choices on that? I've never seen it. Programming language choice has always been dictated by the skill set of the available developer pool and application requirements.]
This page won't convince an XP shop to switch to Smalltalk, but it might convince an individual programmer at that shop to learn it. Small steps... --AdamSpitz
EditText of this page
(last edited July 25, 2005)
or FindPage with title or text search