Static Typing Hinders Refactoring

Perhaps this page should be called something like JavaHindersRefactoring?, CppHindersRefactoring? or TradditionalStaticTypingHindersRefactoring?. Many StaticTyping languages like ObjectiveCaml,MlLanguage, HaskellLanguage or CleanLanguage have much more sophisticated type checking.

May I suggest: ManifestTypingInhibitsRefactoring??

Related: LanguageInhibitsRefactoring

'' I've been programming in Java for the last year or so - with the last three in an XP shop again. While it has some UGLY warts in it, the fact that it is statically typed has become a major productivity boon, rather than "hindering refactoring". I cut my teeth on seven YEARS of perl programming, so I was a pure dynamically typed OO programmer when I came to Java, and I can tell you, I *hated* it.

In a few cases these days, I lament the fact that we don't have dynamic typing. Mostly, I miss multiple implementation inheritance (with no diamonds, of course). Those few cases are becoming rarer and rarer.

Here's the fundamental point that almost every single comment on this page has missed. When I start refactoring, the compiler tells me at every step that I'm not done yet, until I'm done. Then - and here's the missing part - about 85% of the time, the tests just pass. The compiler has step-by-step shown me exactly what changes need to be made in order to finish my refactoring.

As a case in point: about two weeks ago, I did a major refactoring involving destroying two old classes and replacing them with new database backed objects with similar functionality. In the course of about 6 hours, I introduced and fixed over a thousand compiler errors, modified over a thousand lines of code, and touched at least 40 files. I reran our 1500 test point master suite, and had approximately 100 test failures, which required another 45 minutes to fix. Once I had a green bar, I started the web server and started using the application, and uncovered 3 (count them - THREE) new defects introduced by the refactoring, wrote tests for two of them, and fixed these three as well.

Without the compiler to find all the references to all the methods and objects everywhere in our 30000 lines of code, I can't even *imagine* executing this coding effort, much less in one pair day.

If your static typing isn't helping you refactor, then either you aren't using it right, or you aren't seeing all the places where "making the compiler happy" helps you avoid writing extra tests, avoid writing extra assertions, and avoid making stupid, time consuming mistakes that the compiler could find immediately.

My best co-worker listens to me talk about this experience, and shakes his head because I attacked a refactoring all at once that was too large for his tastes, which is probably true. It was risky, but yea though I walk through the valley of the shadow of defects, my tests AND my *compiler*, they comfort me!

-- JeffBay ''

I have no experience with TypefulProgramming, so I can't comment on that. But regarding what about static type can be a hindrance, I'll pipe up here: Static typing can make refactoring tremendously difficult, especially if your programming style requires a lot of discovery-through-refactoring. Most of the time you understand your domain well-enough that you don't necessarily need this EvolutionaryDesign, but if you push yourself hard enough you'll get to that point sooner or later. Case in point: I tried coding a custom modification of a Markov-chaining algorithm in Java, and I couldn't do it. The drag on small refactoring moves made it pretty much impossible for me to discover the solution.

Now, maybe you could argue that I might have been well-served by a more-intensely theoretical analysis of the problem. Maybe I should have even written a paper, who knows. All I know is I switched over to a dynamic typing language -- RubyLanguage, in this case -- and I was able to solve the problem in less than a day. -- francis

You don't need to write a paper, Francis, but a few lines of code would have been nice. Not knowing exactly what you were trying to do, I'd still make a wild guess that you were suffering more from lack of functions as first class objects than from static typing. In any case, should you have been programming in a typeful language, my bet is that your code and refactoring moves would've looked the same as in Ruby (or maybe a little better since you have functions and PatternMatching).

So, if you show me a few lines of RubyLanguage code, I'll show you the same lines in say, Ocaml. And I'll ask you to please point out where static typing is a hindrance.

Sorry, my RubyLanguage code is currently in a Linux box that I haven't set up since I moved apartments. Otherwise I would've dug it up for you. Having functions be first class objects would be nice, I'd imagine, but I'm quite sure that's not what I was suffering from in Java. What I was suffering from is this: If you're doing a lot of refactoring that's so drastic that it involves constantly changing a method's interface, then static typing can be a problem. For example, if you have a class A with a method doSomething(B, C), and you decide it needs to be doSomething(B, D) -- that's a piece of cake in Ruby. You change the tests involved and you fix the problem only in your one class, then you run your tests for everything, and change all the clients of class A.

This doesn't quite work out in Java -- you'll have to change all class A's client code before you even compile the whole thing, never mind test it. So refactoring becomes much more labor-intensive. A refactoring browser could help here, but only so much. And when you're exploring a domain that's fuzzy to you, trying to put it down into code, it's really distracting to have to spend lots of time satisfying the compiler. Distracting enough that you might lose your train of thought entirely.

I don't know if this is a big digression or not -- like I said before, the TypefulProgramming you're describing on this page doesn't quite seem to be static-typing in the way Java does it. -- francis

Let's say you wanted first a method to return

return this.doTheAThing(b.doTheBThing(true), c.doTheCThing(2));

and next you change your mind and want to refactor to:

return this.doTheAThing(b.doTheBThing(true), d.doTheDThing("mystring));

Of course, in Java you'd have to do heavy work to declare some A B C D classes or interfaces. But Java is irrelevant in the context of TypefulProgramming.

Here's how it works in OCaml using TypeInference.Create a function (because it works slightly easier with functions than with methods):

 let computeSomething a b c = a#doTheAThing ( b#doTheBThing true ) (c#doTheCThing 2);;

And this will be typed (reported back by the interactive interpreter) as val computeSomething :

  < doTheAThing : 'a -> 'b -> 'c; .. > ->
  < doTheBThing : bool -> 'a; .. > -> < doTheCThing : int -> 'b; .. > -> 'c =
  <fun>

In translation that is a function that takes three objects of any class types at all subject to the constraints: a's class must have a method doTheAThing , b's class must have a method doTheBThing, c's must have a method doTheCThing, doTheBthing method of b's type must take a bool, and it's return type must be compatible with the type of doTheAThing's first argument, similar for doTheCThing. The resulting type is whatever a's doTheAThing result type will be when you actually invoke this function.

Please note that you don't need to declare any interfaces a priori, you have a perfectly valid function definition, and the code looks very much the same like the code you'd write in a dynamically typed language.

Next, let's change our mind and refactor:

 let computeSomething a b d= a#doTheAThing (b#doTheBThing true) (d#doTheDThing "MyString?");;

And we get back the message:

  val computeSomething :
< doTheAThing : 'a -> 'b -> 'c; .. > ->
< doTheBThing : bool -> 'a; .. > ->
< doTheDThing : string -> 'b; .. > -> 'c = <fun>

Well, you get the drift. The basic thing is that most often than not (I'd venture a guess of >90% of the cases) you let the type system connect the dots at compile time. This provides you a very effective and useful safety net when you venture into more powerful (albeit more complex abstraction) like higher order functions. --Costin

Looks like that would also fix the MockObject problem described in StaticTypesHinderTesting. --ChristianTaubman

It's hard to imagine code with a signature change working at all even in a dynamic language. The code will be expecting arguments that are no longer there. Any you may miss some instances that need changing. You have unit tests you say? Well they will all fail too unless you change the code. --AnonymousDonor

In a dynamic language you'll be able to get the unit test for the changed class passing. Then you can move on and deal with integrating it with the rest of the code. --ChristianTaubman

Well, I can test my class in isolation as well, and the move on to changing what broke. No difference. --AnonymousDonor

Here's what it looks like in a dynamically typed language:

Change the unit-test to reflect the new method signature.
Run the test to confirm that the test fails.
Change the code to satisfy the test.
Run the test to confirm that it passes.
Run all the other tests to see which other classes you've broken.
Fix their method calls.

But in a statically typed language:

Change the unit-test to reflect the new method signature.
Change code in all clients of this class to satisfy the compiler.
Run the test to confirm that it fails ...

That added step 2 is the refactoring drag I've been trying to explain. Unless I'm doing something really wrong, you can't even fix the one class before changing all of its clients, since you can't run unit-tests on code that won't compile. -- francis

Again, you don't have to compile the entire code base all the time. You can test your class in isolation, then compile and fix everything else. I do this *all* the time. --AnonymousDonor

Yes. But sometimes you can't test that class in isolation because it references something that references something else that references a class that uses the one you've changed. Maybe this indicates poor design but it happens. --ChristianTaubman

Since you are presumably refactoring then fix it. You should use an abstract class to decouple the components. Arguing for dynamic typing as way to get around bad code seems especially weak. --AnonymousDonor

I like to refactor one thing at a time. After this refactoring I might go and do an ExtractInterface or something to fix the coupling problem, but for now I'm stuck with it. --ChristianTaubman

I gave up on "pure" unit testing because of this and instead test classes in isolation from above, but using the production classes below it. This avoids having to write and maintain stubs, but the cost is sometimes having to write code to force the lower level classes into a starting configuration. This may also force the addition of some test methods in your production class to look at embedded classes. --AnonymousDonor

Something else jumps out at me from the example Francis gave. If you are able to test your class in isolation, the steps in a static language become:

Change the unit-test to reflect the new method signature.
Run the test to confirm that it fails. Wait a second, the test didn't even get to run. I'm getting some obscure error message about a "Semantic Error". What test is the problem in? Oh, it's just compilation failing. Add an empty stub method to the class. Another darn compilation error. Ahh, forgot to return something out of the stub method. Ok, should be all set now.
Run the test to confirm that it fails ... What was I doing again?

The compilation error indicates the test failed, but generally I find compilation error messages harder to understand than error messages that come out of a running test, even it if is something like "Did not understand." --ChristianTaubman

This compilation failure problem is obsolete in VAJ and EclipseIde, classes with compilation error will still run, you will just get exceptions when you reach those lines with error, just like in dynamic typing.

Doesn't that indicate that the people who make VAJ and EclipseIde prefer dynamic typing behaviour in this case?

Perhaps, since VAJ is originally written in Smalltalk, but the point is compilation failure is an development environment problem not due to static typing. Any compiler for static type can insert exceptions or abort() at compilation error points and still complete the compile instead of quitting, and VAJ/Eclipse is a real life example of that. If compiler error is what makes StaticTypingHindersRefactoring, the same case can be made for SyntaxCheckingHindersRefactoring?.

I don't think there is ever a need to break the tests during refactoring. Try changing the method signature in the following manner.

Take the existing method body, add a new, temporary method name, and delegate the temporary method to the old method name.
Run the existing tests, everything should pass as before.
Create the new method and signature. The old method should call the new method, which should call the temporary method.
Run the existing tests, everything should pass as before.
Clone the existing test, modify it to call the new method interface.
Run the existing tests plus the new test, everything should pass.
Copy the code from the temporary method into the new method.
Run the tests.
Refactor the new method as necessary.
Run the tests.
Migrate the production code to calling the new method. This may take a lot of schedule time, but does not need to be accomplished in one step. Both methods can coexist.
Remove the test for the old method.
Remove the old method from the class.

Sometimes it's nice to see the new test fail before you make it work.

When refactoring, it is best to avoid failing tests. Nothing new is being added, rather one needs to ensure that existing functionality is preserved across the refactoring. Not breaking tests is crucial to refactoring success.

Whether or not functionality is being changed depends on the scope you pick. We're not (strictly speaking) ReFactoring the class that has this method, we're ReWorking? it. That's why we have to change its UnitTest. We are, however, ReFactoring the larger system that uses this class. So the AcceptanceTests shoudn't have to change. If I'm changing a test, and therefore adding or changing functionality, I often like to initially have the new test fail so I know I've succeeded when it passes.

The scope was defined at the top of this section, changing a method signature. When refactoring, one does not want to change the functionality, therefore the passing test must be preserved. If one starts with a new, broken test, it is unclear that making the new test pass still maintains the old functionality. Remember, this is refactoring not TestDrivenDesign.

Ack, you're right, I had slipped into TDD without thinking about it. Looking back to the statement that started this all off I see it was complaining about problems when doing DiscoveryThroughRefactoring?. I've certainly experienced that, but I'm having a hard time nailing down exactly what the problem was. Perhaps classic ReFactoring is fine, but the inherent messiness of TestDrivenDesign and DiscoveryThroughRefactoring? makes it more likely the compiler is going to start complaining about problems you don't really care about.

Data types provide a quick reference to the available interface.

The biggest advantage to data types is they provide a reference to the available methods that can be used. The data type tells me the class of an object and thus lets me know what functions I can call. The same holds true with primitive data types.

I can't understand why refactoring from static typing will be different from dynamic (of course you do not have compilation problems). If you want to change signature of a class, say if a parameter changes in its type, then anyway you have to update the method so that it can handle the new type properly. Only place where this is a hindrance is where you rename a type. In that case, any IDE worth its salt should help you there. In VAJ, you can rename a class and it will correspondingly rename the corresponding references.

In VAJ, you can rename a class and then answer "yes" to a confirmer that asks if you want to rename the corresponding references. You then descend into refactoring hell, with missed matches, lists that don't update, and an ever-growing pile of edit windows. It is far easier in VAJ to just make the change, save it, and then look for and correct the little "problem" icons.

Well, in my case, it has helped me a lot. Anyway, changing little problems is also O.K and I would say its same as the time you would spend in correcting the failed test cases.

CategoryLanguageTyping