System Mutation

SystemMutation is the practice of changing (or customizing) the existing behaviors of a system (in particular, a programming language) to achieve some end (usually for the convenience of the developers, not the customers)--especially in manners where such mutation is inappropriate. Examples include:

Recklessly extending Object or Behavior in SmalltalkLanguage
Changing global operator new/delete in C++; replacing malloc() and free() in C.

Note that extending a language/system in intended ways (by writing new functions, macros, classes, objects) isn't SystemMutation; SystemMutation consists only of causing existing operations on existing datatypes to have different behavior.

Often an example of ThreeStarProgramming, though not always. Some examples where this is useful:

Debugging. The main reason for replacing malloc() and free() above is to use a tracing version (or a GarbageCollector-based tool such as Purify). Nothing wrong with this at all.
If done in the application/framework, rather than in a library. The HighlanderPrinciple applies here. A common rule in C++ (for overloading global new and delete) is that "the module that has main() is the only module where this may be done". Otherwise you run the risk of MutuallyIncompatibleExtensions being propagated.
Customization to a particular environment; in particular if the customization concerns a rather specific area of functionality and is both limited in its scope and orthogonal to the rest of the system.
Fixing of defects. Sometimes acquiring a repaired-system from the source is not possible or practical; modifying an existing system is the only way. (OTOH, it can be argued that bringing a faulty system into spec is fundamentally different than modifying a system which was correct to begin with.)
Development in an ImageBasedLanguage. In a EditCompileLinkRun language, each successive invokation of the program "starts fresh"; changes of the program definition between successive runs don't affect each other (notwithstanding persistent state and SchemaEvolution). With an ImageBasedLanguage, mutation of an existing system is how development gets done.

When I change the existing operations on, say, KeyboardController? (an existing class) so that it supports Win standard key bindings, when those changes are captured in an Envy application or subapplication, so that dependencies are maintained and changes are atomically loaded or unloaded, and when I satisfy a customer requirement (GUI compatibility) as a result, I have taken powerful advantage of important functionality. I do the same when I use the same mechanism and process to identify and correct bugs in the system "as shipped" from the vendor, while waiting for a new version from the vendor that incorporates those fixes. The vendor uses the same facility to provide up-to-date workarounds for problems that need immediate responses.

Those are all legitimate reasons, I have added them to the list above. Good call.

Your concern strikes me as mythical, rather than pragmatic.

Not mythical at all. In numerous times in my career, I have been called upon to perform maintenance of or extension to a system which was worked on by a ThreeStarProgrammer who saw fit to redesign the world to his liking. Or to write code using different libraries from different vendors (addressing orthogonal problem domains) which make gratuitous assumptions that they own the world. Such can make aggregating systems difficult or impossible. Had the library designers shown a little restraint, much wailing and gnashing of teeth could have been saved.

The key is to maintain professional discipline, as is needed in any engineering endeavor.

Agreed. However, the call to maintain professional discipline doesn't make redundant more specific advice in specific problem areas--and in my experience, mindless redesign of the world is an area which professional discipline requires careful treading. While it may solve your problem elegantly, it may make the lives of others miserable. If you are writing application code (the ultimate SW deliverable) and no other programmers require your stuff to get their job done, go ahead. If, on the other hand, you are writing reusable components, libraries, and the like--and your customers are other programmers--then writing PinkyAndTheBrain software is doing your customers a grave disservice. And I'm not talking about customizing the keyboard controller for Windows here--I'm talking about the hackery discussed in DelegationInSmalltalk and other places, where the fundamental behavior of the language itself is changed by the applications programmer.

I've had to provide DelegationInSmalltalk when I needed to extend behaviors in classes that I could not subclass. This happens much more frequently in Java -- what happens when you need to extend a class marked "final"? The DecoratorPattern doesn't require "hackery" and is very helpful.

[Depends on why the class is marked final; java.lang.String is marked final for a very good reason. OTOH, many classes are marked final for bad reasons--chief among these seems to be "If I make this class/method final, then the compiler can inline it and my code will run faster". Things to do with inappropriate final classes/methods:

Go find the developer, and ask him/her to fix it. Assumes that the developer of the class is available, might not be the case for a third-party library.
CopyAndPasteProgramming. Normally, I wouldn't recommend this.... If you have the source for the class, make a copy and call it MyClass (or something similar), remove the obnoxious final references. Useful especially when the name of the offending class is only used in new clauses--all usage of the class is through references to a base class.

Perhaps this little side-discussion should be its own page...RefactorMe. I think here is ok for now, maybe later.

I find that applying the DecoratorPattern, as mentioned above, gives much better results than either of the two alternatives you mentioned. In particular, CopyAndPasteProgramming is surely worse.

DecoratorPattern is clearly better than CopyAndPasteProgramming...assuming that you don't need subtyping. (In other words, you don't need your modified class A' to be a subtype of A). If you do, then pestering the original designer (or God forbid, hacking the .class file!) are your only choices, though copy-and-past can come close (if it suffices that A' is a subtype of the base types of A, but A' need not be a subtype of A itself).

]

Only reinvent the world if you are sure that you own it.

Look through the world in whatever color glasses you prefer.

I'm talking about the hackery ... where the fundamental behavior of the language itself is changed by the applications programmer.

In both Lisp and Smalltalk, the "fundamental behavior of the language itself is to encourage change in the fundamental behavior of the language itself. This is why these languages exist. Smalltalk intentionally moves conditional and loop constructs from the language to the environment, so that a developer can clearly express behavior. This allows, for example, code like this:

 aVariableOrNil ifNil: [anObject doSomething].

instead of

 aVariable isNil ifTrue: [anObject doSomething].

The result is that the community has evolved various iteration and looping constructs -- here are some examples:

 [anObject isSomethingTrue] whileTrueDo: ["Do something exciting here"].

 [anObject isSomethingTrue] untilTrueDo: ["Do something exciting here"].

 ["Do something exciting here"] untilTrue: [anObject isSomethingTrue].

 ["Do something exciting here"] whileTrue: [anObject isSomethingTrue].

 aStartIndex to: aStopIndex do: [anIndex | anObject doSomethingWithIndex: anIndex].

This stands in contrast with the various hard-coded control constructs of Java, C, or perl (although perl of course simply floods the space with every variation imaginable]. The syntax has been made simpler, by moving constructs from the syntax to the environment.

This style of code may create some discomfort, especially initially, and it can certainly be abused (as can anything powerful), but it does not follow that this is "bad", "hackery", or -- one of my favorites -- "less pragmatic".

[And for the examples above--I would agree with you; these are extensions more then mutations. While they do involve modifications to Object, they don't change the existing behavior of Object, and they are of sufficiently limited scope that they aren't likely to pollute the atmosphere. Furthermore, while they may or may not be standardized--there is a general consensus in the Smalltalk community that these are good extensions.

Some of the other changes to Smalltalk (or other languages) that I hear about, OTOH...ye gods. Especially if done by library code... ]

Thanks for staying with me this far. So now try this on. Because everything in Smalltalk is an object, behavior that started as just plain behavior can be morphed into "frameworks" by careful extending. These are the way that many frameworks come into existence. One path is for a class way out at the leaves of the class tree, installed by a very specialized seldom-used application, to override a system message and do something new and useful with it. As other people learn about it, and say "HEY! That solves a problem I have too", the overrides migrate up the class tree, usually multiplying along the way. Sometimes groups of messages do the same. It's still controlled, managed, and backwards-compatible.

#printString example In Smalltalk, every object answers a string that identifies itself in a browser, debugger, or similar pane. This string is answered in response to the #printString message on class Object. Many times, a particular class will want to answer something special in response. A very long time ago, our ancestors learned that almost every situation that causes a class to want to change its printString also wants it to change the way it gets written to other places as well. The printString method was therefore modified so that it calls a method called #printOn:, which is supplied a stream as an argument. The community discovered that it can override #printOn:, from class Object, and do whatever it wants -- and the whole system will automatically get the change. If this is a "framework", then the entire environment is the framework. It's simply part of the Smalltalk culture -- look how the system does something, find a safe way to change it, and do it.
#error: example Old versions of Smalltalk, created before exception systems existed, used Object>>error: to inform the user of problems. Typically, the implementation posted a complainer dialog, aborted the operation with a log entry, or something like that. Applications, in the old days, were "hooked up" to the system by overriding #error:. Then, exception systems started being created. Long before the vendors hooked exception systems into the environment, the community realized that Object>>error: could be *changed* so that it signaled an exception, rather than anything else. The old code for reporting the exception was then moved into a handler for whatever error was raised. This is deep surgery on the environment -- adding an exception system and changing the system error reporting behavior is certainly as deep as providing automatic delegation -- and was done smoothly, gracefully, and without major problems. The result was that applications could define their own exceptions, change handling of system exceptions, even change the SYSTEM handling of system exceptions, all seamlessly and all without special dispensation from any vendors.

To wrap up, there is no doubt that reckless changes to the system can create problems. But making pervasive changes to system behavior is far less dangerous than some of your earlier comments suggest. The overwhelming majority of the changes made by the community, including the sort of thing discussed in DelegationInSmalltalk, fall into what this page describes as "extensions".

This sort of incremental development--where extensions are tried out, passed around, and either are accepted by the community or rejected--is a good use of SystemMutation. Nobody is suggesting that systems should reach a point of done-ness and be frozen ever after.

OTOH, what if the Smalltalk community were more like the communities of Java or C++--larger, far more diverse, and far less connected? (If many in the Smalltalk community were to get their fondest wish--and Smalltalk were to become a (or the) de-facto standard language for many problem domains--and thus acquire an ArmyOfProgrammers and hundreds of vendors peddling all sorts of stuff--this would likely happen). The model of community-driven incremental development would start running into scalability problems. MutuallyIncompatibleExtensions would become more and more commonplace, as various vendors were no longer interested in sharing (and instead tried to lock each other out). There are many organizational structures which can deal with this; but they all involve some imposition of order--whether it be via a standards body (à la C++) or by a dominant vendor (such as Java).

Kind of like a small town that gets too popular and starts growing. Sooner or later, it isn't a small town anymore. Or as DonHenley? put it, "Call some place paradise, kiss it goodbye."