Customization Via Programming

(Another AntiPattern; comments extremely welcome!)

Let's start with the seductively good-sounding pattern:

Problem:

A software system needs to be extensively customized by end users.

Forces:

Each end user needs to do very different kinds of customization. Some customization requires computations well beyond the system being customized. e.g. We use a commercial bug tracking system, but we need to associate individuals with the teams they're on; that's not part of the bug tracking system per se.

Proposed Solution:

Invent a small programming language, which lets users do arbitrary (in at least the Turing equivalence sense) computations. Allow escapes to C, executables, etc. Perhaps write all but the core of the basic system in this languages.

New forces:

There is no such thing as a small programming language! Programming languages need extensive documentation and good debugging support. In my experience, it's not there. (Exception: Microsoft's Word Basic. Note this is the first language for writing platform independent software viruses!) If it's not there, the HammingDistance between a useful configuration and a useless one is slim.

Examples:

It's unusable by itself, "an assembler language for writing documentation systems". That's a feature; you're not supposed to run, troff you're supposed to run, troff -mm or troff -man or something.

Writing a troff macro package is hard! No one, except maybe BrianKernighan, knows how many backslashes to put in a macro; troff programmers use trial and erroruntil they get it right, and even so they can can burned when macros are used in diversions or other macros. Text often disappears, or shows up in funny places, and there's no way to ``ask the system'' what happened.

This program runs as root, so it should be small and simple enough to be completely understood. Maybe by Eric Allman, but not by anyone else. The O'Reilly book runs eight hundred pages! Even with fairly good debugging support, some rules are incredibly hard to get right. (Hewlett Packard has a short list of changes system administrators can make to sendmail.cf and still be supported; beyond that, they're officially on their own.)

Alternatives:

Simple data-driven customization. Microsoft Word and Framemaker (and Scribe long before them) use styles, collections of formatting information that can be applied anywhere.

But TeX does everything troff does, only more, and better. It has its peculiarities, but it is quite programmable. I have been using Word because everybody else uses Word, but I still have many fond memories of LaTeX.

I believe that CustomizationViaProgramming can be a good thing. It is hard to do right, so it is only the right thing to do when you are prepared to pay the expense to do it right. But it can be done right. -RalphJohnson

I see the perceived Problem as a problem...

A software system needs to be extensively customized by end users.

End users should not need to extensively customize a software system. Think about it. The Alternatives given above does the trick... It stresses simple customization. If I develop a little language for my software system, it is for me and other programmers to tweak functionality.

Users that use the little language are no longer users. They become programmers. Users that want to become programmers of software systems that they purchase are asking for trouble and at worst are fooling themselves into thinking that they don't need you (the system developer) or another programmer to extend and support the system.

But why the distinction? Why not ComputerProgrammingForEverybody?

Yeah, this sounds radical, maybe even a bit hostile against users, but -- most people shouldn't work perform surgery on their selves or family members. Me, I have enough problems working with MS Word styles to worry about hacking MS Word Basic. (The programmer in me still wants the neat little languages!!)

My advice: develop the little languages, but don't encourage the users to mess around with them. Give them styles and templates...

-- ToddCora?

I'm in favor of making users into programmers. It's a natural extension to data driven programming. BUT you have to be aware of the dangers. Typically, for example, user changes to the data (like codes and scripts) that control system behavior are not subject to change control, testing in a non-production environment, or any of the other production control procedures that responsible development organizations employ. A user change to the control data takes effect immediately, and there are normally no rollback or recovery procedures.

Users love this, because it makes the system much more responsive (by orders of magnitude) to their demands for functionality changes. And when it corrupts their data, they can still blame the IS department, who's "responsible" for data integrity.

There is no such thing as a small programming language! Programming languages need extensive documentation, and good debugging support. In my experience, it's not there.

No, it is not there, currently. But if it was, would this still be an AntiPattern? And how do you define the terms "Programming" and "Language"?

Consider applying a set of PhotoShop filters in a certain order to a set of images to produce a single final image. This might be expressed formally as Filter((I1 * I2) + I3) but the user never sees this representation. Isn't this "programming"? And isn't the representation of the operation, which the user sees and manipulates, then the "language"?

Once you can take operations and create macros, haven't you defined a function?

I know CustomizationViaProgramming hasn't worked in the past, but I would prefer to believe this is because it's been done terribly rather than being an unworkable solution in the first place. -- EddieEdwards

I like to think of this problem like a spectrum. On the one extreme you have uber-ridgid configuration files, on the opposite you have uber-flexible Turing complete languages. Between these extremes fall mini-languages or glorified configuration files that are not Turing complete but are flexible.

Where on the spectrum your implementation falls is a design decision--too complicated and you're wasting time writing a whole language, too simple and you're not going to be able to meet all the requirements. The idea is to provide just enough flexibility and not an ounce more. The more highly specific a language the better (this is why FoxPro is great--it does data driven applications with style).

A technique I have found useful is a code generator, usually a perl script that reads a __very simple__ grammar and outputs code in a target language. In this way, you can customize in a Turing complete language (the output language) using a non-Turing complete language (the input grammar). LEX and YACC are two programs that do this--they read an input file and output source code in C.

Another good example is this wiki. There are simple commands for styling the page, and that is all.

As for turning users into programmers, don't do it. Computers and software are enough of a problem for many without the looming menace of writing code. Remember, what seems easy to you only seems so because you know how to do it. That's why you're a programmer.

-- JohnWestbrook

CustomizationViaProgramming has had some spectacular successes; Emacs is one of the most notable ones. It is, however, very correct that there is no such thing as a small language for long. Emacs itself has troubles because its language, EmacsLisp, is slow and sometimes difficult to work with for ProgrammingInTheLarge?; also, recent Emacsen have incorporated a system for customizing things without necessarily having to descend into Lisp. (This system is, of course, written in Lisp :)

I would perhaps amend this to be CustomizationViaMyFavoriteToyLanguage? is not a particularly good thing, though. History shows that for successful projects, people will bend the limits of the language over time to such an extent it is less painful to use a real language from the get go; this also addresses your documentation and implementation issues.

-- GrahamHughes

Invent a small programming language...

ToolCommandLanguage, is an extensible, embeddable scripting language. JohnOusterhout designed it to save people some of the effort of designing yet another LittleLanguage.

Another language with similar properties is PythonLanguage.

Python is designed to be simple to embed in applications written in other languages. There is also a restricted execution facility. It's fairly straight forward to extend Python with code written in other languages, so that your Python macros can do something with the data in your application.

There are certainly plenty of applications that need these kinds of extensions. That sendmail and troff are difficult to use hardly voids the entire idea of allowing some kind of macro facility in applications. Although that's not what it said... I do agree that writing a new little language is usually not such a good thing as it might seem at the start.

I think it's much better to embed an existing language like Python or Tcl in your application than to invent a new language. Then your little language has documentation, support, debugging tools and your users will be able to find uses for it that you never imagined (which might be a challenge for the base applications ;-).

Another thought that comes to my mind is that the OpenSource movement somehow seems to blur the distinction between programmers and users. EddieEdwards mentioned PhotoShop above, and it's OpenSource cousin Gimp seems to make a lot out of it's scripting ability, and the open API for writing plugins. For instance there are web sites that produce logotypes and buttons from written text and some style selections using Gimp. This kind of scripting ability certainly beats any GUI if you want to try out custom icons in many different styles or if you need to produce images on the fly based on content that is generated from a program etc.

I'm probably mistaken but it seems to me that Smalltalk people (there are some of you here, right?) seem very focused on interactive software, the kind of programs that do nothing unless you tell them too, and go back to idle as soon as you turn your head. Now, I'm a very lazy person, so I like my computers and programs to do the work when I play with my son and only request my attention when things get to difficult for them. Ok, I'm not quite there yet, but Linux and Python certainly help me more to reach this than Windows or Mac and the popular GUI tools there.

-- MagnusLycka?

In my experience, customization by programming is done -again- by programmers. But at least, you don't need to recompile that huge program because of a little formula change.

We must mention mIRC here for its bad but very very successful programming facility. -- SavasAlparslan

Each end user needs to do very different kinds of customization

This is a very real situation. If creating a mini-programming language best addresses the problem, then do it, and don't complain about the non-programming masses taking advantage of it. An alternative you may want to explore is evaluate whether you are trying to address different market segments and it is more appropriate to produce multiple custom solutions. -- WayneMack

I have written large enterprise systems that require flexibility when it comes to the crunch. In many cases the users themselves may never touch such a language, but it allows your company to provide support or consultancy to cater for their differing requirements. Otherwise you build bespoke systems per client, which in the products I wrote is just totally without value. -- AlexCraxton

CategoryAntiPattern

EditText of this page (last edited January 22, 2005) or FindPage with title or text search