Is Scheme Suitable For Developing Larger Systems

Is the SchemeLanguage suitable for developing larger systems? The reason I ask is the following: suppose that I have the following definitions, which are supposed to define an abstract datatype:

 (define (make-customer name telephone)
        (cons name telephone))

 (define customer-name car)
 (define customer-telephone cdr)

A bit too simplistic, but you get the idea. Now if I want to add some more data later, e.g. an e-mail address, this is extended to:

 (define (make-customer name telephone email)
        (list name telephone email))

 (define customer-name car)
 (define customer-telephone cadr)
 (define customer-email caddr)

OK, so any code that used customer-name and customer-telephone will be fine after this change. But what if someone used car and cdr directly, instead of going through the accessor functions? Before the change, all UnitTests still work if someone breaks the abstraction layer.

Is there a way to prevent this accidental breaking of abstraction to happen? Can we perhaps devise some UnitTests that check for this? Or do we just have to live with it? It seems to become a rather big issue when one tries to create a large system.

-- StephanHouben

This seems like an odd objection: there is a mechanism (described below) for enforcing this kind of information hiding. That the language does not enforce its use is probably a good thing. Neither CeePlusPlus nor JavaLanguage, nor CommonLisp for that matter, enforce their available mechanisms for information hiding. Does any language? EiffelLanguage maybe?

Yes there is a way to stop this. Change from using simple functions to using closures. If the 'customer' is not a list, but a function with access to the list that others cannot see, then you can use it in an object-oriented style and rely on the encapsulation.

 (define (make-customer name telephone)
        (lambda (msg)
          (case msg
                ((name) name)
                ((telephone) telephone))))

 (define customer (make-customer "Kent Beck" "1-800-SmallTalk"))

 (customer 'name) => Kent Beck

I may the syntax a bit off here, there are schemers hanging around like ChristianLemburg who'll be able to correct me. The point is though, that you can no longer directly access the name and telephone number.

Thanks! This is cool! -- StephanHouben

[Many Schemes also provide a range of other data structures so that you don't have to emulate everything with closures.]

Anyway, our company did have a big system built with a scheme engine at the core - unfortunately the runtime leaked memory so they replaced it with a new language for the job. For my sins I wrote the program to turn the scheme into this lingo.

You mechanically changed Scheme into some other language? May I ask what that other language was?

It was called 'DSL' (Dave's Scripting Language) for the guy here who wrote it. It was very much a vertical language - over the two years since they first used the scheme engine they had gained enough experience to know exactly what was needed from the scripting engine. I wrote the translator twice - in Java and also in Perl; eventually we used the perl version because it was faster, and could do a more complete conversion (though the java code was cleaner).

It was pretty brutal really - while most people had used the functions 'they were supposed to' some people had hacked in real scheme (not quite as bad as the situation you describe above, but nearly!), so I only converted the subsets of scheme that were actually used. DSL didn't have nested scope, but all of the scheme code had used this, so I handled it by adding numbers to uniquely identify symbols as I flattened the scripts. It was a bit of fun, but if it sounds like a hack that's because it was.

I have to say that the Right Tool for this job was probably scheme itself, but my scheme wasn't nearly good enough. (I did write a skeleton program in this direction but my progress on the other two versions was much, much faster).

The system the scheme engine, and later the dsl engine, were deployed in was big, mission critical, and would have got us on the front pages of the national newspapers if it had fallen over. Hence why they were so keen to get the memory-leaking scheme runtime out. It's also the reason why I haven't signed this, or identified the system more directly, as I seriously doubt if my employers would be happy about me talking about its bugs...

One more question: why didn't you switch to another Scheme implementation that didn't leak memory? It's not as if there's a lack of Scheme implementations; every Comp.Sci. student probably has to write 2 of them during the first year. -- StephanHouben

: Really? I've been trying to think about how to write one (in some language other than Scheme) for some time, and I can't figure out how to do it.
: I have two important books to help me: StructureAndInterpretationOfComputerPrograms and LispInSmallPieces. But they use Scheme itself, sometimes with extensions. One of them produces really bad C code as output. -- EdwardKiser

What's wrong with C output? To write a Scheme interpreter you basically need to implement the primitives for the register machine in SICP. From memory the main things are read, closure representation and garbage collection. Then you can write the rest in Scheme. I don't see why you'd want to write the whole thing in any language other than Scheme - it's more trouble than it's worth. Have you looked at the many Scheme implementations available. MzScheme (my favourite), Gambit and Bigloo all come with compilers to C. -- NoelWelsh

: Nothing is wrong with the general idea C output from a Scheme compiler, but the C output from the compiler in LispInSmallPieces was really bad. The author used C function calls as if they were tail calls; this of course would eventually overflow the stack; he then recommended the use of setjmp and longjmp to "adjust" the stack every now and then. I thought this was horribly terribly ugly, but then, my "beautiful" solution was RunAndReturnSuccessor.
: I have since implemented a small Scheme core in Java using this object framework I dubbed the ExpressionObjectModel?. It supports call/cc! But I want a C++ implementation, so that it's easier to communicate with the libraries and APIs available from C++. The ExpressionObjectModel? uses functionality from Java which C++ does not have, so it's not portable to C++. I am still working on it, very slowly... -- EdwardKiser

That "setjmp/longjmp" mechanism sounds like HenryBaker's "CheneyOnTheMta" algorithm. It lets you use the the C stack as the first generation of a generational garbage collector. There's a Scheme implementation called "Chicken" (http://www.call-with-current-continuation.org) that uses "Cheney on the MTA", re-writing code in ContinuationPassingStyle, then compiling it to C.

Another (if a bit heavier?) source text has become available on-line at http://research.microsoft.com/%7Esimonpj/papers/slpj-book-1987/.

Is Scheme suitable for developing larger systems?

Depends on what you mean by "larger". The ACIS SolidModeller? (the engine underneath AutoCad and a bunch of other full-strength CAD/CAM applications) wraps its C++ core in Scheme. The DocumentStyleSemanticsAndSpecificationLanguage is written in (a dialect of) Scheme.

Going back to the example code above, why not use an alist for each customer. Since the alist is dynamically extendable and only accessed with you have the customer, why would using an alist not work? -- MikeEggleston

If there is a language around which is suitable for large projects, I have yet to hear of it - and I know more languages than many programmers are aware exist. Scheme does about as well as any of them, and considerably better than most. - JayOsako

Naturally many implementations these days also have class/object models. A few seem to aim for something like CommonLispObjectSystem.

CategoryProgrammingLanguage CategoryScheme