Data Driven Programs

Data Driven Programs are programs which process data files whose contents cause the program to do something different. The extreme case is an interpreter and the interpretable program files.

There are a lot of other variants. For example, we use a Data Driven Program to operate test systems. The inputs are test scripts which tell the system what instruments to set and read. The primitives are things like SET(frequency,amplitude,generator) and MEASURE(from freq, to freq, at increment) etc. The target is a file which captures the data in a delimited format that can be readily loaded into a spreadsheet for display and graphing. Simple and effective.

Another example was a simulation program which used database files to set up the order of battle and the elements of the order of battle. The data files were a hierarchically ordered set of things like: ships, armaments and sensors on the ships, orders for the ships, etc. Once the data files were created you could just type RUN and the program would read the data files and run the simulation. Very flexible and accessible to non-computer people because the data files were easily formated and prepared ASCII text files which anyone could understand and prepare.

As you can see -- however, each of these examples is of a specialized interpreted script. A third example is a portable instrument which we manufacture. It's user interface is constructed from a scripting language which is embedded in the instrument. It pastes labels on the screen, buttons, etc. collects data arrays and displays them. The look and feel of the instrument can be changed in a very short time. This is primarily used for extensions of the instrument's functionality.

I'm rather fond of this architecture for making small useful applications especially in a test environment. Anyone else have similar examples? --RaySchneider

There is also a good discussion of this in JonBentley's ProgrammingPearls books - he calls these LittleLanguages.

Perhaps this should be called ScriptDrivenPrograms?, or FileDrivenPrograms?? For me the word "data" does not distinguish these kinds of data from rich, interactive event data. The key difference being between interactive and batch modes -- DaveHarris

There may be not much difference between data coming from a file or from a keyboard/mouse. But the data you describe are richer than what is normally implied by "data-driven" methods (opposed to function-driven as I remember from a former life). For other examples -- but we don't call them data-driven -- have a look at http://www.argo.be/OoFrame/framework.pdf (long version) or http://www.argo.be/OoFrame/oops97tp.pdf We interpret "data", rules, authorisation, content, context, system, business and meta-data, all stored in a database. -- MartineDevos

I have collected some patterns for scripted applications and documented their relationships with other patterns, such as those by the GOF. Take a look at http://www.doc.ic.ac.uk/~np2/patterns/scripting -- NatPryce

I have been using the idea of data driven programs with great success so far in the game industry. It speeds our development cycle tremendously. One of the hard things about doing games is that it requires tremendous and continuous cooperation between the programming staff and the art/sound/game design staff. We are learning that rather than specifically programming all the game characters we want, we can program the basic behaviors we want. By programming the game in such a way that the actual characters are created from script files (both art, sounds and behavior tables) new game play can be created at run time by non-programmers. This approach is the only way to go. --JeromeKaraganis

The data doesn't have to be stored outside the program. Making the code data-driven from a data structure which is within the program (but readily changed) often makes the code smaller, simpler and more flexible.

The antithesis of DataDrivenPrograms are programs which include a lot of information about what needs to be done rather than how to do it. -- JamesYoungman

Another use would be that we can layer DataDrivenPrograms on top of another. This way, a module is configurable by a particular file, another module that depends and lays on top of this module can be configured by another file. In a way, if we use XML 'A' for say an 'X' module, another XML file 'B' would make use of this 'X' module in its configuration entry. So 'B' would contain 'X' and its configuration reference and so on. Hence 'X' with its configuration 'A' is a resuable component. -- VhIndukumar

A good tool for creating DataDrivenPrograms is YAML (YamlAintMarkupLanguage). The beauty of YAML is you can start with a code-driven program and turn it into a data-driven program. For example, suppose you were writing a test driver for a "median" subroutine in PerlLanguage:

    my @tests = (
        { input => [1, 3, 5], median => 3 },
        { input => [4, 5, 6, 10], median => 5.5},
        { input => [], median => undef },
        { input => [13], median => 13 }
    );

    for my $test (@tests) {
        die unless median(@{$test->{input}}) == $test->{median};
    }

    # sub median {} not shown

With YAML you change the program to this data-driven version:

    my @tests = YAML::Load(<<END);
    ---
    input: [1, 3, 5]
    median: 3
    ---
    input: [4, 5, 6, 10]
    median: 5.5
    ---
    input: []
    median: ~
    ---
    input: [13]
    median: 13
    END

    for my $test (@tests) {
        die unless median(@{$test->{input}}) == $test->{median};
    }

YAML changes your code to data. Of course, the YAML looks similar to code, but it's data in the sense that you can extract it to a file that never gets directly interpreted by Perl.

and http://www.faqs.org/docs/artu/ch09s01.html

What about programs that store CODE in the data That's right, in the code. There area plenty of languages that will let you execute script at runtime, so if you store your code in a database, and interact with your code, treating it as data, you will be able to build and expand on your code very quickly, and often get users to help you build the robust code that you need.

I created a hyphenation system using the data-driven technique. Actually I inherited the code initially and expanded on it. The rules of how the hyphenation should work is contained in the data, and the data is highly structured, and it must be exact...so garbage in is guaranteed garbage out. When you need to hyphenate a different languge, the UI allowed to select the diff language, which called up different rows from the table. So one can switch hyphenation, or add new hyphenation, without any code changes. Just add new rows for the new hyphenation rules. The code acts as a Daemon as it does what it does based on the incoming data. So the code becomes "general purpose". I've also developed a validation system using the same concepts as was done for hyphenation....Russ Marano, REM Software, Inc, 602 705 5819