Continuous Integration Gem Stone Style

The VcapsProject uses GemStone as the persistence mechanism. Problem is that most of the SmalltalkLanguage code in the systems resides in GemStone. We use GemKit to manage our GemStone code in EnvyDeveloper 4.0. It works reasonably well. There are still a few bugs, but we work in such a way that they do not bother us. Without GemKit it would be much harder.

Many projects such as the ChryslerComprehensiveCompensation project have taken the approach of coding, testing, and managing code in the VisualWorks environment. On a regular basis all the code is dumped into GemStone, tested and released to production.

It was decided here to code everything that was to be in GemStone in GemStone only. Several problems present themselves. How do we do FrequentReleases and ContinuousIntegration? Each developer must be able to work in a stable version of the system until integration is desired. Each developer must be able to obtain the latest version of the system and merge changes quickly. Each developer must be able to find differences between modified code and released code. Developers must be able to quickly resolve conflicts.

Our solution was to create a GemStone database for each developer. We also created a database for the SingleReleasePoint workstation. Each database has the same user id created. Each developer then used the same GemStone user id on each of the databases. Three scripts were written. One would backup the extents (physical files) of the SingleReleasePoint workstation. The second would restore the database of the SingleReleasePoint workstation from the backup. The third would replace a developer database with a copy of the backup of the SingleReleasePoint database.

FrequentReleases is accomplished by each developer taking turns at the SingleReleasePoint workstation. Changes are loaded into both Envy and GemStone. Any last minute integration is done. Changes are committed into GemStone. The UnitTests are run. (Most of our UnitTests are run in GemStone.) If a score of 100% is achieved then the changes are released into Envy and the database backup script is run. At this point the backup of the database and the Envy released versions are exactly in sync. This is very important and requires discipline to maintain.

If the UnitTest score was not 100% then the code is either fixed or removed. To remove the offending code, the restore script is run to put the database back to its original state when the developer first approached the SingleReleasePoint. While it is true that a developer could "back-out" all code changes from GemStone and then re-commit, it is much faster, easier, and more accurate to run the restore script. Then the released versions of code are reloaded to reset Envy. Since the person using the station before you performed exactly the same process, the Envy and GemStone versions remain in sync.

ContinuousIntegration works for each developer by opening Visualworks and loading the latest versions of code from Envy and running the copy script for the database. Now the developer's image and database are identical to those on the SingleReleasePoint workstation (and in sync). Local changes are then added and GemStone is committed as changes are integrated. This method is very fast and only takes a couple minutes.

Every week an empty GemStone database is created for the SingleReleasePoint workstation. All of the code and data is dumped into it fresh, UnitTests are run. This ensures any out of sync code will be brought back into sync. We accept that mistakes happen.

We do things slightly differently.

Our developers also have their own GemStone databases, but we manage our Smalltalk code in CVS [ConcurrentVersionsSystem]. Whenever someone edits a class or method in VisualWorks, the updated .gs file is written to the checked-out CVS tree. We have a script that files the .gs files back into GemStone.

Instead of using a single integration box, we manage the configuration of our LinuxOs development boxes so that they are all the same, and each pair can integrate on any computer. Then the other pairs merge via CVS and 'make' fresh GemStone databases one their boxes.

The neat thing is that this script can either start off with a backup of the pristine, empty database that ships with GemStone, or with a backup of production. We can 'make' either a development database (recent code but no real data), our a upgraded version of the production database (recent code + our live production data, migrated) on our development boxes. The same script is part of our deployment process, when we deploy to the real live production server.

-- PieterNagel