Jit Is For Prototypes

JIT = JustInTimeCompilation

JIT Compilers are great for prototyping, but a commercial product, especially a high performance computer game, should be compiled using a full-strength compiler and should ship as binary.

Problem:

We require

High performance for example the software product is a game
A rapid development environment
Some protection against our source code being ripped-off

Therefore:

Develop in a rapid turn around environment. However, as code design stabilizes it should migrate to pre-compiled modules, and the final shipping product should be entirely precompiled using a full-strength compiler. By doing so:

Your final code is faster, maybe by as much as 10% for there is no JIT compilation overhead.
Your final code is smaller, you do not need storage for a compiler, its methods, for the 'source code' nor for working space for compilation.
You do not have to worry about making the compiler work on a diverse range of machines. If you ship a program that uses JIT, you do have to.
Source code stays in-house.

For best results the JIT compiler and full-strength compiler should compile exactly the same language, indeed one compiler with a switch that lets you set JIT mode or pre-compile mode could well be best. This way very little additional work is involved in making the transition from prototyping to production version.

Running the compiler at development time gives the compiler plenty of resources (time and memory) to examine and cross-reference the code, to determine the best possible optimizations. Optimizing polymorphic behavior into conditional tests, for example, is possible.
Some traditional techniques to improve locality of reference have only been done at compile time (not run time). Consider traditional techniques used to link overlays (an old technique used on machines with limited RAM and no virtual memory management).
Reliability can be an issue: How can you be sure that you'll get the same code, the same functionality, and not discover new bugs in your compiler/linker, when you may "compile/link" your program in a different order every time you run it?

But:

As JIT technology improves, the justifications for this approach may be reduced:

An overhead of 10% may be a massive overestimate.
Potentially JIT compilers can perform significant optimizations that traditional (C++) compilers cannot, because more information is available at run time.
- Example: This is how the SynthesisOs does its optimizations -- when you open a file, it generates read and write functions optimized for parameters you've selected (like buffer size and filtering options) and also optimized based on what other files you have open at the time.
Space is rarely at such a premium.
Pre-compiled code makes a fixed trade off between exe size and efficiency. A JIT compiler that uses a caching strategy for compiled code can make a dynamic trade off, based on the memory actually available.
JIT compilation may make adapting to different platforms easier.
JIT source code can be encrypted / hidden so as to makes it at least as hard to reverse engineer as compiled code.

The page title is inspired by SpecializationIsForInsects. There is a thematic link too, in that:

Whilst developing flexibility/generality of the code is important, but..
A shipping product is specialized, the design decisions have been made.

See Also: InterpretersAreForTesting

Discussion

[Initially between EddieEdwards and JamesCrook, but now made OpenAuthor]

As I see it the JitIsForPrototypes approach is a temporary phase, and it really comes from the immaturity of JIT technology. If space and speed overheads for JIT compilation were demonstrably below the 1% level, the other advantages of JIT for production code as well as 'just for prototypes' would shine through.

Q: Games programmers in general want speed above all else, and when they want to cut 10% off their time, they will say "why does this stupid *JIT compiler* take 10% of runtime to begin with?"

A1: Done well JIT should take closer to 1%. On code you run a thousand times the JIT overhead is tiny in percentage terms. On code you run once performance is nearly irrelevant - in some systems such code is interpreted as doing so reduces transfer times and frees memory. JIT compilers can potentially be very efficient. Conventional compilers spend MOST of their time in tokenization and file I/O. A JIT compiler does not have to do that. Target speed for a JIT compiler would be around 10,000 lines per second on a P100. [Compare Rick Booth's figure of 50,000 lines assembler per second on a P100 including tokenization].

A2: Think of JIT compilation as a very advanced method for compressing / decompressing exes.

A3: A collision detection demo might convince skeptics of JITs ability to optimize beyond what a normal compile time compiler can do. This source code is table driven and can detect collisions between two objects. It is fast when objects are far apart but gets slower the closer and more complex the decision is. The JIT derives custom code for these decisions. Moreover, only the code for collisions between objects which have actually come close together gets generated, (for the relative displacement directions in which they have approached). This is a 'virtual program'. Were it written out in full it would be impractically large.

This page made me immediately think of the SynthesisOs. (But I see you've already noticed it.) -- JeffGrigg

CategoryOptimization