Single Language Operating System

It is typical today that an OperatingSystem directly supports execution of programs of two sorts. One sort is a binary 'executable' file that contains both meta-data for expansion into a virtual memory space and regarding which dependencies must also be loaded. The other sort is a script (or generic support for various scripts using something like the *nix #! (SheBang))... though often scripts are simply associated with some sort of external interpreter (which might, itself, be a script, though this will recurse eventually back to a compiled base).

Fundamentally, the executable files an OperatingSystem supports are simply values in a language... and, for most operating systems today, that language is very low level - below even assembler, processor-specific, indicating only how to set up the memory layout and where to set the PC register's initial value (if that isn't fixed), and where to load other components or how to hook them in (since most OS's today support shared libraries and relocatable code).

The advantages of such a low-level executable language are as follows: (a) obfuscation of code, including both intent and mechanism - this is a perk for commercial release. (b) speed - already run through optimizing compiler for speed, so it will run fast and might even load faster (and will certainly load faster than JIT-compilation). The associated disadvantages are: (a) obfuscation of code, including both intent and mechanism - this is a perk for spyware, adware, trojans, viruses, malware in general; it also allows for code that crashes, code with MT-issues, code with type-safety issues that should be statically detectable; and, further, it runs against the 'ideals' possessed by certain groups (e.g. GNU), (b) speed - code cannot be well optimized based on the data or runtime profiling by the OS.

I've spent hours today contemplating the advantages and disadvantages of using a MUCH higher-level language as the 'base language' the OS understands - higher than machine code, higher than ELF files, higher than assembler, higher than C, and higher than C++. I'm talking a language designed to capture intent as well as mechanism, that describes agents and services as first-class, that handles parallel operation with ease. with considerable static typing available to it (even if it CAN be executed dynamically). i.e. a SingleLanguageOperatingSystem that refuses, on principle, lower-level 'executable' formats... except as something the OS, itself, caches and produces to speed up loadtimes, and optimizes as needed. The OS would need to come with its own compiler, of course. Other languages that were to compile on such a system would need to compile to the OS's base language as a target instead of the native CPU or any special 'ELF' style formats.

The advantages of such an OS are that it can do the following: (a) run policies (e.g. regarding security, privacy, resource allocation) against the code it is compiling or executing (e.g. to ensure that privacy is never violated, to test whether it actually CAN be run as HardRealTime or SoftRealTime? if that is the demand, etc.) - allows for far greater consistency within the OS, (b) implement strong safety and security at the language and static-check level instead of the runtime/executable-interface level (this allows more optimization, too, since it solves the same problems as paged memory and kernel-space/user-space but without the higher costs when it comes to context switches and communicating between processes), (c) compile code based on the resources it possesses or is willing to provide to the user (e.g. 1000 CPUs distributed on a network vs. 1 CPU with low memory)... which is nice when bringing code in off the network, (d) provide tools as part of the OS to develop code that also handles dependencies, sandboxing, that sort of thing. Moves towards 'LanguageIsAnOs' - the OS is simply support of a language with first-class processes and services. Moves towards 'NoApplication' - there are just pieces of code that connect to services, values, procedures provided by or within other pieces of code. Moves towards NakedObject?, that being any piece of code containing identifiers to cells (which may be managed by a memory service/filesystem hybrid). The fact that it also pushes the OpenSource agenda is a good or bad thing depending on whether you're a commercial software developer that writes code as a product (rather than a service), though use of declared copyright and license statements is still doable. The guarantee that a compiler is available and directly accessible makes programs that write programs a far easier venture, along with such things as mobile code.

The main disadvantage is the cost to get started (there is a lot of inertia in modern OS's, which include all their applications... this would need to build everything from scratch, even text editors), and the general snubbing of those groups that obtain 'job security through obscurity' (though, depending on the language and whether it is graphical or textual, obfuscation may still be quite easy... even easier if they're willing to compile to a bytecode-interpreter + string of bytes, which itself might compile to native code due to partial evaluation; it'd still need to pass policy checks, of course, but it might skate by.) However, for those intent on OpenSource anyway, this is very much a good thing.

... Maybe I'll put this into the WishList? called NewOsFeatures.

Examples for SingleLanguageOperatingSystem:

InfernoOs seems to go into this direction strongly.
MozartProgrammingSystem
JavaLanguage or rather its VirtualMachine (discussed below)
DotNet or rather CommonLanguageInfrastructure/CommonLanguageRuntime. Even though this fixes you more or less to MS it is still a much higher abstraction layer.

What you want sounds like a VirtualMachine without an underlying OS and instead with attached caching of the JIT compiled code. I think there are Java VMs that bring with them their own OS for some platforms. The bytecode verifier ensures type safety and the permissions security. But I guess the language you mean would be somewhat more high level than the language corresponding to ByteCode.

That the environment provides essentially a VirtualMachine (a set of abstracted services) with JIT seems accurate in essence, though I don't believe Java and its bytecode would be highly suitable to the extra burdens it would need to shoulder - any language for a SingleLanguageOperatingSystem must, by nature, be readily capable of supporting the various OperatingSystem tasks handled today including security, user-management, data and persistence management, communications with the external world, hardware control, etc. Java's got rather poor support when it comes to efficient concurrency, communications, managing atomic operations, and doing any bit-twiddling for hardware manipulation. I think we'd need to include NewOsFeatures when designing the language and its architecture if we were to make this really work in practice.

Hmm. Interesting list. Java fails short on some points but not that many. Maybe some JSR are in order :-)

security - OK in Java I'd say
- Only if every user/programmer trusts every other user/programmer.
- That's due to the missing user-amnagement of course.
- Java almost (but not quite) supports CapabilityOrientedProgramming, but also has plenty of security issues the moment it comes to concurrency management and atomic operations and various 'global' objects like the 'System' object.
- I agree. But that is nothing that couldn't be fixed without trouble technically.
- I expect that assertion is easier said than done.
user-management - missing in Java
data and persistence management - standard File and DB abstractions are present in Java and NIO is very fast
- Java's VM doesn't actually provide file abstractions, only access. It's one problem it's always had... that its file management is not portable across systems. Anyhow, we'd be better off writing up a database service in Java or some sort of translucent persistence support.
communications with the external world - standard sockets
hardware control, bit-twiddling for hardware manipulation - missing, but this is not a disadvantage but rather a must because otherwise any security could be circumvented. Except maybe if you provide some hardware abstraction layer that allows to compile specially typed hardware access bytecode.
- Something like that might work. However, I'd aim for logically secure hardware access rather than worrying myself about circumvention thereof. Indirection slows things down, and layers of indirection is the typical mechanism for implementing layers of abstraction. But it might be acceptable in this case.
efficient concurrency - as fast as your local process/thread model in Java
- I actually meant to say efficient 'safe' concurrency. Threads don't really qualify in almost any language, excepting where the only communication between threads is by message passing.
- OK. I see you really aim at NewOsFeatures.
  - Indeed I do. But I don't believe that relevant on this point. I think it reasonable to assume 'safety' and 'security' and 'efficiency/optimization' are implicit to ALL other statements of desired features because the desire for them was made explicit earlier... and they just happen to be emergent properties that require consideration relative to every other feature.
- In addition, many of the objects in Java aren't designed for multi-user concurrency - esp. the 'System' object and its I/O support.
communications - don't know what this means above sockets
- Efficient communications doesn't use sockets except as a fallback. Something like transparent or translucent distribution would be a worthy feature of a language. Safe communications doesn't use shared mutable state except when really necessary. Java having real support for immutable value-objects would be a good boost to its usefulness in this venue.
- Ditto.
  - Ditto ditto.
managing atomic operations - Like IRQ locks? (->synchronized()) Or like atomic primitive operations? (->AtomicInteger?) Or like DB transactions? What else?
- Like concurrent AcId transactions. Atomic locks and 'synchronized' is just one really massive security hole due to potential for intentional deadlock. Something like optimistic STM would be a desirable feature here. SoftwareTransactionalMemory can be implemented with just CompareAndSwap? as a primitive atomic operation. (TransactionalActorModel should also work very well, and would better support a distributed OS.)
- Java at least has CompareAndSwap? so this could be built as a library. I agree that a thread model is subject to intentional deadlock which is hard to garbage collect.
- Clojure, a lisp-like language based on STM, was implemented on the Java runtime - so I know it can be done. But capability and realization regarding the way people build things in the language are entirely different concepts. At the moment, 'synchronized' and box monitors and such are how Java handles concurrency, and changing this wouldn't be a small task.

As a note, I consider Mozart/Oz (of MozartProgrammingSystem), ErlangLanguage, and EeLanguage to be much better starting points for this sort of system. EeLanguage has an implementation atop Java's bytecode.

Many of these have at least some of the above flaws. They are more well designed though.

Yeah, they are still flawed - I fully agree. But it'd be more accurate to say that they and their standard libraries and runtime are already more strongly aligned with the LanguagesAreOperatingSystems concept. They have less distance to go to reach the SingleLanguageOperatingSystem approach than does plain Java, which is why I consider them to be better starting points.

Sounds to me like someone is pining for a LispMachine or a SmalltalkMachine?, at least to some extent. Or am I missing the point?

To some extent, yes. And you could add ForthMachine? to the list. But summarizing the above as "pining for a LispMachine" ignores the relevant "pining" for such properties as design for secure multi-user operation, policies for resource management, and sharing of hardware - that is, the properties that make a SingleLanguageOperatingSystem more than a toy - to be a real part of the language such that everything can safely be in one runtime without those irritating hard memory barriers in common use today. Lisp, due to some of its features that offer it so much runtime flexibility, may fail to accomplish various properties necessary for a secure multi-user environment.

Related: MicrosoftSingularity? (http://research.microsoft.com/os/Singularity/)

CategoryOperatingSystem