Persistent Language

In the 1980's some research was done towards persistent languages. It was a good idea and I am still not sure why it didn't pick up.

Recently PersistenceEngines e.g. ThePrevayler for Java constitute a similar approach.

A great divide in computing that remains to this day is between programming languages and databases. Persistent languages eliminate the divide by putting the database inside the language.

Database systems tried to eliminate it by putting languages inside the database e.g. StoredProcedures.

Personally I believe in this second approach and I plan to realise it in my Mneson system.

--MariusAmadoAlves

What exactly is a "persistent language"? How does it differ from CollectionOrientedProgramming and ProgrammingWithoutRamDiskDichotomy?--AnonymousCoward

And PrevalenceLayer? No difference in essence I think. Persistent languages were a more pure approach. They tried to extend minimally the programming languages to achieve the essence. Prevalence layers are more complicated.--MariusAmadoAlves

RE: What exactly is a "persistent language"?

"Persistent language" implies TransparentPersistence for any state within an execution of the language (objects and data are recoverable), but also implies more than that, because prior to disruption any program is more than a static collection of objects and data. For example, active threads or continuations must also be restored and scheduled so that processing can pick up where it left off.

"Persistent language" could reasonably apply to any programming language where:

language semantics that implicitly allow most, ideally all, programs written in the language to recover after disruption.
- Recovery semantics are implicit to whatever degree they are achieved without programmer intervention. Programmer intervention would include needing to explicitly weave API calls, context, containers, callbacks, overload serialization methods, etc. into the source-code.
recovery of a 'program' that has been disrupted includes:
- recovering data
- recovering objects and relationships between them
- restoring communications including subscriptions, publish-side operations, broadcasting to retrieve waiting messages from other programs, pushing outgoing messages, and even raising the GUI again (ideally with the same window-positions and z-ordering, automatically re-establishing video input, etc. insofar as the environment allows).
- restoring continuations, threads, and behaviors that were active at the time of disruption
Said recovery is robust, reliable, safe, secure, and high-performance.
- "Robust" means that it isn't fragile or likely to break due to variations in environment, be they trivial or otherwise (i.e. changes in DLL versions or locations, change in keyboard or monitor, etc.).
- "Reliable" means that recovery almost never fails or has problems.
- "Safe" means that properties of the program are well-defined, predictable, and consistent, even in the presence of problems. The program shouldn't hang, crash, or thrash. Any exceptions must be something programmers can anticipate. Operations can't execute twice simply because the first time wasn't logged - not unless the operation is idempotent. The state of the program must be 'consistent' (for some acceptable definition of the word) with the state immediately prior to disruption.
- "Secure" only means that the recovery process doesn't compromise whatever level of security is normally provided for programs written within the language - i.e. 'does no harm'. If recovery was a significant security weak-point, that would be a problem. OTOH, if the whole language sucks for security, then recovery is not a significant weak-point.
- "High-performance" means that the memory and CPU cost for both recovery and for maintaining the persistence data is kept to a minimum, an acceptably small fraction of the CPU and IO resources used. If the language is intended for use with scalable programs (upwards or downwards scalability), then the maintenance and recovery costs must also be scalable. Maintenance costs should be incremental with the activity of the program, such that a very large but mostly inactive program has a cost proportional to its activity rather than its size. Recovery should, for long-running programs, be much faster than restarting it or running all transactions from the beginning; snapshots with a relatively small bit of roll-forward or roll-back is more acceptable.
the 'standard' implementation (be it a De-Facto standard or otherwise) supports persistence implicitly and as the default option, such that applications tend to pick up wherever they left off unless explicitly terminated and restarted. This is probably the most telling feature of 'persistent languages', since it's impractical to call 'persistent' a language that nobody uses in a persistent manner, even if it were theoretically doable.

"Disruption" would typically include loss of power or loss of communications. For a distributed programming language, one might extend that to temporary loss of nodes (CPU cores and data segments). Disruption might be characterized as predicted vs. unpredicted; ideally, a persistent language could handle arbitrary unpredicted disruption, yet also take advantage of 'predicted' disruptions (such as voluntary shutdown of a laptop, or a notice that power is low) when said predictions are available in order to improve performance.

Realistically, a persistent language can only handle certain unpredicted causes for disruption, and with some limits to integrity that may require a trade-off between performance and robustness. Thus, "persistent language" is by no means a binary feature of the language.

Also, persistent does not imply 'disruption tolerant'. That is, there is no implied GracefulDegradation. The program is free to stop entirely while disrupted, then continue again when the cause of disruption is alleviated.

Bonus features: not implied by PersistentLanguages, but enhancing them

transactional memory (related: InfiniteAmountOfTransactionalMemory, SoftwareTransactionalMemory). This greatly helps with achieving 'well-defined state' (AtomicConsistentIsolatedDurable) especially when dealing with unanticipated disruption.
- Perhaps counter-intuitively, transactions can enhance performance above non-transactional persistent memory (even in the absence of concurrency) because transactions form useful 'volatility' barriers. That is: restoring to a 'well-defined state' may very easily be done by simply aborting transactions that weren't committed or in a prepared-to-commit state, whereas non-transactional persistent memory may need to continuously stream data to disk in order to ensure the persistent state is consistent all the time.
- Additionally, transactions can serve as a very effective concurrency management tool capable of fine-grained parallelism for performance, safety, and composability at the same time (cf. locks usually force you to pick just one of those three properties). Transactions have better disruption properties than other synchronization primitives (cf. PersistentLanguages that use locking must have well-defined properties for what happens when disrupted while holding a lock). Use of transactions does introduce some limitations on other synchronization primitives (both joins in the JoinCalculus and explicit serialization or message-queues are difficult to safely reconcile with transactions)
ResumableException support explicit recovery from failure states wherever the language is incapable of achieving it implicitly.
FirstClass support for "subscriptions" or "connections" or "event source objects" of some sort. The FirstClass support ensures the PersistenceEngine can eagerly restore just the objects that must be restored eagerly (event sources) and lazily restore all other objects (i.e. when they receive a message), which offers recovery-speed advantages.
Object reconstruction via some sort of AbstractConstructor/AbstractFactory for objects and operations described in persistent memory can be used to make 'persistent' programs more modular and extensible over their lifetimes, in particular allowing plugins, dynamic upgrades of plugins (which can be implemented by gracefully 'disrupting' some sub-graph of objects then reconstructing them in the new plugin), and even some PolicyInjection (when you have competing plugins for the same task). This is as opposed to simply serializing objects and requiring recovery to a particular version of particular classes of objects based on simple serialization of data, which is likely to be quite fragile and have very poor debugging properties (i.e. ability to update code and have it reflected in the live, persistent environment).
Distribution of objects and operations - enhances PersistentLanguages by supporting both reliability-through-redundancy and performance-through-load-balancing, in addition to allowing processes to be automatically moved closer to the data - which offers both performance and tolerance against communications disruption. Allows one to use a full server-farm or 'cloud', and all the storage and processing capacity associated with one, instead of being bound th the performance and storage of a single box. (This feature, in turn, is aided by ObjectCapabilityModel.)
Language recognition of "private" or "secret" data and objects (i.e. password fields, military dossiers, object capabilities) that must be protected against certain theft cases (e.g. if someone gets physical access to the drive, it must be infeasible for them to access the data without also possessing the right smart-card and password). Additionally, recognition of private or secret data can limit distribution of it to unauthorized devices (perhaps using a DigitalRightsManagement technology), especially if 'privacy' or 'secrecy' is defined in terms of certificates required to host or see the data.
Explicit volatility: an inversion on the traditional persistence metaphor is "implicit persistence, explicit volatility". Volatile connections, operations, and objects might be described as an annotation. Volatility might be used to improve performance where reliability under disruption isn't as critical as performance.
- Note: Transactions reduce need for explicit volatility for performance by providing well-defined points from which programs written in the language are expected to continue processing.
- Note: If one supports 'liveness dependencies' (described with 'consistency', below) then volatility may be transitively "contagious" (if A is volatile, and liveness of B depends on liveness of A, then B is volatile) from the very 'edge' of the programming language. This allows one to integrate, say, 'volatile' applications with the language, where application-objects might be specific to the application-instance (and thus evanesce when the application is shut down be it gracefully or otherwise). This sort of contagious, implicit volatility within an otherwise implicitly persistent language forms a fine-grained and modular basis for "TransparentPersistence" where the persistence decisions are invisible to the programmers, but yet the programmers have a powerful, declarative influence on the decisions. (Note that application-specific objects may also represent as persistent and upgradeable via AbstractFactory or AbstractConstructor provided by the application, same as plugins. Whether each application-instance is considered 'unique' versus a 'continuation of the previous run' would generally be up to the application developers.)
- Between those notes, one could get a great many performance advantages of explicit volatility without actually making it explicit. But programmers might still benefit from explicitly annotating certain operations (i.e. sending messages, performing promises) as 'unreliable' or with a loose 'expiration' timeout, for example, to tell the runtime whether the preference for a given operation is reliable recovery after disruption vs. performance (for some operations, failure is more acceptable, and usually the language can't distinguish those without programmer help).
DisruptionTolerance? (GracefulDegradation and Resilience in face of disruption): a "persistent language" does not imply that useful processing can continue while disrupted, only that recovery to some consistent state occurs after the cause of disruption is alleviated. Above "PersistentLanguages" are what one might call "SurvivableLanguages?". These languages must provide implicit (and possibly some explicit) support for such things as fallback data sources (perhaps selected based on policy or quality), local-caches for read-mostly data, etc. in order to achieve useful (albeit degraded or reduced) "liveness" in the face of disruption. Further, and just as importantly, the language must implicitly try to recover full service in a timely manner after the cause of disruption is alleviated (i.e. if a node is powered back on, it will automatically restore communications and disconnect the fallback data-sources in a "timely manner" for whatever definition of 'timely' is appropriate for applications written in the language). The design for disruption tolerance doesn't require much on top of a design for both distribution and persistence, but support for fallbacks and caching may need to be improved, and it helps if the role of mutable state is reduced in common libraries and such (since cache-semantics and extensive use of mutable state are somewhat contrary).
Language definition for "consistency": a typical data-manipulation language for the RelationalModel will define 'consistency' in terms of properties within a table and for relationships between them (one->many, one->one, foreign key, etc.). This allows cascading deletes, the ability to reject construction of inconsistent data in the first place, and the ability to restore data to a "consistent" state or flag inconsistencies for an administrator. I posit that the same advantages can, and probably should be achieved for all PersistentLanguages, though the exact nature of the dependencies may vary. One might describe dependencies for PersistentLanguages in terms of: state-invariants within an object or configuration, 'liveness' dependencies, and so on. These could be used for restoration to consistent state (e.g. regeneration and redistribution of live dependencies after a node failure) and for cascading destruction or disruption when a subprogram's "liveness" has become inconsistent. (Cascading destruction and disruption, in turn, allows other programs to more readily detect failure and recover explicitly from it, as opposed to sending commands and queries that succeed half-way then fail.) Support for liveness graphs and consistency basically allows programmers to choose LimpVersusDie on a subprogram scale. This, in turn, is fantastic for GracefulDegradation.

RE: How does it differ from CollectionOrientedProgramming and ProgrammingWithoutRamDiskDichotomy?

PersistentLanguage in no way implies support for CollectionOrientedProgramming, and CollectionOrientedProgramming in no way implies PersistentLanguage. The concepts are orthogonal. They complement one another very well for database applications, though.

PersistentLanguage also does not imply ProgrammingWithoutRamDiskDichotomy. That is, the "persistence" state might still be distinct from the "active" state (or RAM state) of the objects in question. Programmers who add extensions or plugins to the runtime itself might be quite aware of a RAM-disk dichotomy, needing to 'regenerate' certain RAM-only resources for communications, open files, and so on. The modular use of plugins and AbstractFactory would tend to imply a dichotomy between what-is-stored and how-it's-interpreted, which naturally becomes a RAM-disk dichotomy.

But PersistentLanguage does imply TransparentPersistence; that is, a programmer within the language needn't take special efforts to bring objects into 'active' state or to push them back to inactive states on disk. That can all occur behind the scenes by the PersistenceEngine of the language runtime.

But when you have that ability, then you have to start worrying about issues that database handlers have to worry about, such as where the one-and-true "official" copy of a given fact is, what happens when you "share" and/or change it simultaneously, what if other apps/systems want access to that same info, etc. The more valuable a given fact is, the more it needs to be accessed by different concerns. Volatile (traditional) variables allows one to sweep most of those concerns under the rug. You don't get power without corresponding responsibility (unless you like chaos). I invoke GreencoddsTenthRuleOfProgramming. -t

RE: "that" ability - which one? Do you refer to TransparentPersistence, or the possibility that the language-runtime distinguishes between RAM and disk?

That if you start "saving" your variables beyond a given action or task, then new practical issues and decisions arise that weren't there or weren't needed before.
PersistentLanguage doesn't imply saving your variables "beyond a given action or task". The scope of the variables within each program stays exactly same. The tasks are also the same, except for being persistent. That is, the task survives a power-cycle, but its task-local variables still don't survive past completion of the task.

RE: you have to start worrying... - Of whom does the word "you" speak? Do you refer to the programmers implementing/extending the language, or to the programmers using it? I would imagine the concerns you named are implementation issues for the language.

Mostly app developers, but it would complicate language design also.
App-developers for the most part have an easier job in a persistent language, since they don't need to push stuff to file as often for plain-old persistence. (They'd still need to load/save to files for import/export, of course.) But, in light of your earlier misconception, I can see why you'd think it'd hurt app-developers.
Please clarify. Source code doesn't normally disappear.
Huh? Where did I say "source code normally disappears"? (Are you making assumptions about the IntegratedDevelopmentEnvironment and coding cycle?)

RE: You don't get power without corresponding responsibility - what a ridiculous notion, TopMind. It's easy to obtain or grant power without responsibility. Give a gun to a child. Use a persistent language that someone else implements and maintains. The effort to produce the device that contains power is generally much higher than the effort to transfer that device to someone. If you mean to say that introducing persistence, concurrency, etc. will raise the bar for implementation, or introduce language-design concerns that a 'weaker' language might not possess, that's a reasonable argument. But that burden doesn't fall nearly so heavily on the programmer using the language.

With regard to the child-gun analogy, it is well covered via my "chaos" disclaimer. As far as how it "burdens" the app developer, I suppose we'd have to study specific scenarios to explore this in more detail.

RE: "share" and/or change [a given fact] simultaneously - You assume both unrestricted concurrency and shared mutable state. Persistent languages don't need to include either of those features, much less both of them at the same time. Restrict either feature in various ways and you can readily avoid the race-conditions and synchronization-challenges of determining what the 'official' value for state might be at any given moment. Even without shared mutable state, "persistent language" is still applicable, as continuations and activities have implicit state that must be recovered after disruption.

If your language does need both unrestricted concurrency and shared mutable state, then there are techniques for taming it. For example, transactions, which are useful for persistence (as they allow partial volatility in the implementation), may readily be extended to support isolation.

RE: what if other apps/systems want access to that same info - If what you have is a "persistent program" written in a persistent language, these other apps/systems would certainly need to talk to the persistent program through whichever open communications channels it offers access. And that is exactly how it should be.

You mean like a query language and/or ODBC?
Not in particular, no. It could include RPC, HTTP, whatever language they wish to talk. Persistent programs should look, to outside programs, exactly like volatile programs, excepting that they'll automatically recover and continue processing any commands or queries after power cycle or other disruptions. This makes it easier since outside programs will know that once a command is acknowledged, it will be completed barring some unexpected disruption or explicit cancellation.
It sounds like you are thinking of something like bubble memory, which keeps its state even if power is lost. That's a hardware feature, not a language feature (but could be emulated).
Sigh. I already distinguished between ProgrammingWithoutRamDiskDichotomy vs. PersistentLanguage, above. Merely keeping memory of what you were doing around isn't worth much unless you can recover and continue, which will often require regeneration protocols for sensors and actuators, graphics, GUI, communications, and so on. Bubble memory by itself cannot do that. It might help a bit, but not as much as you're imagining.
Perhaps there's a perception mismatch here. Once RAM (real or virtual) is no longer volatile, I expect that many people will start to program differently and start using the language as a kind of database. If you view it merely as a reliability feature that makes existing apps (or existing app style) more reliable, we are talking different things.
Persistence is fundamentally a reliability feature. I do agree that people would take advantage of the more reliable language to avoid certain questionable uses of databases (e.g. database used as a hacked-in PersistenceLayer), and some might go further than that and implement databases in application (so it would be useful if persistent languages also have a database API). But persistence does not make that a language is suitable for data queries or management, though it may make the language suitable for implementing database management systems.
And, why would it be a property of a language instead of say a RAM chip? We limit this investigation to just languages? Virtual everlasting RAM would be nice period, language or not.
Persistence in general is a property of programs or applications or data. But PersistentLanguage is a property of languages, of how well they support development of persistent programs, and the degree to which this support is implicit or automatic. Persistent RAM is useful (indeed, it's part of the GodRamIllusion), but making use of it to support persistent programs must still be done either explicitly (startup the volatile program, scan RAM in order to explicitly recover to a consistent state, re-establish communication, etc.) or implicitly (by use of PersistentLanguages). You seem to be under the naive impression that persistent RAM would, by itself, be enough to make window managers, browsers, IRC programs, etc. 'persistent'. Do you have any clue how much effort it takes to make 'Hibernate' functions of laptops work, under-the-hood? No, you probably do not. Making that work implicit is what marks PersistentLanguages.

As to your "GreencoddsTenthRuleOfProgramming" - it is not unreasonable to suggest that a "persistent language" is slightly more database-like than a volatile one, and that a "concurrent, persistent language" is even more so, and that a "concurrent, persistent language with shared mutable state" is just a few steps away from being a (not necessarily relational) database. I'm not sure what you mean by 'invoking' it, but to suggest GreencoddsTenthRuleOfProgramming applies (in some capacity) to the circumstances of this PersistentLanguage page is reasonable.

RE: "Persistent languages eliminate the divide by putting the database inside the language."

This isn't true. A PersistenceLayer is not a database, and not all databases are persistent (http://en.wikipedia.org/wiki/In-memory_database). PersistentLanguages are not necessarily suitable, without some API, for storing or querying data. A persistent ObjectCapabilityLanguage would almost certainly forbid ad-hoc queries unless they occurred across a special capability provided by the runtime (i.e. for debugging).

That said, it would certainly be easier to implement a persistent database (with its own data-manipulation language and such) within a persistent language. This is especially true if the language also has good support for distribution, transactions, and disruption tolerance - features that would achieve some of the more complicated database aspects of load-balancing, mirroring, and concurrency.

The performance achievable within the database would be limited by the language runtime if achieved this way, but the theoretical performance limit is better because one can eliminate much serialization and translation overhead. The better theoretical limit for performance only extends to use of the database within the persistent language, of course, but that may be sufficient for most applications, especially given the ability to push more of the program to the data.

RE: transactions, which are useful for persistence (as they allow partial volatility in the implementation), may readily be extended to support isolation

Again, adding transactions is a case of GreencoddsTenthRuleOfProgramming.

Indeed, it is. That wasn't a useful statement to make, though, since it repeats a point I already made.

I'd rather let languages focus on being good languages and databases focus on being good databases rather than a language try to be a half-ass database, risking scaling and sharing problems.

Now you contradict yourself. GreencoddsTenthRuleOfProgramming essentially suggests that being a good language requires being (with respect to some concerns) a good database. That said, it doesn't mean the language IS a database. Sharing concerns like robustness/reliability/concurrency/performance/scalability/etc. doesn't make a system a database. It's the data-manipulation language that makes a database. It'd be perfectly reasonable to write a database in a persistent language, leveraging the free support for persistence/transactions/garbage-collection/etc. and giving you time to focus on a data-manipulation language, dataflow to track how views change over time, support for querying streaming data in real-time, and other neat stuff.

I'm not sure how you conclude that about Greencodds.

I conclude it by use of logic, TopMind. But perhaps your axioms are different.

Axiom A: A good language has everything it needs to avoid forcing programmers to repeatedly reinvent things the hard way.
Axiom B: The features of a good language should be well-integrated, first-class, and symmetric, as opposed to being stapled on and requiring explicit translation.

Do you believe good languages should force programmers to reinvent things the hard way and have features that are hacked on and require lots of work to use? No? Well, then you agree with the above axioms.

Discussion regarding need to integrate with existing 'standards' moved to ExternalServiceVersusIntegration.

GreencoddsTenthRuleOfProgramming states: Every sufficiently complex application/language/tool will either have to use a database or reinvent one the hard way.

Note 1: It can further be noted from your comments in this page that 'Greencodds applies' that by "database", you really mean database-like-features such as persistence or transactions, as opposed to any strict or formal concept like DatabaseIsRepresenterOfFacts. Since you coined 'Greencodds', this can be taken as canon. (In my eyes, persistence, concurrency, and transactions are completely orthogonal to databases, but are related to 'management systems' - the 'MS' in 'DBMS'. But since you coined GreencoddsTenthRuleOfProgramming, I'll allow your opinion rule how the Greencodds is interpreted.)

See "Typical Services Provided by Database Management Systems" under DatabaseDefinition. The more of these features it has, the more database-ee it is.
You insist on conflating 'database' with 'DBMS'. That is a fallacy, on your part. I won't buy into it.

(Conclusion 1) Therefore, a good language will include a database-like-features (from Greencodds, Note 1, and Axiom A).

(Conclusion 2) Therefore, a good language will have tightly integrated, symmetric, and first-class database-like-features (from Conclusion 1 and Axiom B).

Thus, if you agree with Axiom A and B and Greencodds, you are logically inconsistent if you further say that a good language shouldn't concern itself with being a good database. After all, your GreencoddsTenthRuleOfProgramming says (under the above logic) that being a good language and a good database are not separable concerns.

[... risking scaling and sharing problems.] (Or perhaps have a partial implementation of a database and/or query system using standards so that we can use mostly the same code when we do need to migrate to a full-on DB.)

One doesn't need ad-hoc queries and data-manipulation to implement a programming language. When debugging or doing LiveProgramming, I can see weaving a database operations into code to help 'view' or 'tweak' its activities, but other than that it seems to be a violation of PrincipleOfLeastPower, an enormous back-orifice for security violations, and a rather inefficient AbstractionInversion.

Perhaps I should have said lite-duty database-like API's for things that are in-between arrays/lists and dedicated databases. At times it's nice to have transient work-tables whose scope is tied to the programming language's units, yet use the same query language(s) as dedicated DBs so that we can upgrade when needed with little or no rewrite.

Sure. Good support for relations and queries - as found in LogicProgramming - are high on my list of desirable features. And support for FirstClass relations or databases, such that they can be composed logically and maintained separately (as per 'rulebooks' in the InformLanguage) is also nice.

As far as using "the same query languages as dedicated DB's", I think that's terribly short-sighted and stupid. The query language should be well integrated - types and all - with the rest of the language.

But all that has nothing to do with PersistentLanguages, or even with persistence. The features of 'persistence' and 'tables/relations/db-structure-of-choice' are totally orthogonal.

Any useful data tends to be shared by multiple apps and outlives specific app languages. If the relationship is very tight and will stay very tight with a given app and/or app language, then tight integration makes sense. The problem is that the "natural" partitioning lines of "app" and "data" tend to be different or drift apart over time. It's somewhat analogous to political boundaries versus postal boundaries in the geographical sense. Perhaps they started out one and the same, or similar, but have drifted apart over time. Based on past observation, app languages tend to fall in and out of favor. But the data is often necessary for the organization, and must remain. And different tools, such as report writers, also are used with existing data. Things like CSV, ODBC, and SQL were designed to facilitate relatively easy sharing and standardization of data access and extraction among systems and tools. (Perhaps merge/link into SharingDataIsImportant. I forgot it existed until after.) -t

Your complaints are not specific to PersistentLanguage, and thus seem inconsistent. Why must using a language with persistence features for storage hurt sharing of stored data more than does Oracle implementing its own private language and API for accessing a flat-file or HDD directly? You complain about the lifetimes for languages, but why aren't you complaining equally about the possibility of the language in which Oracle DBMS was written falling out of favor? You shouldn't be pointing fingers at PersistentLanguage for problems that are independent of the persistence feature.

Generally when one needs "persistence", they will also likely eventually need "Typical Services Provided by Database Management Systems" as found in DatabaseDefinition. Needing *only* persistence is rather unlikely for anything beyond say app-specific stuff that one might find in an appFoo.INI file.

A PersistentLanguage doesn't need to be *only* persistent (what would that mean, anyway?). And those other features common to enterprise DBMS systems (good support for concurrency, support for automatic distribution and load-balancing, support for automated failover redundancy and self-healing, and a built-in security model) aren't 'DBMS' features. They are features any sufficiently complex system will desire, even if it isn't a DBMS, and thus could be considered KeyLanguageFeatures for a GeneralPurposeProgrammingLanguage. If I were going to implement any shared service - be it a DBMS, a WebServer, a WikiWiki, a FileSystem, an IntegratedDevelopmentEnvironment, an OperatingSystem, etc. - I certainly would prefer to implement it in a language with all these useful features.

That said, even persistence by itself (minus those other CrossCuttingConcerns) is powerful. Persistence is the entire basis of hibernate modes for computers (especially laptops), after all. It would be perfectly reasonable to want even higher quality persistence than is typically achieved by hibernate - i.e. automatically re-establish network connections (e.g. audio and video streaming, restoring downloads and uploads, ComplexEventProcessing, MultiCaster, PipesAndFilters) along with GUIs (dialogs, windows, etc.). Even better: good support for persistence allows one to never 'close' any applications... that is, all applications are open all the time, but they're simply not in main memory all the time. (Virtual memory only is half a solution, acking network layer persistence.) This allows considerable more scalability at the OS layer; persistence with publish/subscribe can support millions of active applications and services instead of a few hundred max, and also support much faster startup times.

You are drifting into your GodLanguage desire again. Most of that would need more than mere persistence to do right. For example, concurrency management will need to play a role, and also tools to monitor all that state, like say maybe a.....query language? You'd have to at least partially reinvent a database engine of some kind. You are confirming GreencoddsTenthRuleOfProgramming. -t
What do you mean by 'concurrency management'? Are you saying persistent language couldn't be achieved atop threads and semaphores or even multi-OS-process designs? (I certainly see no reason why not.) And why would I suddenly need special tools to monitor "all this state" significantly more so than I would require prior to supporting persistence features?

Configuration files are not used for just persistence (they also serve as a sort of PolicyInjection/DependencyInjection and support AbstractFactory at the OperatingSystem layer) but could be replaced given better support for sharing persistent resources among different services.

And using Oracle-specific features such as PL/SQL is indeed a product-lockin risk that one has to be careful about. But at least the data itself is still sharable through semi-standard SQL.

You totally missed my point. Oracle DBMS is a service that uses non-standard storage (flat-files and block storage were mentioned explicitly) yet provides a standard interface/API. Services in any PersistentLanguage can do exactly the same. Your complaining that a PersistentLanguage doesn't directly support sharing in a standard way is very much analogous to complaining that Oracle doesn't directly support sharing of the flat-files in which it stores data. That's a silly and invalid complaint, and completely separate from the issue that Oracle's SQL API doesn't strictly adhere to every standard.

Further, Oracle scales better than most app languages, and thus one can be fairly confident that they won't run into an unfixable scalability wall. They just have to open their wallet wider than before. -t

Call me pessimistic, but rather than seeing this as a strength of Oracle, I see it as a weakness of most app languages.

I'd rather see an app language that is designed to work well with existing RDBMS rather than replace them. It could also provide it's own local SQL engine for local processing (or perhaps borrow say SqLite's engine). That way if you need to migrate or scale up that portion to a server-based RDB, one mostly just changes the data source configuration and minor SQL tweaks rather than re-code everything. (SQL indeed has some annoyances, but being a de-facto standard still overpowers those warts.) Related: EmbraceSql. -t

[It's called Java, C#, Python, Ruby, etc. The unnatural divide between application and database is unlikely to endure for long, however, once more sophisticated languages make it possible to define applications as abstractions, with the physical location of database definitions, presentation functionality, business logic, etc., being a matter for automated optimisation rather than programmer-defined directives. Programmers should no more have to worry about these low-level trivialities in the future than concern themselves with the location of file nodes on the average disk today. When that is achieved, SQL will, at best, be reduced to the role of a protocol for communicating with legacy DBMSes.]

Well, paint me skeptical. The base idiom and syntax differences between those languages and SQL are too wide and complex to hide behind dot-path API's. The tight marriage between app language and query language of ExBase taught me of possibilities that are hard to forget. But, I imagine the future will be more like MS-Access-Done-Right rather than be language/code-intensive. -t

[I've no idea what you mean by "[t]he base idioms and syntax differences between those languages and SQL is too wide and complex to hide behind dot-path API's". I don't even know what languages "those languages" refers to.]

[Whether language is presented graphically or textually is of negligible consideration compared to language capability. Graphical language vs text-based language is like worrying about the colour of the paint on the wheels, when the significant consideration is whether to use a car or an airplane.]

[As for the ExBase, see TutorialDee for an example of application language and database integration done properly.]

Given a choice, I'd make a SMEQL-friendly (TopsQueryLanguage) app language if we are going to bypass SQL as the root query language. But for now we'll have to live with SQL.

[There are at least two TutorialDee implementations, a working one in the RelProject and a work-in-progress one from Ingres.]

I meant as far as established or accepted standards. TutorialDee is still a lab toy.

Does KayLanguage belong to this category, since it has KDB built in?

KDB provides something of a "persistent language" (ApiIsLanguage), but does require explicit storage (KDB is an in-memory, volatile database by default, though it makes persistence easy). KDB gets bonus points for supporting redundancy and distribution.

KayLanguage itself... I don't believe it qualifies. The language is designed for interactive programming, and its implementation is unlikely to 'continue' an action started by a user/programmer across, say, a laptop reset (i.e. picking up where it left off). That said, I've never used the language.

CategoryPersistence