External Service Versus Integration

Should semi-specialized services such as databases and GUI kits be generally separate services with language-specific adapters or be heavily integrated into application languages to better fit its specific idioms and conventions?

Let's add to that: mathematical expressions, numbers, collections, logical operators, standard console IO, string manipulation and regexps, and so on. These are all semi-specialized services to the same level as databases. One could easily use a separate service.

To some extent I agree and some could indeed be nicely treated as a separate service. But some tend to differ in that they largely don't require or build "complex structures". Services that have simple inputs, complex processing, and simple outputs are generally not at issue and could be internal or add-ons without being able to tell the difference or care. But, as soon as the need to manage and manipulate complex structures arises, then one generally has to pick between arrays, linked lists, objects, tables, etc. and decide if and how to integrate these with specific languages and their type systems (if they have one). It is difficult to make a "generic structure" that closely fits the style and feel of each different app language. -t

Complex structure and complex processing are one and the same. You've argued elsewhere (with me in ViolentAgreement - i.e. same conclusions, different reasoning) that DataAndCodeAreTheSameThing. To suggest that mathematical expressions are any less of a 'complex structure' seems a bit naive, or perhaps depends on an assumption as to exactly which is occurring: do you ship code to the data, or do you ship the data to the code? Whichever one is 'static' seems simpler, but (for many reasons) neither should 'in general' be static: ship the smaller to the larger for bandwidth performance, ship the public to the private (to protect privacy), ship the fast-changing to the slow-changing (to allow greater indexing or compilation-based optimizations), ship both for load-balancing, redundancy, reliability, and utilization. To suggest arbitrary distinctions between 'language' and 'separate service' based on yet another arbitrary distinction between 'code structure' and 'data structure' only succeeds in being twice arbitrary.

You also seem to be arguing under the impression that, say, dedicated databases will necessarily be 'better' at their job than would be a language with database-like-features. However, this impression simply fails to account for the AccidentalComplexity and DiscontinuitySpike that such a separation introduces. Management of data lifetimes and GarbageCollection (i.e. volatility), visibility and scope (for security and safety), and so on will be hindered. Difficulty with type-translation is well-known, but far from the only significant challenge such artificial division introduces. The discontinuity cost is felt by every user of the tool, whereas the cost of database-like-features is felt heavily by the language developer and high-performance implementions - but then results in a savings for each user, and quite feasibly integrates with foreign utilities (like ODBC) better than ever before due to a reduced gap between them.

One might argue that, supposedly, the reliability and performance features of the database (i.e. persistence, consistency for transactions, ability to 'export' data into a human-readable and machine-parseable format, LDAP-style distribution, caching, load balancing, security) will somehow be 'better' than those of the language. But there is nothing inherently true in that claim. If one can query for ad-hoc data, one can back-up any persistent database. If one can modify arbitrary data, one can thrash any persistent database. Failure to support persistence, concurrency, distribution, load-balancing, and security within a language only results in it being reinvented repeatedly in a buggy, slow, ad-hoc manner.

As far as needing to 'pick' between arrays, linked-lists, objects, tables, etc. - well, the language designer does need to make some choices or distinctions, I agree. But one needn't always pick and choose. Arrays and linked-lists, for example, can easily be one and the same under the hood. I use ropes (a k-arity tree of arrays of values) to represent linked-lists in my language, getting both the purity of linked-lists and (nearly) the performance of arrays (same algorithmic performance for access after accounting for virtual memory, much better performance for appends, inserts, deletes). I support 'sets' with uniqueness and permutative identity, and those also get an optimization: sets of records (aka 'relations') get free binary k-space partitioning to support high-performance lookups and inserts (effectively, they're indexed on every attribute). Similarly, natural numbers are logically represented in unary (Nat = s(Nat)|zero), but physically represented in binary.

Some choice needs to be made for the abstraction and the interface, but it only takes a few "intrinsic functions" and "de-facto optimizations" to massively reduce the need for LanguageIdiomClutter.

I'm curious as to how "language-specific adapter" is incompatible with "heavily integrated".

Does the above answer this to your satisfaction?

It did not. A "language-specific adapter", to my eyes, could easily involve a great deal of design to support composable abstractions when adapting to a common class of services. Further, I note that many such adaptations can easily become heavily integrated with the language (i.e. common to its standard libraries).

Working abbreviations used in this document:

Moved from PersistentLanguage

One thing we have to consider is standards.

Standards are languages.

It's not practical to re-invent things like DBs and GUI engines for each and every app language.

Disagree. It is entirely practical to re-invent APIs to DB and GUI engines (and other features like MultiCaster, filesystem access, control of actuators) for every significant application language, then provide behind-the-scenes adapters to outside systems that match the reinvented API.

Your approach is to force programmers to use some third-party language explicitly from the application language. This introduces explicit translation efforts, which are repeated by every programmer for every project for the lifetime of the language. Not only is the application language tied pointlessly to lowest-common-denominator legacy systems, but the productivity cost for very large projects or across different projects probably is worse by at least an order of magnitude.

This implies that there will be a separation between such components and the app language.

Even if your prior statement wasn't false, this separation isn't implied. Integration for any given app language is a much smaller effort than integration for "each and every app language".

Ideally everything is fully integrated into our favorite app engine and we don't have to worry about the outside world.

What a ridiculous statement. Ideally our favorite app engine is designed to effectively integrate with the outside world. What good is an application language that cannot do so?

In practice, that's not going to happen due to the combinatorial explosion of integration effort. In my observation, the popularity of SQL and HTML is largely because they can be used with just about any app language. -t

Most integration efforts are incremental.

A "combinatorial explosion of integration effort" will only exist if you did something horribly wrong, such as distinct ad-hoc adaptations to slightly different but largely similar systems, poorly composable designs using locks, using name-based references between project components (which require each component know how to map and use the name) instead of using objects, forcing programmers to explicitly translate types between systems at any place but the very edge of the system, overuse of opaque data structures like strings and binaries, or overly prolific use of HelpersInsteadOfWrappers - which forces programmers to know every detail of every system they integrate, and which also compose poorly because every part of the program needs to know how to use the helper instead of a more generic wrapper.

As an aside, your comment on SQL and HTML is irrational, illogical, and poorly reasoned. It's a bit like saying: "The reason TopMind is unpopular is because he's human" without accounting for the fact that not all humans are unpopular. Unless your 'observation' could say why all other PowerOfPlainText languages aren't, 'largely', equally popular with SQL and HTML, it's sunk.

It's difficult to "prove" population preference reasons unless you dissect thousands of neurons directly. We can only observe patterns and imperfect clues. Thus, there is no double-blind certified study for either side, so stop your whining and name-calling. That being said, PowerOfPlainText-based protocols have been more successful than the alternatives. HTML and SQL have proven popular (despite their flaws) and there are few if any AL-competitors that are as widespread. You are welcome to name some. -t

We can do more than observe patterns. We can apply logic, which you failed to do. Your "side" seems to consist of making a few observations then pulling some bullshit out of your ass to explain it. Intelligent and reasonable people systematically process and refine your observations so they at least reasonably match your observations and imperfect clues. If you want people to stop calling you on your fallacy, then shape up or ship out.

As far as this "either side" business goes, that's another objective point against you. Pointing out fallacies in an argument for position X does not imply arguing for position 'not X' (or: ~argumentFor(X) does not imply argumentFor(~X)). There is no "either side" involved outside your own mind. Any reasonable opinion or explanation you have about the popularity of HTML or SQL must account for the lack of popularity of its competition. Some of its competition has been equally plain-text. Requesting that I name some successful alternatives has no logical bearing on this discussion whatsoever.

I'm not saying popularity is the only metric to use or attempt to use. But it is one we can explore in isolation as one of many useful factors. So far it looks like there are more successful plain-text-based KS's than non-plain-text-based KS's. Sure, there are also plain-text failures. This indeed does seem to imply many plain-text-based one's can and do fail and that being plain-text KS by itself is not a guarantee of success.That I don't dispute. But almost all the successful cross-platform/cross-app-language KS's are based on plain-text. Successful "binary" KS are few and far-between. Taken by itself, this suggests that text-based KS's have more potential to be successful than non-text one's.

{What about CORBA? Isn't that arguably a popular and successful cross-platform, multi-language, API-based (except for the IDL) KS construction set?}

It's fizzling despite lots of initial vendor support. Plus, it does not work well with non-OOP languages. I've never met anybody using it in at least 3 large organizations I've worked at, contracted at, or visited in the last 7 or so years. WebServices are taking over its niche.

{Indeed, WebServices are taking over. Curiously, these use a text-based protocol (SOAP) but almost invariably employ API-based interaction.}

I don't see much SOAP, but anyhow, aren't those API's language-specific?

The APIs being language-specific was part of the point being made.

Are they heavy or "light"? I am not against light adapters, in fact expect them. -t

{What do you mean by "heavy" or "light"? Please be sure your distinguishers are unambiguous and measurable/identifiable.}

I am so feeling the love between you guys! Tell you what: With my super-secret super-powers, to whomever of you first coughs up something useful (i.e., some working code you've written, which both reflects and exemplifies your views), I shall grant world-domination. Sound fair?

Promise? ;)

First we have to get our premise strait before our promise strait. -t

See Also: GodLanguage

CategoryInfoPackaging

EditText of this page (last edited December 16, 2009) or FindPage with title or text search