Great Abstraction

What makes an abstraction a GreatAbstraction?

I give short explanations in terms of the Unix file system calls.

it serves the needs of the time but also of the future (30 years and growing is not bad)
it is easy to use by client code (no exceptions, simple return values, well understood, simple state model)
it hides implementation details to client code allowing for future changes of implementation without affecting clients (use cat for playing digital music)
it serves a multitude of different clients so well, that they are happy to use it (how many programs use open(), read(), write(), close()?)
it provides a StableInterface that allows many client software to be developed and deployed (the signature of these calls haven't changed since)
its implementation provides extensibility hooks (in traditional UNIX device drivers, concrete strategy classes are called by template methods of the surrounding kernel code; user-space hooks are provided by the ioctl system call.)
outside configurability (ThirdPartyBinding), i.e. a third party can configure a specific client to use a specific incarnation of the abstraction without requiring both to know about each other except for the interface shared (shell connecting programs and files, dynamically loaded device drivers of today)

An example of a GreatAbstraction which is not an API might be nice. File formats come to mind, but I don't think there are any qualifying ones. Perhaps a non-computing example would help? -- DanielKnapp

Pure mathematics is composed almost entirely of examples. (Say, group or topological space. Or even, though this is an informal abstraction only, that of space -- what is it that the ideas of "topological space", "metric space" and "vector space" have in common?)

Within the computing field, the notions of class, list, window, and relational database (picking a few examples from many) are great abstractions of this "informal" kind. Going from "informal" to "formal" generally means providing an API or something similar (a type system, or a set of classes or functions, or whatever). Programming languages at any level above assembler are abstractions. Some of them are great, but I shall avoid saying which ones.

Four dimensional space plus time is without a doubt the greatest abstraction ever devised. Humans learn this abstraction at the toddler stage and humanity first learned it about three thousand years ago. It's such a fantastic abstraction that even blind people can learn it just from their hearing and sense of touch. And without the abstraction of time, it becomes exceedingly difficult to remember the past or plan for the future, though obviously it's doable since humans managed to build civilizations before they invented modern 'extended linear time' as an abstraction.

I wish the first author above hadn't used the Unix filesystem API as an example of a great abstraction, because it's a piss poor abstraction. open() is one of the GrossDeficienciesOfUnix. Great abstractions in computer science include:

functions
- lambdas
expressions
variables
objects
recursion
- proper tail recursion
static scoping
ST-style hard returns
...

Exceptions are mentioned in the list above. Is their absence really a necessary property of an interface which is a GreatAbstraction? In C, any exception-like mechanism is ugly. In other languages (not naming names to avoid looking like a troll), it is not. Perhaps the issue with them is that they supposedly make it difficult to understand a program's behavior? Yet littering code with checks for out-of-band return values and other tests for error conditions obscures the algorithm it implements and is at least as non-ideal as a well-designed exception system. -- DanielKnapp

GreatAbstraction produces ClearEncapsulation.

Which is true when there's something to encapsulate. What does the concept of space encapsulate?

{Space encapsulates spatial identifiers and a means of partitioning them (potentially allowing for inside/outside, left/right, above/below, etc.) At the very minimum, reflexive identity will do (for the partitioning) with each 'point' (as identified) having only the position 'at this point' - and no ordering relative to other points. Thus, to have a space, ALL you need to have is a spatial identifier (for 'point') with a reflexive identity property (pick a value type! any type!). In the general sense, partitioning can be formed by arbitrary predicates over that value type - anything you can write as a predicate or query over a set of points that leaves some points inside of it and some points outside of it (or gives a distance function... e.g. a fuzzy number 0.0 to 1.0 on how 'inside' it is).}

I looked up "Space" in the Shorter Oxford Dictionary and its definitions take up an entire page. I guess that was what you were saying - that the concept of space cannot be encapsulated in that form of ClearEncapsulation. True. But when we need to access the contents of a current concept of space within a computer process, we have a clear enough encapsulation of "space" to define access. Like we can say Customer.Data = Get.Customer.Data(Customer.Reference), which abstracts away any physical implementation details, states clearly the intention and allows the Get.Customer.Data process to access the relevant database in any appropriate manner.

{Even 'space' has a GreatAbstraction. The GreatAbstraction for 'space' would be something common to all meanings of 'space' that is still distinctly 'space'-like. In this case, it is: a space is a system of identifiers for locations in the space, along with means to partition them. Once you have space, you get graphs for free (since you can draw edges between points in any space). Anyhow, for space in computer systems, ConceptOrientedProgramming is one of the few designs that fully formalize and embrace it into the model. However, some people have made noises about modifying computer spaces (especially the filesystem) to better support arbitrary partitions, rather than simply partitioning on the hierarchical folder names - better 'space' abstractions are certainly on the NewOsFeatures wishlist in the form of query-based ObjectBrowser support.}

Actually, I was wondering what the concept of space encapsulates in physical reality. Software abstractions tend to encapsulate something, some implementation of the abstraction. But in physics, what do the concepts of space+time or spacetime encapsulate? Or going back to software, PlanNine and Unix both have the concept of a namespace, and I'm not sure what makes it a clear encapsulation, although it's definitely a great abstraction.

{spacetime encapsulates identifiers for spatial coordinate systems (which are all relative, and generally involve distances and rotations), and partitions these (nearer/further for distances, positive/negative spin for rotations, inside/outside for volumetrics and spatial-temporal points, etc.).}

Saying that a concept encapsulates something runs into problems when it only abstracts part of that thing and not the totality of it. For example, physical space doesn't abstract away fields and field values at specific points. And namespaces don't abstract away the semantics of the objects in the space, just the syntax to access them. This raises some questions.

{Neither of those observations seem correct. Physical space isn't at all about the properties of the points within it, except insofar as those properties shape the partitioning functions of interest. Field and field values are attributed to points in space via observation, but are not part of space itself. Similarly, namespaces partition the names of things, that being their primary purpose (since names in one partition don't clash with names in another partition). Namespaces are only about semantics insofar as we readers attribute semantics to the names - yet again, an attributed property, not one intrinsic to the space or its identifiers.}

Going back to encapsulation in software, what if the reason some software abstraction encapsulates is the mere fact that it's an API or an object? Objects encapsulate and isn't it true that even bad abstractions rendered as objects nonetheless encapsulate? If that's the case then 'great abstraction => clear encapsulation' hinges on the meaning of 'clear' more than 'encapsulation'. And at this point, the term 'clear' is much too vague though it does provide a hint.

I think a great abstraction hsa something to do with the boundaries of the abstraction. That's the hint which clear encapsulation provides. Some desirable properties in abstractions are,

robust
persistent and long-lived
natural

So a great abstraction is one that will stand up to various use and abuse. It's also one that will persist in the minds of those who know it both because easy to grasp and recall, and because it's difficult to simplify or otherwise improve. And it's one that is easily understood, transparently fits the uses and needs it's put to, and will be reinvented again and again and again every time it disappears.

A GreatAbstraction shouldn't be described in terms of its persistence, but rather in its utility. Persistence and longevity are incidental properties of abstractions, created only by the social environment in which the abstraction proliferates as a meme - you don't need to let anyone else know about a GreatAbstraction for it to be great. Utility for abstractions is more measurable: a GreatAbstraction is readily applied to a wide array of different domains, lacks any properties one should consider 'accidental' or 'arbitrary' or 'unnecessary' (degree of abstraction is ultimately determined by the number of properties abstracted away; you can't abstract away everything or you end up with nothing, but for any property in the GreatAbstraction you ought be able to provide a reasonable explanation as to WHY it is there and WHY it is necessary), and is readily combined (without arbitrary restrictions) with other GreatAbstractions (since any abstraction, by nature, always has variables - those properties that have been abstracted away). As a consequence, any pre-implementation of a GreatAbstraction (with all its open variables ready for injection) should be a great addition to a language library, if not the core language. By observing this fact, one should note that GreatAbstraction should lead you straight to KeyLanguageFeatures

Unfortunately, GreatAbstractions often aren't easy to understand. A lot of people have difficulty with folds, monads, CategoryTheory (abstracting mathematical topologies), homomorphisms, coinductives, etc. The problem seems to be keeping the abstraction in one's head - the lack of details makes them a bit slippery, at least until one possesses considerable experience with their applications. As an example, I've seen some GreatAbstractions in the Boost and Fusion C++ template metaprogramming libraries (including field accessors, field iterators and visitors, expression templates, lambdas, class-independent interface capture, formatters, loop inverters (so you can iterate over a multi-layer for-loop), etc. ... and I can say for certain that many of them truly blow the minds of many traditional C++ programmers (i.e. the vast majority for whom template metaprogramming is weird compiler voodoo magic). That forces me to add addend the above paragraph: GreatAbstractions are excellent to already have in place and accepted by the infrastructure, because they'll fit any problem; but programmers won't be able to learn or use the GreatAbstraction if and until they bump into several problems that require it (i.e. after they've written several specializations, and are then getting ready to refactor). And writing specializations to handle stuff that your old programming language already did perfectly well is, very simply, both irritating and frustrating. Therefore, there had darn well better be some GreatSpecialization?s and GreatDefault?s readily available for day-to-day use; these will avoid frustration for the normal cases AND provide examples of the GreatAbstraction when comes time to use it.

As a final note, your wonderful GreatAbstractions can be a major source of frustration if the infrastructure isn't already there to support it. You can easily wrap anything from the environment or operating system in the GreatAbstraction, but you can't go the other direction. You CantAbstractMuchPastInterfaces; if your GreatAbstraction isn't supported outside the language, you'll be forced to write all sorts of irritating interface code that will only work for some specializations of the GreatAbstraction, just to let you communicate back outwards towards your OS and environment. But LanguageIsAnOs. GreatAbstractions for languages are also GreatAbstractions for OperatingSystems. There isn't any reason NewOsFeatures shouldn't include support for adding arbitrary new GreatAbstractions.

CategoryAbstraction