Net Kernel

"1060 NetKernel is a resource oriented microkernel and RESTful application server [...]."

Basic Idea: "Oh! Neat! I have a command line in my address bar!"

In other words, the NetKernel architecture is based on the assumption that you should access URI identified resources, rather than calling subroutines.

For best performance, resources should be stateless, IE: functional, in that their results depend only upon their (URI) parameters. Thus, the results are automatically cachable.

I find the idea intriguing, so I did a bit of reading up on this. It is true that they base operations around the concept that 'resources' can be procured functionally via combination with other resources. But resources can identify stateful concepts, such as an attendance list, ticket sales, or even just a mutable file; they can also be sinks for data and messages (SINK), can be created (NEW), and may be deleted (DELETE). Only 'representations', which are snapshots of a resource at the time a 'SOURCE' request was performed, are stateless and cached. Of course, resources that really are static are unlikely to invalidate the cache very often.

I find some of their examples unconvincing: Using caching to improve the performance of a bad recursive Fibonacci sequence implementation doesn't strike me as a compelling business case. -- JeffGrigg

It makes a fine didactic example of caching, but you're correct that it is a poor business case - NetKernel is not really designed for such fine-grained operations as performing steps of the Fibonacci calculations. The real advantage of keeping an explicit caching model is that code can be written to take advantage of it, much like requiring a TailCallOptimization allows people to take advantage of that. Ultimately, it simplifies other pieces of code because programmers don't feel the need to invent it themselves. In a sense, it really is one of the core services of NetKernel; consider: without such caching, how practical do you think computation by use of addresses would actually be? I know that I'd be looking at it and thinking: "does this thing really fetch xyz.txt, or perform this-really-expensive-query, on every operation? ugh!"

I see the possibility of such caching being very useful for business applications: I led an application team, recently, to do all business logic as if database access were free, and making the business logic easy was most important. Naturally, this lead quickly to some relatively inefficient queries -- which were easily addressed with intelligent caching at the data layer. It worked out quite well for us, in spite of having to write some (admittedly relatively simple) caching logic. It saved us a lot of time and effort on the business logic, and yet we still got reasonably good performance overall. -- JeffGrigg

I'm still not sure about its usefulness in common business applications -- "CRUD" applications that focus on enabling users to Create, Read, Update and Delete data in an essentially user-visible database schema. It seems that database transations, being passed as paramters on resource fetches, would disable NetKernel caching functionality across transactions: That is, multiple fetches of a State list within a transaction could be cached, but the same fetches in another transaction couldn't use the cached data because the "transaction number" parameter would be different.

But I haven't tried it out with the real software. -- JeffGrigg

I believe the caching is finer grained than that due to the recursive nature of operations: resources requiring resources. The 'active:some-resource+operand1@some-other-resource+operand2@yet-another-resource', for example, could (though not necessarily, since a resource could be a sink or an output rather than a fetch) result in both the operand resources being cached individually, along with anything they depend upon (since 'some-other-resource' could itself be an 'active:' or 'ffcpl:'). The system also tracks invalidation on a sub-resource level. Thus, depending on how one designed the URIs, one could probably ensure that most of operations across transactions could be cached. I wouldn't expect magic, but I believe that a savvy programmer would rapidly find ways to take advantage of the caching.

So, as I understand it (without getting the library and writing tests to determine its actual behavior... ;-) database transactions pass the URI boundary by being operands to all the resources that need to participate in the transaction. So in the business system example I alluded to above, NetKernel caching would be quite useful for caching within a transaction -- which is what our system, described above, needed. But NetKernel caching would seem not to be very useful for cross-transaction caching, unless one wrote one's "static lookup data" services to operate outside transactions, and to manage "transaction like semantics" heuristically -- with polling, for example. This could be useful for most applications, but it doesn't strike my fancy for correct transactional semantics. -- JeffGrigg

I'd imagine it really depends on how you go about writing up the transaction and where you put the transaction identifier. Suppose you had a transaction that performed a query on two tables (2 resources), joined them with some complex aggregate functions (creating a new resource), then wrote a few results to a third table, and did all this as a transaction. Now, it should be trivial to note that the results of the query on the first two tables, and the result of joining them with an aggregate function, would not be invalidated by the transaction itself - they might be invalidated by a later action, or by somebody else's action, but not by your transaction. As a consequence, you should be able to access those cached results across a variety of transactions, and any organization that makes doing so impractical could probably be dismissed as 'incorrect'.

The problem you're imagining is that the operand to access these results requires the transaction-identifier (i.e. to see the query-resource, you need to know the transaction-identifier that was part of the query). But a slightly different organization of the URIs could place the transaction-ID as something passed to a transaction manager, such that it isn't part of the URIs for the queries themselves. It would look a bit more like: 

  active:transaction-manager+tid@myTID.txt+retries@literal:3+operation@(update-function(join-function(query resource1, resource2),table3))
In this case, the identifier for the queries themselves - which would probably look an awful lot like a big, ugly 'select' phrase shoved into a URI - would still be good until invalidated, and would NOT have that annoying 'TID' attached to them that makes it impossible to (in practice) use them across transactions. If you needed and the transaction-manager would handle the task of ensuring the operations were performed atomically, and retries, timeouts, etc. (As a note, the use of a transaction-manager is generally required for potentially distributed transactions of any sort.)

EditText of this page (last edited July 9, 2010) or FindPage with title or text search