Policy Injection

PolicyInjection in general is when user-defined policies and priorities are injected into a decision-making process that broadly affects the behavior of a system.

To have PolicyInjection for a given system, the prerequisites are:

This page is about a more specific instance of PolicyInjection: in particular PolicyInjection above DependencyInjection.

DependencyInjection exists (in some minimal capacity) when a user can, via a general abstraction on the required capability or behavior, indirectly set behavior properties of another system. I.e. one has a generic need for a font package, and so one fills it, not necessarily knowing exactly who needs it. (It's one indirection from SetterInjection, where one directly modifies the behavior properties of another system). PolicyInjection is one step further removed from DependencyInjection: you now have a pool of font packages (i.e. a pool of factories that will return fonts on demand), and the user is now defining a policy to choose one font-package over another.

Context:

You have a decent DependencyInjection framework. You provide AbstractConstructor features via AbstractFactory with PluginArchitecture.

Let's suppose, for the sake of discussion, that your DependencyInjection framework takes URI-styled 'constructors' and returns 'objects'. Example constructors:

Presented with such a string, the DependencyInjection framework (in this case) will automatically break off the first word (up to the colon), filter the AbstractFactory objects based on that first word, then present the remainder of the string to each of them until one returns an object. The code for doing so looks something like this: 

 interface FunctorObject {
  Object call_with(String method, Array<Object> args);
 }
 .
 interface AbstractFactory {
   void   set_constructor(AbstractFactory); // SetterInjection
   String service_class(); // for filtering
   FunctorObject construct(String arg); // the construction command
 };
 .
 class AbstractConstructor : implements AbstractFactory {
   Set<AbstractFactory> factories;
   AbstractFactory fallback;
   .
   String service_class() { return "factory"; } // factory composition!
   void set_constructor(AbstractFactory af) { fallback = af; }
   .
   // Add factory to this!
   void add(AbstractFactory f) { 
    f.set_constructor(this); // loopback!
    factories.insert(f); 
   }
   .
   FunctorObject construct(String arg) {
     String choice, rest;
     choice = arg.substr(0,arg.find_first(':'));
     rest = arg.substr(choice.length()+1,inf);
     foreach(f in factories) {
       if(choice != f.service_class()) continue;
       FunctorObject o = f.construct(rest);
       if(null != o) return o;
     }
     if(null != fallback) return fallback.construct(arg);
     return null;
   }
 } // end class
 .
 // Example Objects
 class CHttpFetcherFactory : implements AbstractFactory {
   AbstractFactory ctor;
   void set_constructor(AbstractFactory f) { ctor = f; }
   String service_class() { return "http"; }
   FunctorObject construct(String arg) {
     if(!starts_with(arg,"get")) return new CSimpleHttpOp(arg);
     FunctorObject cache = ctor.construct("cache:http");
     return new CCachedHttpFetcher(cache,arg);
   }
 }
 class CCache implements FunctorObject {
   Map<String,Object> cachemap;
   Object call(String method, Array<Object> a) { 
     if("get" == method) {
       return cachemap.find(a[0]);
     } else if("set" == method) { 
       cachemap.insert(a[0],a[1]);
     } else if("reset" == method) {
       cachemap.clear();
     } else return null;
     String op = (String) a[0];
     if(!op) return null;
     if("get" == op)
 }
 class CCacheFactory : implements AbstractFactory {
   Map<String,CCache> caches; // singleton CCache objects.
   void set_constructor(AbstractFactory f) {}
   String service_class() { return "cache"; }
   FunctorObject construct(String arg) {
     CCache c = caches.find(arg);
     if(null != c) return c;
     c = new CCache();
     caches.insert(arg,c);
     return c;
   }
 }
The above 'AbstractConstructor' does some useful work - it filters by service class, which allows conventions to be developed on a per-service-class basis and potentially allows faster dispatch and construction. A points to note are that it injects itself into the factories it provides, which means that those factories will call back into one another to build objects. The 'semantics' of each constructor string are somewhat informal... or determined by convention rather than edict. I.e. it is likely expected that "cache:http" returns the same object every time, even if this rule isn't enforced or documented.

There are a number of problems with the above code. Performance could be improved a great deal (e.g. presort AbstractFactory objects by service class). StaticTyping could readily be extended for the common classes we know about ahead of time (i.e. have AbstractFactory<Type> and AbstractFactory<FunctorObject> only as a fallback). The 'synchronous call' nature is also severely problematic... it would be very easy to make the above enter an infinite constructor loop, especially when building two services that are interdependent, so making the system more asynchronous would be wise. And speaking of asynchrony, the above isn't MT-safe, and making it MT-safe without risking deadlock means going a lot more asynchronous than one might initially think. The fact that the constructor is limited to strings is also problematic! That approach causes string URIs to end up containing the 'constructor' details for other objects... and every AbstractFactory becomes an "object configuration" language rather than having one common ObjectConfiguration? language. (But that's a problem for later.) Also, the semantics could be further solidified (i.e. provide FirstClass support for singleton constructors, make the 'service_class()' much less ad-hoc) and security could be improved (distinguish factories based on security level, for example, so that high-security factories can't access lower-security factories, and a security-level may also be placed on areas of the application-configuration).

But I'm writing this page to focus on just one problem: the order in which AbstractFactory is selected is poorly defined. That is, we may have many factories for a given service class. If we have many "http" fetchers, which should we select? Which one is the highest performance? What about selecting a guishell... which windowing system should it use? (KDE? Gnome? WxWidgets? Motif?). If we have many windowing systems, which one is selected?

In a PluginArchitecture, where the AbstractFactory objects are added to the AbstractConstructor based on a pile of DLLs in some directory (or code in a CPAN database, etc.), this problem is greatly exacerbated. I.e. adding a new DLL adds new AbstractFactory objects which might suddenly take over construction of certain objects by an accident of ordering. (The application is also able to add application-specific factories.)

Problem:

We want options and plugin-extensibility. No reason to give those up. I.e. we want the ability to support two different plugins that provide font-factories, because this allows us to fall back on one plugin when the other fails. But we also want a predictable system (determinism). More than that, we want the ability to influence the system. Pursuing the example, if we have two different font-factories, it would be rather nice if we no the order in which fonts are selected isn't going to change randomly, and it would be even better if we could choose beauty vs. performance.

An extension of this problem is that of "discovery" - i.e. informing the user which options are available, so that they may make informed policy decisions and have knowledge as to the impact of those decisions. In a PluginArchitecture, especially, discovery is important. But this only becomes an issue after static user-influence is supported. I'll get back to it after fixing the first issue.

Naive Solution #1: My first attempt was a simple priority system. Each factory is given a priority number (an integer) and the order the AbstractFactory objects are tried within a service_class() is determined by this number, and additionally by other properties (like plugin DLL name) such that the final decision is deterministic. 

 interface AbstractFactory {
   void   set_constructor(AbstractFactory); // SetterInjection
   String service_class(); // for filtering
   FunctorObject construct(String arg); // the construction command
   int    priority(); // for determinism
 }
 // plus sorting in AbstractConstructor
This does provide determinism (which is a significant improvement). Additionally, and less obviously, it allows some useful idioms. For example, a plugin can provide several factory objects for the same service_class(), but at different priorities. This would allow a plugin to be 'good' at some things, and provide 'fallbacks' for other things (in a simple two-tiered factory injection). Plugins would then be able to take over for the weaknesses of other plugins, resulting in a best-of-breed design. That's pretty damn good for a single integer!

Actually, this is pretty good before the whole "best-of-breed" design idiom gets involved. If we wished to select between Cairo and WxWidgets, we could normally just move that decision into the application-configuration. But this Naive Solution is a victim of its own success! The ability to support best-of-breed designs results in users (well.. me, in this case) wanting to use ever-more-generic service_class() names. As an example, I'd end up using "guishell:root" instead of "cairo:root", or I'd end up using "joystick:discovery" instead of "direct_input8:joystick_discovery". This allows more programs to overlap, improves my chances for at least one plugin to provide the service, and makes the configurations much less specific to an OperatingSystem or machine.

Naive Solution #2: To overcome the issues of Naive Solution #1, I started by going even further in the same direction. (Hey! It was working!) I broke plugins into more tiers, and put more 'selectors' into the request. As a demonstrated example, the above clock specification was "clock:GMT-7.00,accuracy:minute,precision:nanosecond,drift:nanosecond/hour". That gets pretty busy, doesn't it?

It turns out that way lies a TuringTarpit. Selectors add ever more parsing issues, which means it takes longer for factories to reject messages, and takes longer to write those factories. There are more factories, too. One needed to be really careful to properly 'reject' based on certain features in the higher-priority factories. It's painful - like slow torture by a papercuts painful. That is not what I want.

Further, it still doesn't allow broad-spectrum decisions - one cannot, for example, make a decision to favor performance over quality for font packages for the application-as-a-whole. Instead, one would need to go around and edit the right-hand-side of that colon for every font reference in the whole application-configuration! Effectively, as the left-hand-side gets more abstract, the right-hand-side gets more specific. It was WaterbedTheory at work ^_^.

Additionally, all these interweaving priorities have a hidden performance hit that occurs elsewhere. As a consideration not named above, it is rather useful of plugins can be usefully cataloged in advance, with metadata about them (which factories they provide in terms of service-class and priority) pushed into a database so that we may later load a minimum number of plugins to get the work done. This catalog would need updated if the DLLs are updated, of course, but that's likely to be a relatively rare event. I.e. in the context design, one could at least filter plugins by the service classes they supported, and so one could load only plugins that support particular service classes. Then one would simply try each until it succeeds. With the specific service classes, this would usually mean loading only one plugin to get a given service. But, in solutions 1 and 2, once the whole 'best-of-breed with fallbacks' design gets involved and the broader service classes, it may require loading two or three or many plugins to select the appropriate AbstractFactory.

Better Solution #1: Okay. So simply having more tiers on one number doesn't seem to work. Let's try more numbers! I.e. for a factory, I might have (performance:10, quality:20). Essentially, AbstractFactory was changed to look like this: 

 interface AbstractFactory {
   void   set_constructor(AbstractFactory); // SetterInjection
   String service_class(); // for filtering
   FunctorObject construct(String arg); // the construction command
   Map<String,Int> features(); // factory selector
 }
Of course, maps of integers lack a TotalOrder. For determinism, which we still want, we need a total order for these factories. This is where the user-influence comes in. AbstractConstructor is modified to contain an arbitrary predicate operator for choosing factory priority, and users provide their own priorities... i.e. this predicate-operator becomes a form of PolicyInjection because user priorities are being injected into a decision-making process for automatic system configuration. 

 interface ICompareThings {
   int compare(Object,Object);
 }
 class CWeightedFactoryComparison {
   Map<String,Int> global_weights;
   Map<String,Map<String,Int> > service_weights;
   .
   void set_weight(String q, Int n) { global_weights.insert(s,n); }
   void set_service_weight(String s, String q, Int n) {
     Map<String,Int> m = service_weights.find(s);
     if(null == m) { m = new Map<...>; service_weights.insert(s,m); }
     m.insert(q,n);
   }
   Int get_weight(String srv, String q) {
     Map<String,Int> m = service_weights.find(srv);
     if(null == m) m = global_weights;
     Int i = m.find(q);
     return (null == i) ? 0 : i;
   }
   Int score(AbstractFactory f) {
     // ought to be simple, so people can understand it.
     // I suggest: sum of square roots of products of common features
     Int result = 0;
     foreach( (k,vf) in f.features() ) {
        Int prod = vf * get_weight(k);
        if(prod < 0) result -= sqrt(-prod);
        else         result += sqrt(prod);
     }
   }
   .
   Int compare(Object o1, Object o2) {
     Int score1 = score((AbstractFactory) o1);
     Int score2 = score((AbstractFactory) o2);
     return (score1 - score2);
   }
 };
I haven't yet implemented this new design (came up with it very recently), but it promises advantages over the prior solutions. For example: But, still, this solution seems to be missing something... not quite certain what, though, so I'll list my thoughts:

I'll give this a try and see what I bump into.

Discovery:

I posit that primitive 'Discovery' for the application is the ability to present a user with a catalog of available factories and their weightings, along with descriptive strings (names, text, license, manual), service class (and a description for that, too), and so on. This lets the user know exactly what is available, exactly which weightings and requirements can be used to select features, relative quality of implementation issues (priority, quality, performance, stability vs. experimental, security properties), and so on. In a distributed system, this simply extends to describing which features remote systems are willing to produce.

Once discovery is performed, appropriate weights can be selected, and a valid application-configuration may be written (i.e. one that is unlikely to fail). With a good app-developer, discovery could make construction of applications very easy... especially if arbitrary objects are added to the AbstractFactory 'constructor' so the app-configuration can build objects and hook them together.

Beyond primitive discovery is knowing where to get more features, and having prebuilt templates for tying features together.

I'll give this hypothesis a try at some point in the future.

This page raises an interesting issue, one that I've encountered in a somewhat glancing fashion in implementing DateAndDarwensTypeSystem for the RelProject. I see a possible correspondence between the points noted above and the use of specialisation-by-constraint to automatically instantiate (or, more accurately, to use Date & Darwen's parlance, select a value for) the most specific subtype of a given supertype. In DateAndDarwensTypeSystem, subtypes in a type hierarchy employing specialisation-by-constraint are not selected explicitly. Only the supertype may be selected explicitly, and the subtypes' specialisation constraints are evaluated to determine the value's appropriate subtype at run-time, i.e., the most specific subtype which defines a specialisation constraint that evaluates to true.

In short, whilst not specified by Date & Darwen's model (and, in some respects, possibly in violation of it as currently written), it seems reasonable that the weights discussed above could be incorporated into specialisation constraints in a type system that supports them.

-- DaveVoorhis (ThinkingOutLoud)

There is certainly a shared relationship to AbstractConstructor. I think, however, in your case it would be sufficient to support deterministic selection rather than give users control over the heuristics used to select in cases of ambiguity. PolicyInjection is much more useful at the higher layers of the system, where projects are developed piece-wise by so many developers that the right hand knows not what the left is doing.

EditText of this page (last edited November 10, 2014) or FindPage with title or text search