What Is Closure

Wiki pages such as TransactionClosureObject, UseClosuresNotEnumerations, CommonClosurePrinciple casually use the term "closure". I'm still having problems understanding what a "closure" actually is. I'm sure my problems stem mainly from overlapping or conflicting associations in the name.

Yes, there are a lot of definitions of closure. When a businessman says "I would like closure by this afternoon", he means something entirely different from a programmer who says "this function returns a closure", which is different from a mathematician who says "this function returns the transitive closure of f" which is still far different from your shrink urging you to find closure from your last relationship.

To a programmer, a closure is a function that can access interesting non-local variables. See what I wrote about closures on SmalltalkBlocksAndClosures. Someone else suggested "Perl FAQ: What's a closure" http://www.perldoc.com/perl5.6/pod/perlfaq7.html#What%27s-a-closure- (was at http://language.perl.com/faq/v2/Q3.14.html ).

The next section confuses the programming definition and the mathematical definition. The author tries to show how they are related. But though they might have some tenuous relationship historically, in practice they are two different definitions. The part that says that a closure "runs around and collects all the things in the environment at the time" is bogus; none of the programming language closures that I have seen do that. They are created knowing what is related to them. But except for that, it is pretty good. --RalphJohnson

If you have many things, some of which are related or connected by some function, and you traverse the set of things using one of those functions, then you will pull out a subset of all the things. That subset is called the "<function> closure" of the starting thing or of the set.

The example I grew up with was "transitive closure". If person A knows person B and person B knows person C, then, by chaining along the "knows" relationship, I can find out everyone who can be reached from A. "Chaining along" is the transitive property, hence, we would say, "Compute the transitive closure for "knows", starting at A." Six Degrees, one of the SocialNetworks, is doing this in fact, to see if all people in the world can be related by only 6 jumps.

In TransactionClosureObject, the GuySteele/GeraldSussman "Closure" is an extrapolation of this: "a closure is a function that captures the lexical environment in which it was created" - i.e., it runs around and collects all the things in the environment at the time. Over time, different things are around. The Transaction closure is about looking for a function that will connect all the parts of the transaction that have to be committed at the same time.

OK, now I've run aground on "lexical environment in which it was created". In my ignorance I had assumed that a "lexical environment" only exists during lexical processing, for example while parsing or compiling. This seems at odds with your notion of "runs around and collects". Can you give an example of such a closure, and what it would "contain"?

"Lexical" refers to the binding of local variables, and contrasts with "dynamic"= binding. Early Lisps had dynamic binding, so if there was a global "a" and a function "f" had a local "a", if that function called a further function "g" that also referred to "a", it would pick up the "a" from the function "f", not the global. Lexical binding is how more recent Lisps work -- and most modern block structured languages work -- "g" would see the global "a" because that's the one visible from the structure of the program (hence "lexical", since what's in scope comes from the text of the program).

So, the lexical environment is the set of variables visible to the function at the point in the program where it is declared/created.

Perl has both lexical and dynamic local variables. 

 $a = "global a";


 sub dynamic_test {
local ($a) = "dynamic a";
&print_it;
 }


 sub lexical_test {
my ($a) = "lexical a";
&print_it;
 }


 sub print_it {
print "a is ", $a;
 }
Then 

  &dynamic_test;
will print 

 "a is dynamic a"
and 
  &lexical_test;
will print 
  "a is global a"
So (in Perl): 

 sub create_bank_deposit_fn {
my $balance = shift;
return sub { $balance += shift };
  }


  $closure_1 = create_bank_deposit_fn (1000);
  $closure_2 = create_bank_deposit_fn (500);
creates two different functions in $closure_1 and $closure_2 which refer to two different $balances. 

  print $closure_1->(10);
would print 1010 

  print $closure_2->(10);
would print 510

Closures are really useful things. For instance for callbacks, instead of passing a callback function pointer + argument for that function, and having to work out in that function what you've been called back with, just pass a closure. When that function's called, it has access to all the lexical environment of when it was created.

This and much else is well described in the wonderful StructureAndInterpretationOfComputerPrograms (aka SICP).

Using the definition from the Perl FAQ at the top of the page, would it be fair to say that Java "Inner Classes" implement a form of closure? An inner class has access to the enclosing class' private instance variables. Likewise, an inner class in a method has limited (read only) access to the local variables of that method. -- JeffGrigg

A Java inner class is a *partial* closure. It only closes over the variables that are declared final. A Smalltalk block, OTOH, is a complete closure that closes over *all* the variables in its lexical environment. (Well, Smalltalk does not even *have* the variable attribute final. Sweet simplicity!)

Closing over final variables is a limitation that causes the developer to write more code to achieve the desired result. Instead of closing over an int that can be incremented (it can only close over a *constant* int!), the application developer uses an int[1] instead and can now update the int that is stored in the only element of the array!

-- PatrickLogan

Would you say that a JavaInnerClass? defined within another class (not a method!) is a full closure over the objects attributes? But a JavaInnerClass? in a method is only a partial closure? -- JeffGrigg

I wouldn't call it a full closure because a full closure in other languages do not have this restriction. An inner class is a useful thing, but it is not the same as a functional closure and arguably not as simple or expressive. I don't think it should be considered a closure, but something akin to a closure. -- PatrickLogan

Generally, languages that want to do closures (or closure-like things, like InnerClasses), have several choices for design/implementation.

As an aside, I get a bit fundamentally bothered about closures pointing into referencing environments that don't exist. What state should the environment assume? That which it had at exit? The environment that a closure references always exists, for such environments are garbage collected like any other object in the language. This is why you will never see closures in any languages that do not have garbage collection.

Actually, inner classes have to to with access permissions (public, protected, etc) and not with scope. You can create closures with a technique similar to BlocksInJava, giving the Runnable the variables it needs for each run. Of course you don't have to use a Runnable, you can use any class. -- TiagoSilveira

But inner classes are commonly used as closures... passing in the parameters isn't quite the same thing. It's the way that you can close over final variables declared in the same scope as the inner class. Nested classes on the other hand are a bit closer, in that methods in the nested class can access all the members of the parent class. --WilliamUnderwood

The Borland/Inprise/BorlandAgain? C++ compiler CeePlusPlusBuilder? defined a type which it called a "closure". It wasn't; it was more akin to a CsharpLanguage "delegate". It had no hooks into the referencing environment whatsoever; it basically was an OO-ish function pointer (that bound both the object and the method being invoked; so it could be called just like a C function).

The inclusion of closures in a programming language seems to doom it from popular acceptance. Closures seem to either frighten away people, or get the language labeled as "overly geeky".

This seems to be changing, slowly but surely - JavaScript has had fully-fledged first-class closures from the very start, and is probably one of the most widely used languages around (although, probably 99% of people using it wouldn't know a closure if it crept up behind them and shouted in their ear, even if they use them on a regular basis...). And now C# is getting proper closures, in the form of anonymous delegates, perl has had them for years, java and python both have things that almost work like closures, etc. -- MikeRoome

But then again, languages which don't commonly run into the "How can you get anything done in that? It doesn't even have 'x'!!" issue. E.g., I'm nowhere as productive as I feel I should be in java. Thanks to a large library I've built up, pretty much all of my java projects have an informally-specified implementation of half of common-lisp... just to ease that pain. A set of classes which emulate closures and operations on them are included in that set. And I guarentee that upwards of 80% of my project classes make some use of those. --WilliamUnderwood

(moved to SelfReproducingProgram)

I have noted several places where the term "closure" is used where "block" would be more appropriate. Many examples of so-called "closures" I've seen are merely examples of blocks. In particular, in UseClosuresNotEnumerations, the Java example at the top is an example of a block, not a closure. I'm being picky on terminology here, because to me blocks and closures are two entirely different beasts. I also abhor sloppy use of terminology.

Having just grokked the idea of closures in the last few months, I have yet to see a compelling (ie: practical) reason to use a closure instread of a block with pararameters. Why would you want to couple your block with its enclosing scope? Why would you want your block to <gasp> modify the enclosing lexical scope? Doesn't that severely break CommandQuerySeparation ?

I would love to be further illuminated on this subject! I also have a gut-feeling that this is why it seems there is more talk than walk about closures.

Many of the typical uses of enumerators involve some compositing of the values, returning a final value. In languages which don't easily support returning a result through multiple methods, making direct use of the defining context's variables is a typical solution. In effect, something akin to a closure is used when we need to pass a block to something which doesn't allow us any other easy way of passing information back and forth. 

 { 
        final AnUnaryIntegerFunctor functor = someFunctorReturningAnInt();
        final int[] totalSoFar = { 0 };


        AList list = someListUsableByTheAboveFunctor();
        list.forEach(new UnaryClosure(){
                public void perform(Object current){
                        totalSoFar += functor.result(current);
                }
        });


        return totalSoFar[0];
 }
It isn't a full closure, as only 'final' variables are accessible (although this makes no practical difference, see the 'int[]{0}' bit for instance), but I believe makes the point.

--WilliamUnderwood

I am about to read this page. I would hope that nobody is deterred by the size of the page. If closure requires this many words, I obviously have not yet any idea what it is. Although I thought I did before I got here. Well here goes --

Well. I did not know what a closure was. I thought it was a block. But it is not a block - it is a block which can have access to a caller's variables as if the variables were local to the block.

While I can vaguely recall something in the past where I have wanted them, I would only use closures (if they were available) where there was no pleasing way to use a block, where the argument list explicitly provides the called block's context. -- PeterLynch

The blocks that I am familiar with are Ruby's. In Ruby, one needs to list the variables that can be referenced in the block, and those variables can (but need not) exist in the surrounding lexical context. Blocks in Ruby can be converted to objects of class Proc, at which point they really are closures. I find that the listing of the block "parameters" is a bit confusing; they can either be declaring block-local variables, or they can be declaring a reference by the block to a variable in surrounding scope. (And of course, the term 'block' is confusing, too, since in many languages this refers merely to a delimited section of code, often with its own lexical scope.)

Closures are not that hard to understand if you get a firm grip on the notion of a variable binding. In a given context, a name is bound to a variable, which is a location that can store a value. Every time you execute some code that defines a local variable, a new binding is created. Examples are the bindings of parameters to their values within a function body, or any local variable definition (via LET in LISP, or an automatic variable definition in C/C++).

A closure is really nothing but a dynamically-created function, but when it's created, any variable bindings that are active in its lexical scope can be referenced by the closure, and persist as long as the closure does. This means that the exact same program text which creates a closure, executed multiple times, produces multiple function objects. Each of these function objects references a potentially unique set of variable bindings from its environment. Here's a fairly common toy example from Lisp that illustrates all the significant characteristics of closures. We first define a function that returns a list of two functions. These returned functions are closures; because they're created in a shared lexical context, they each reference the identical binding of ctr. The first function is capable of adding numbers to ctr, the second of subtracting numbers from it. If you're comfortable with the notion that 'lambda' is just a keyword meaning 'unnamed function'', the syntax seems quite simple. Each function is referring to the lexical binding of ctr, and also has a parameter that is locally bound (inc or dec, respectively): 

  (defun make-ctr-functions (ctr)
    (list (lambda (inc) (setq ctr (+ ctr inc)))
          (lambda (dec) (setq ctr (- ctr dec)))))
Now we can create two pairs of these functions. Each pair shares a binding of ctr, initialized to 1 or 5, respectively: 

  (setq functions-1 (make-ctr-functions 1))
  (setq functions-5 (make-ctr-functions 5))
Now we can call the closures as if they were functions. FUNCALL takes a function object and applies it to its remaining arguments. The return values give evidence that changes to ctr's value persist between calls, and that the closures stored in functions-1 and functions-5 are referring to different bindings of ctr with different initial values: 

  (funcall (first functions-1) 3)
    => 4
  (funcall (first functions-1) 3)
    => 7
  (funcall (second functions-1) 3)
    => 4
  (funcall (second functions-5) 3)
    => 2
  (funcall (first functions-5) 3)
    => 5
Now, personally, I don't often see a use for the 'multiple-closures-with-shared-bindings' technique. In cases where this would be useful, I would usually define a class and methods. But I guess there might be simple cases where this lighter-weight approach (in terms of amount of source code and conceptual overhead, at least) might seem preferable. This toy example does, I think, demonstrate all of the fundamental characteristics of closures, including how simple they are to create.

-- DanMuller

Thank you for the example and explanation. I have not yet given your description enough thought to justify comment really but I cannot help but think that the concept of blocks you describe in brackets above is closer to the english word closure, while the example shows exposure of variables in other blocks. More openure than closure. (-- PeterLynch?)

Hehe, I like that, an 'openure'. And some of the detractors will no doubt now come along with a reference to 'man-ure', but since closures are so simple to create, they're more like 'auto-ures'. :)

Perhaps you won't find the term so odd if you focus on the bindings. A closure 'closes over' the lexically visible bindings and includes references to them inside the function object, which you can think of as an opaque, or 'closed', object. Works for me, anyway.

It might be worth mentioning that language implementations typically optimize away unreferenced lexical bindings, AFAIK, so supporting closures doesn't have to be terribly heavyweight in the typical cases where you're actually referring to zero or very few outer bindings.

Although confusing when first encountered, once they click for you, they seem quite natural and simple to use.

-- DanMuller

Auto-ure gives me an inkling of a conceptual framework which I hope will fit - that of automation - closure appears to make what I think traditionally of as passive, active. It adds a level of automation at the bottom, available for almost arbitrary application. The idea of passing not the command, but the procedure for achieving the task. Anyway I hope I will see it more clearly. Your input has helped, or is it encouraged, me greatly.

CategoryClosure

EditText of this page (last edited August 16, 2010) or FindPage with title or text search