Why Is The First Arg Special

From LanguageGotchas, in reference to why it seems natural to pass the 'this' parameter in a method call by reference, and yet seems natural to pass the others by value:

I wrote, deleted, but now restored this because I'm just a wishy-washy person: There is plenty of evidence that pass-by-reference as a default rule is evil, nor have I ever heard anyone claim that pass-by-value all by itself means "not OO". That cannot possibly be true. -- dm

: Well, consider how dispatch is done: basically, self is the first parameter. If this isn't by reference, you're going to have a tough time convincing anybody that you have an OO system. Given that, things like 'move method refactoring' as a simple source code modification can't really exist; move a method from one type to another is basically reordering the parameters, and if the parameter ordering matters (as it would in a value-passing-except-for-self style system) then the semantics wouldn't be consistent.
: I know this isn't a waterproof argument, but I think it is a starting point.
: -- WilliamUnderwood

That troubles me for reasons I'll have to think about.

Since I made the troubling statement, may I ask exactly which part troubles you?

For exactly the sorts of reasons that DavidHopwood? is currently irked about in JavaPassesByValue and JavaDoesntPassByValue (and also see CallByReference AlternativesToPassByReference).

Regarding "evil", pass by reference as the basic language rule for all parameters certainly is evil, because of the unfortunate frequent appearance of unobvious aliases, which leads to large numbers of hard-to-find bugs. This is an ancient, well-known issue.

But Cwillu's point above makes it harder to formulate the correct insightful rule (and my deleted/restored sentence above obviously isn't the right thing to say), as does the strong lack of agreement on the pages David is editing.

I feel like I'm forgetting something important about all of this, that would cut through to the heart of it all. -- dm

[Next comment was left on LanguageGotchas, as it doesn't relate to the 'this' parameter specifically]

I don't think I'm disagreeing with any of that; that's similar to points made on those other pages. I'm talking about something else. The thing is, it is known that it is wrong for a language to have default semantics of pass by reference for all parameters, but cwillu clarifies the point others have made: that we all expect pass by reference as the default semantics for the passing of the object that owns the method.

I hadn't thought of it like this before, but basically we have an inconsistency between what is correct for the first parameter (the method's object, which is implied rather than explicit in some languages) versus what is correct for all other parameters (in the default case).

That's a Smell, so I suspect that something is being overlooked -- something that I already know, but which is not coming to mind. (And I do not see how this is resolved by the distinction someone attempted between "passing objects by reference" versus "passing object references by value"; that smacks of sophistry rather than insight). -- dm

I agree that it is a smell. It's one of the reasons that I've become more interested in MultiMethods (as implemented, e.g., by CommonLispObjectSystem) than in traditional OO notation. MultiMethods don't have this asymmetry. -- DanMuller

(They actually do have an asymmetry, though not one that people usually think about. But that'll have to wait till I have time for WhyIsTheFunctionSpecial?. -- jt)

What, never? -- dm

No, never. MultiMethods don't belong to a particular single class or object, and all parameters are treated equally. Put them in whatever order you like. Yet they still give you runtime polymorphism; they can be polymorphic on any single parameter or any combination of parameters that you'd like them to be. -- danm

I know what they are; your paragraph didn't use the word "reference", which is the topic, thus: cue the next line, playing straight man for you again:

What, never? -- dm ...after which, since you were already on track, should have come:

CAPT: I am never known to quail At the fury of a gale, And I'm never, never sick at sea! ALL: What, never? CAPT: No, never! ALL: What, never? CAPT: Well, hardly ever!

ALL: He's hardly ever sick at sea!

Captain Corcoran and his crew, H.M.S. Pinafore, Gilbert & Sullivan, Song No. 4 -- Act I

"...which in it's time became so familiar that it is told that one newspaper editor forbad it to be used in his paper. 'What, never?' asked one of his staff. 'Well', groaned the editor, 'hardly ever!' "

(Digress? Who, me?)

Well, I think that the distinction between PassByReference and "passing object references by value" (which I'll call PassByPointer? for brevity) is legitimate. Most of the difficulties with PassByReference come about because you have no idea what any variables passed into a function hold after the call. With PassByPointer?, you at least know that they'll point to the same thing, though the internal state of the object may have changed. The whole point of OO is to localize state, though, not eliminate it.

Also, the "natural" mode for passing the first argument is really PassByPointer?, not PassByReference. When you call anObject.aMethod, it can't change the value of anObject in the calling scope, only the state. It's not the same as aMethod(Object& anObject, arg1, arg2 ...), where assignments to anObject can actually change which object the caller points to. This is congruent with the way Java/C#/VB pass the rest of their parameters.

Is there something besides that that's smelly? I think there're a lot of problems with thinking of OOP method calls as syntactic sugar for aMethod(anObject, arg1, arg2), but I want to see what you're driving at first... -- JonathanTang

(Responding to dm.) Ah, now that I know who wrote that, I also know that I didn't need to explain MultiMethods in more detail. But my answer still stands: No, never. An instance of a multimethod may not treat all the parameters equally; one parameter may seem more "right" as by-reference, another by-value. But it won't be because one parameter is intrinsically special due to syntax. It will be because of what the routine is supposed to do with the arguments, or because of efficiency considerations -- or because of other limitations of the language (e.g. type-based limitations), which might be another (distinct) source of unpleasant odors. (One can detect this one wafting through CommonLisp, too.) Aside from the latter case, this leaves the programmer who defines the function's interface free to choose the appropriate parameter mode. It doesn't smell bad to me if both modes are available; they both have their uses.

Try looking at it this way; leaving aside efficiency and aliasing considerations, by-value means you're interested in the value, by-reference means you're interested in modifying state. In an OO language that has const methods, there's nothing to prevent the system from passing "this" by value to them. The effect would be the same. So by-reference should only seem "natural" if the intent of the method is to modify the object. (If we bring efficiency into the discussion, we'll have to mention "const references" and aliasing issues.)

(Responding to jt.) I agree with dm that the distinction between "passing by reference" and "passing references by value" is not much of a distinction. Some sort of information (value) gets communicated from caller to callee, so every "pass by reference" is a "pass by value" at some level. I'm not a Java programmer, but isn't your example "aMethod(Object& anObject, arg1, arg2 ...)" showing a parameter which is a reference to a reference (seeing as all object variables are implicitly references)? (Can you even do this in Java? I just looked at some BNF and it doesn't seem so. But see http://www-106.ibm.com/developerworks/library/j-praxis/pr1.html.) -- danm

[strikes me as a java + c++ syntax]

And just to throw a wrench into things, where does smalltalk's 'become: <anObject>' fit in? In other words, the reference to a reference bit has its place. It's not quite the same consideration, as you couldn't split off an object given to a method from the same object used everywhere else, but it still seems somehow on topic.

--cwillu

It's C++ where I got lazy and left out the class and the types of the other args. I'm using C++ for all these examples, as the debate is now on various parameter-passing mechanisms, and Java only offers PassByPointer?. I know about JavaPassesByValue; I don't think it's the most useful terminology, but I'll live with it, which is why I say Java passes by pointer and hides the referencing/dereferencing.

Take a look at this C++ code and its output. It's all pretty basic value vs. reference vs. pointer stuff, you'll probably recognize it:

    #include <iostream>
    #include <string>

    struct A_Class {
        std::string foo, bar;
    };

    void print_object(A_Class an_object) {
        std::cout << an_object.foo << " " << an_object.bar << std::endl;
    }

    void by_value(A_Class an_object) {
        an_object.bar = "has been modified.";

        A_Class replacement;
        replacement.foo = "A replacement";
        replacement.bar = "has been substituted.";

        an_object = replacement;
    }

    void by_reference(A_Class& an_object) {
        an_object.bar = "has been modified.";

        A_Class replacement;
        replacement.foo = "A replacement";
        replacement.bar = "has been substituted.";

        an_object = replacement;
    }

    void by_pointer(A_Class* an_object) {
        an_object->bar = "has been modified.";

        A_Class* replacement = new A_Class;
        replacement->foo = "A replacement";
        replacement->bar = "has been substituted.";

        an_object = replacement;
    }

    int main() {

        A_Class original = { "The original object", "in its original state."};
        std::cout << "Passing by value: ";
        by_value(original);
        print_object(original);

        A_Class original2 = { "The original object", "in its original state."};
        std::cout << "Passing by reference: ";
        by_reference(original2);
        print_object(original2);

        A_Class original3 = { "The original object", "in its original state."};
        std::cout << "Passing by pointer: ";
        by_pointer(&original3);
        print_object(original3);

    }

Output:

Passing by value: The original object in its original state. Passing by reference: A replacement has been substituted. Passing by pointer: The original object has been modified.

Note the difference between references and pointers. The reference lets you make the original value point to something else entirely. The pointer only lets you modify the fields of the original.

I don't see this, unless it's a reference to a pointer. If it's a reference to an object, you can't change the actual object as defined by its allocation/location in memory, you still can only change its fields. It sounds like you're visualizing a pointer to a pointer, or a pointer to a reference (or a reference to a pointer, for completeness, but not a reference to a reference -- since, unlike pointers, the operation of take-reference is idempotent, not cumulative).
[Correct. If 'this' were passed by reference, it would be a pointer passed by reference (i.e. a pointer to a pointer). A language that supports only passing by value can support passing references or pointers by value, but won't support passing integers or references or pointers etc. by reference.]

Java semantics match by_pointer, except that it garbage collects the heap objects (instead of leaking them ;)), and gives you syntactic sugar so you don't have to worry about reference and dereference semantics. A Java method call won't affect the bindings of any of the variables (either object or primitive) in the calling scope. It may affect the state, as in PassByPointer?, but it'll always have the same object identity.

More to the point (and what cwillu's getting at), PassByPointer? semantics match what we expect for the receiver of the method call, the 'this' pointer. It would be very odd if calling a method on an object meant you no longer had a handle to that object, as in PassByReference. It would also be very odd if calling a method on an object meant you couldn't affect the object's state at all, as in PassByValue. PassByPointer? - semantically equivalent to passing-object-references-by-value, in Java at least - is the only logical choice. There's no asymmetry in Java/C#/VB, but there is in languages that support other parameter passing modes. -- JonathanTang

JT, your example makes a distinction between by-pointer and by-reference that doesn't exist. The smallest of changes (a single character) makes by_pointer() act just like by_reference():

    void by_pointer(A_Class* an_object) {
        an_object->bar = "has been modified.";

        A_Class* replacement = new A_Class;
        replacement->foo = "A replacement";
        replacement->bar = "has been substituted.";

        *an_object = replacement; // Note the asterisk.
    }

By-pointer is not in any significant way different from by-reference, at least not in C++. It's the merest syntactic sugar. A common idiom (which I adhere to) is to use a reference when you intend to access an object only within the context of a function, and a pointer when the function intends to save the pointer value and reference the object later; but semantically the two are nearly equivalent. (Exceptions: You can change the pointer within the context of the function to reference a different object, as you did in by_pointer, but this is rarely useful and rarely done. You can test a pointer for NULL, whereas a reference has undefined behavior if NULL.) A reference does not allow you to "make the original value point to something else" -- what you did by assigning to an_object in by_reference() is change the state of the referenced object. In C++, at least, object identity is tied to storage, and you modified the existing storage that was passed in -- it's still the same object. You may have made some of the object's data members point to different strings, but that's another thing entirely. -- danm

Is that essentially the same as my small bullet point above? Yes, which I think you added after I wrote and couldn't save the above on my first try. :) Yeah, happens to me all too often. I was just checking, to make sure I wasn't stupidly missing the point of a scathing rebuttal of what I said or something. :-) Incidentally, an edit conflict like that prevented me from posting 5 paragraphs above that weren't especially insightful, and instead I went off and thought for a while instead. Sometimes it's a good thing. Agreed. In this case, I scanned quickly and missed your addition. That's what we get for (ab)using wiki like a newsgroup or chat channel. Sometimes I think newsgroups and wikis should come in pairs ... but I digress. --- You don't habitually click on the last edit link to see the most recent diff?? Or you mean, you usually do, but didn't happen to that time? Exactly the latter. It's late here, off to bed.

So my partial conclusion so far:

That this is a deep issue, not a small isolated issue; I think I'm part way to seeing some interconnections.

This is the balance point that distinguishes several different paradigms.

In C++, I want to say "of course the method's parameter should be passed by pointer" -- but you know, that's precisely the issue of pure functional vs not. In a pure functional language, we would in fact want to pass a copy of the object, operate on it, and return the modified object. Which then has to be captured by assignment on return from the method. Which leaves no room for any other return values (naively speaking).

Which points out some part of why the coexistence of OO and Lisp/ML family languages is often perceived to be an uneasy one (although obviously things like Lisp and OCaml have objects). -- dm

In a pure functional language there is no notion of "operate on it" that results in a modified object. All the operations one might utilize to "operate on it" would themselves be functional, so what you're really doing is identifying a value to return (possibly by constructing a 'new' representation-object for the value under the hood, but that is an implementation detail highly subject to optimization). Destructive assignment is anathema to functional optimizations and benefits.

OO and functional can coexist easily enough. The trick is to interweave them but keep them separate: use pure functional to perform calculation, and use OO to perform behavior (IO). This works well with the SeparateCalculationFromIo? pattern. The sort of intertwingled impure functional approach to accomplishing this is what creates much confusion and violates a number of assumptions for useful optimizations. Lisp and OCaml are good examples of how one shouldn't go about achieving both OO and functional in one language.

That's a strong statement, considering that CLOS is considered by some to be the best object system around. Which functional languages do achieve the right blend of OO, in your opinion?

If you think about distributed programming, you'd realize there shouldn't be any problem with passing by value a reference-to-an-object. Essentially, one is passing a URI to a communications service. This value can then be utilized functionally (e.g. to produce an XML document containing the URI) or behaviorally (e.g. sending an HTTP 'get' request to the URI). There is nothing special about this 'value' excepting that some service identifies it also as an 'address' for sending things (which, for electronic messages as opposed to FedEx deliveries, must be messages represented as values). There shouldn't be anything jarring with thinking of passing a URI as passing a value. Similarly, there shouldn't be anything jarring with passing 'this' or 'self' as passing a value.

The only thing 'special' about 'this' and 'self' is that their underlying representations are often opaque and tend to be optimized for the memory service. For orthogonal distribution and persistence and mobility and replication and survivability of services, this optimization creates significant AccidentalDifficulty in mapping and communications. For operations wholly upon a local machine, however, they are a step of indirection faster.

Since a method can't assign to self in Smalltalk, this hides any distinction between references and pointers for that argument, and you can think of the implicit argument self as being passed by reference. As a result, all arguments, even implicit ones, are passed in the same manner in ST. That you can't change self in the callee is an orthogonal limitation.

Further, in Smalltalk, you have all of the passing modes; by reference, by pointer, and by value. It's just that only one is by default and the mechanism to access the other passing modes are oddly asymmetric.

To pass by value in Smalltalk, all you have to do is use copy. So "Car with: door1 with: door2" becomes "Car with: door1 copy with: door2 copy". This is simple and is exactly how passing by value is implemented in VW's remote invocation framework. The good thing about this is that you can override the default passByReference within the callee.

The weird thing about passing by pointer is that you can't specify it on the caller's side. But you can certainly access that functionality on the callee's side. As cwillu pointed out, it's #become:. And there, there is no distinction between self and any other variable.

To make passByReference the default passing mode in Smalltalk, all you have to do is redefine assignment. So instead of substituting a different object in a variable, assignment would take over from #become:. There is no problem with doing this as long as you reify variables as primitive objects that delegate messages to their targets, and then provide a primitive message #substitute: to substitute the object in a variable for another object. So ordinary variable assignment would be written "firstDoor substitute: door1".

But of course, this is all smoke and mirrors. Reversing the symbols for substitution and becoming wouldn't make one of them any more of the default passing mode in Smalltalk. Not unless there were a way for the caller to dynamically redefine what := means for that argument in the callee. Until this happens, you can't say that Smalltalk is passByReference anymore than it's passByPointer.

The traditional point is simply to distinguish pass by reference from the far more common pass by value (and pass by name: the old fashioned dynamically scoped Lisp rule). The normal case in Smalltalk is pass by value.

Don't blame Java too much, I think they took it from Smalltalk. I'm pretty sure that Smalltalk passes by pointer value. And I know that you can't write a #swap: without resorting to the #become: primitive. Smalltalk won't let you store to an argument. I take it pass by reference only makes sense for functional or bad procedural languages?

The semantics of Java are fine, it's just that they confused everyone by calling their brand of pointers "references". True pass by reference as the default for all parameters turns out to be evil; the last time it happened was in a bad procedural language, long ago.

I attempted to make a clearly definitive statement about this over on JavaPassesByValue. Multiple other people had already made the correct statement, but the arguments were verbose, varied, and there were counter-arguments, so previously it was hard to separate the wheat from the chaff.

I haven't made myself clear. Smalltalkers frequently call ST pointers references.

Yes, nor is that unheard of in various other domains, but in Java the overall situation is that it confuses people. It gave me brain freeze, because e.g. Jonathan said "they're pointers", and my brain replied "but Java has no pointers, Sun said so!" :-) I couldn't resolve this by thinking about it (not being much of a Java expert, and out of practice as well); I had to actually investigate JavaNature? further. So it turns out that Sun lied (I'm being dramatic) -- Java has pointers, but not references, which unfreezes my brain.

I don't recall hearing Smalltalk use "we have no pointers!" as a selling point like Sun does with Java, so I think people have no issue with talking about pointers and references interchangeably.

"pointer" and "reference", in fact, are typically synonyms. Except e.g. C++ has both, and they're not the same thing, and Java claims to have one but not the other, but their claim is backwards of the truth.

pass-by-reference is a decades-old term, and although I don't recall offhand people talking about pass-by-pointer, I would assume the two terms meant the same thing if I did hear it.

Above I said "the distinction someone attempted between "passing objects by reference" versus "passing object references by value"; that smacks of sophistry rather than insight".

However, they were right, that is exactly the distinction (although not the correct phrasing), it's just that they didn't really explain, and that they should have said "passing object pointers by value, not "references".

I couldn't remember whether you could write a swap() function in Java. You can't. Therefore JavaPassesByValue, period.

The problem is that Java redefined the word "reference" to mean "a safe kind of pointer", which confuses many many people. Java does not have "references" in the older non-Java sense of the term. Jonathan was correct above where he talked about pointers.

The first arg isn't special in being pass-by-reference; it's still pass-by-value.

However, I also said that this is non-trivial, although the above question about Java becomes trivial once their terminology and the swap issue is resolved.

The first arg is special in that we want to pass an object pointer (not an object reference) rather than the object itself as the first parameter, but in this light, I don't think it's a smell.

In an OO language that supports multimethods, nothing changes; we still want objects passed by pointer (by default) to multimethods in anything but a pure functional language, and passed by copy in those (aside from compiler optimizations). So I don't think it's really about pure OO methods versus multimethods, even though that is obviously about whether the first parameter is special in yet another important sense: selection of dispatch (which pure functional languages raise to a high art).

As I said above, this is the balance point that distinguishes several different paradigms, and this is what I continue to find non-trivial and interesting. -- dm

I'm hoping to see comments on language paradigms below, so I moved the Java comments up.

The reason that passing "this" by reference seems natural is that everyone uses "this" as a reference. It is, linguistically (in the Chomsky et al sense) a deictic, that is to say that it references something about the context in which it was uttered, and not some canonical entity. Therefore, the oddity of the idea of passing "this" by value has nothing to do with passbyreference vs. passbyvalue, it has to do with the lexical semantics of the word "this" in English; the English word "this" HAS no inherit value. - Josh O.

Sorry, but that's incorrect. For starters, it is false that it has particular semantics because it is a deictic because its name is "this", it's the other way around. We need particular semantics for particular things -- and note that all names, pointers, and references are deictics, so it helps not even a little bit to drag in that linguistic term -- and we call it "this" because that name is a reasonably natural fit for the semantics.

Secondly, the issue of why we need one sort of semantics versus another has been thoroughly addressed above. It cannot work to have "this" passed by value in the sort of languages in question; that would force the languages to become pure, with no side effects, which is a non-starter for these languages. -- DougMerritt