New Attempted Homoiconic Definition

[Moved from HomoiconicExampleInJava, hopefully will eventually move to back to HomoiconicLanguages, and hopefully will short-circuit some of the confusion in all these pages including HomoiconicFaq. P.S. thanks to whoever it was that pointed out a week or two ago that FirstClass was important but getting overlooked.]

For a primer see DefinitionOfHomoiconic

The Tcl/Tk site says "The Tcl programming language is an homoiconic-form language. Program and data are both presented as strings. A Tcl procedure's arguments list and body are not an exception to this rule, but the procedure itself is handled as a name bound to a particular couple of arguments list and body. This name lives in a separated namespace and does not collide with variables names." http://www.tcl.tk/cgi-bin/tct/tip/187.html

Another usage: "Lisp and Scheme are homoiconic: self-representing", p600 of "Programming Language Pragmatics" by Michael L. Scott c. 2000 ISBN 1558604421

Compare with RaphaelFinkel in AdvancedProgrammingLanguageDesign: 

  set a 4 -- a := 4                                     1
  set rhs {expr $a +} -- rhs := "expr $a +"             2
  set rhs [append rhs 5] -- rhs := "expr $a + 5"        3
  set b [eval $rhs] -- b := 9                           4
  set cond {$b > 0} -- cond := "$b > 0"                 5
  set body {                                            6
    puts "b is now $b"                                  7
    set b [expr $b - 2]                                 8
  }                                                     9
  while $cond body                                     10



  set a ile -- a := "ile"                               1
  wh$a {$b > 0} {set b [expr $b - 2]}                   2

Lisp specifies that code and data are both S-expressions, code S-expressions can bet passed/returned/assigned to variables and have contents mutated, so code is FirstClass.

The Tcl language specifies that code and data are both represented as strings, which can be freely used as values regardless of whether the string is code or data.

In Smalltalk, code is a second-class value -- it can be passed as a parameter, for instance, but the contents are not generally mutable at runtime, despite some reflective capabilities, so it's not quite homoiconic.

In Java, code is third-class -- it's not a value at all, by the language definition, regardless of what an implementation does with it.

Essentially all languages represent source code as some sort of value, such as a sequence of bits or a sequence of characters, but that matters not at all if the language definition doesn't specify that that code is FirstClass. Most languages are thus not homoiconic.

[Very useful definitions moved from here to the top of the page.]

Doug,

To justify "homoiconic" as a useful definition, you should either avoid defining the notion through the mechanics of it, as the definition you provided so far is fuzzy enough and I have big doubts that is useful. You can start by defining what functionality, what is the benefit of the programmer in having a homoiconic language. If the benefit is just having an eval function that given a structured representation of an expression, it evaluates that expression, then you only need to observe that eval can be provided as a third party library in Java without modifying the JLS one bit.

I never claimed that Java is homoiconic as it is today. But given what I understand as homoiconic, and what it is good for, I must: accept that Scheme and Lisp are homoiconic, reject that TCL is homoiconic on grounds that its representation of programs as data is unstructured, accept that Java as it is today is not homoiconic. But then I shall observe that the difference between Lisp and Java that if added to Java would make it offer the same end-user benefits as Lisp, well that difference is not a matter of language design, because one could easily leave Java language specification in place, with its syntax and semantics unaltered, and write a library that will make up for that difference. QED.

So please come up with a better functional definition / motivation for homoiconicity if you disagree. The fact that you do not master what it is that you want to define, was proven when you provided a test that actually tested for ability to lazy evaluate an expression. If you think this is a very important property of programming language, then it has to be subject to meaningful tests. I.E. normal users of programming language do not give a damn if its MetaCircularEvaluator or not, wether its structured as cons versus structured as objects etc. A meaningful test can be for example: I can do X easily in LISP, versus I can't do it in Java, nor in Haskell nor in ML. -- CostinCozianu

I am trying to capture what the experts in the rest of the industry mean by the term, and I have a certain amount of confidence that I succeeded this time, despite the fact that, yes, the underpinnings of FirstClass and of 'same representation' are, alas, themselves not defined as precisely as we would like, no doubt leading to further arguments.

It doesn't matter that you disagree that Tcl is homoiconic, because recognized experts in the field say that it is, and I understand and agree with their arguments.

[personal chaff deleted]

Your suggestion for writing about motivations for homoiconicity is not without merit, but you are completely off track in suggesting that such is more to the point than the simple definition I present.

The term "homoiconic" is not at all widely used, but it goes back 30 years, has been used by a number of well-known people, and when it is used, its definition is industry standard: I've never seen any disagreement about its meaning outside of these recent arguments on c2. This is not something I made up! The difficulty is just that it seems to have been used with an implicit rather than explicit definition. Apparently this is the first time in history that its use was challenged endlessly rather than accepted without comment. -- dm

Some motivations for why someone, somewhere, might care about homoiconicity (not limited to just the end user). These are not definitions.

Homoiconic languages make it very easy, sometimes even trivial, to:

Once again, the reverse is not true...just because some or all of the above are offered by a language does not mean it is homoiconic. That's not the point. I'm just addressing a little of Costin's issue with motivation/functionality. -- dm

Having read all this discussion on homoiconicity, I'm a lot more confused than I was when I first read it. I think part of the problem is that it's being referred to as an advantage or a feature, when it's really just a property - it's incidental to what you can do with a language.

Providing runtime access to the languages compiler allows you to do everything you homoiconicity grants you without necessarily being homoiconic. There's a real feeling of "nothing that doesn't match my ideal of a language is allowed" coming from some of the arguments. Would you consider Python to be homoiconic? The standard library provides structured access to code, including code which is currently running.

In reviewing the above, it seems that FoxPro (ver 2.x anyway) fits the definition.

For years I wrote code that wrote code and executed it on the fly. Moreover, I could deal with an exception by creating arbitrary code (manually or automatically) which handled the condition and resumed execution. I had programs in which there were routines that deliberately wrote the appropriate code fragment for handling a specific value or situation and always executed the code thus dynamically created.

I was able to store code fragments in tables, use these fragments to generate modules at run time and then execute them. Code could be stored in variables, which variables could be executed either by &varname or evaluate(varname).

This allowed for extremely flexible code. It allowed a running program to redefine its tables and its treatment of those tables. The TCL example above can (with concessions to syntax) be rendered in FoxPro.

It's possible that my understanding is flawed, but the impression I get is that, intentionally or not, FoxPro is (or was) such a language. -- GarryHamilton

(This was a feature in 8-bit dBASE-II, upon which FoxPro was based: The "&variable" syntax was like a macro -- it put the string contents of the variable in that place /before/ the interpreter parsed and executed the commands of the line. -- JeffGrigg)

That certainly is powerful, and for many purposes, a highly desirable facility. It also is not "homoiconic", at least, not as you described it so far. What you describe could potentially be programmed in any language, it seems (using a code generator, which might be built into FoxPro or might not, you didn't say, but it doesn't matter for the purpose at hand, although it certainly makes such things as you describe much easier).

By contrast, a language is either homoiconic right from the get go, or it is not; it doesn't have anything to do with which programs/libraries/facilities are written in/for that language. A homoiconic language remains homoiconic when stripped of all external executables, all libraries, all include files, all everything. Further, a language that is not homoiconic cannot become so by adding executables, libraries, include files, etc.

Can you write code to achieve roughly the same end effect, in absolutely any language at all, as long as it is a TuringEquivalent language? Well, sure, that's what TuringEquivalent means.

The problem is, when reading this set of pages, listening to (mostly anonymous, and therefore hard to distinguish) arguments from all the people who didn't know the word "homoiconic" until they started arguing on these pages. (Why do people argue so passionately about topics they don't know? It's very weird.)

If you look more carefully, the history of the word is documented hereabouts from its first origins, and the original definition is clear, it's just that it turned into a religious war, where certain parties wanted to claim that all languages are homoiconic, or something, which most certainly and absolutely is not the case.

(There was also argument about whether it is a strict category or a fuzzy definition, but that ended up being merely an irrelevant rhetorical device, since the same issues arise with absolutely all categories and definitions, yet the modern epistemological tools for that topic were never used whatsoever.) -- DougMerritt

It also is not "homoiconic", at least, not as you described it so far.

-- GarryHamilton

Yes, well...leaning, ok, all the way there, it doesn't sound like it. The latter ranges from simple BootStrapping all the way to MetaCircularInterpreters, depending on details, but none of that spectrum is the same thing as Homoiconic, although yes, there is a relationship (that has been discussed hereabouts).

TCL is in fact considered a HomoiconicLanguage, not just because it supports eval, but because code and data have the same essential representation: strings. (Most languages support strings, leading to unwarranted claims that therefore all languages are homoiconic, which badly misses the point, perhaps by badly misunderstanding the nature of strings in TCL vs C/Java/etc.

So does Foxpro use the same representation for code and data? A yet-to-be-evaluated expression in TCL IS a string, it's not a trick or an option. A yet-to-be-evaluated expression in Lisp (the original Lisp 1.5, say, to avoid side issues) IS an S-Expression, not as a trick or an option, but because that's what it truly is. -- DougMerritt

So does Foxpro use the same representation for code and data? 

  store table.cmd to cmdvar
  store table.arg to argvar
  store table.cond to condvar
  if &condvar
    &cmdvar &argvar
  endif
And, of course, more exotic contortions are available. -- GarryHamilton

Interesting. Ok, sounds pretty similar to TCL so far. One last hurdle: is this the normal way that the language handles conditionals? It is, in TCL. If it's merely an optional facility, then it would be going too far to call the language homoiconic, but perhaps it would be accurate to say it has an optional homoiconic facility. I.e. what is the scope of what "homoiconic" refers to here, the whole language, or a subset facility of the language?

Similarly, although Lisp is homoiconic, one could point to some implementations of Lisp compilation as an added facility which is not homoiconic (unless it takes pains to make that compilation transparent to the usual eval/quote/etc), although the rest of the language is homoiconic.

(I do not, however, see any reason to start trying to rate "degree of homoiconicity", despite some such suggestions on these pages; any particular facility either is or is not, any particular entire language either is or is not, etc. It's a qualitative statement, not a quantitative statement, and I don't see any associated metric space.) -- DougMerritt

One last hurdle: is this the normal way that the language handles conditionals?

I would say that the use of the macro ("&variable") notation is common usage in dBASE and FoxPro - largely because some of the commands of the language have such a fixed definition that you need it to work around the limitations of the language. For example, "USE" and "SELECT" commands, used to open and read from a file/table, take a table name -- not an expression that evaluates to a table name. So if you want to use a file/table specified by a variable value, you must use the "&variable" syntax to "modify the code" at run time. -- JeffGrigg

RE: "Java code meets none of the definitions of FirstClass value." [from near the top of this page]

From the FirstClass page:

"FirstClass features can be...

If one accepts that code is represented in Java as a byte array ('byte[]' type), then all of the above are true of "code in Java." ...even the "dynamically typed languages" point. ;->

(If "code in Java" were a String type, all the FirstClass points would also all be true.)

RE: "Java VM code is really opaque a byte array that the classloader provides to the runtime, and which has to conform to an external specification (class file specification). Byte arrays representing code can be manipulated just like the other values, but programmers are not offered structured access to the code represented in those byte arrays, not by default. There are quite a number of industrial and open source Java software that do perform transformations on these byte arrays (including generating new code dynamically -- without access to JavaC, modifying existing code, etc)."

- and -

"[...] So the bottom line Java is currently non-homoiconic, because code as data is not structured. Java could be made to have exactly the same benefits as homoiconic languages by adding a bunch of libraries to the mix. Java would still not meet the structural definition of homoiconicity ("the code is represented the same as data")."

So... If the BCEL library were declared a Java standard library, then Java would be a homoiconic language? ...as the BCEL library provides structured access to read and change the Java native byte array representation of code.

If so, then we need to talk about the .Net environment: Microsoft provides a standard API for translating both ways between source code strings and byte code representations for C# and VB.Net. And it's intended that all other .Net languages should support these interfaces. So therefore, all .Net languages could be considered homoiconic. On the other hand... Perhaps "common usage" or "ease of usage" should (?) be part of the homoiconic definition? [That was my argument lo these many months ago. Homoiconicity isn't boolean, it's scalar. It's also subjective, since "ease of use" varies from person to person. But several folks here are strongly opposed to such a definition. -- EricHodges] Yes, I've been thinking...

"Is Java homoiconic?"

"Yes, but not very much."

"Is C# (.Net) homoiconic?"

"Yes. Slightly more than Java. But still not much."

-- Jeff Grigg

I don't agree that you can exclude the standard Java libraries from "The JavaLanguage". The class 'java.lang.Object' is typically stored in a separate library, not part of the JVM. If you exclude that library (by deleting it, for instance), no Java programs will run. All Java classes are a direct or indirect child of 'java.lang.Object', so no class can be loaded into memory without it. And Java without the other 'java.lang.*' classes wouldn't be much of a language. It wouldn't have Strings or exception handling, for instance. Would one argue that the 'java.util.*' classes are not part of Java?

In my opinion, the JavaLanguage, as defined by Sun, includes the standard libraries. That is, "Java" includes the published interfaces for everything in the Java Platform, Standard Edition (J2SE). But it only includes the runtime, the Java Runtime Environment (JRE). The "JavaLanguage" does not include the libraries unique to the Java Software Development Kit (SDK). Nor third party libraries, even if commonly used. However, if I can write it in standard portable Java, then I can do it in Java. -- JeffGrigg

Your comments seem far removed (on the page) from previous discussions about that issue, but ok. You've got a very good point. Some previous discussion has been about it being irrelevant to add a new library to some language to make it homoiconic. You are correct that standard libraries are, these days, typically considered to be part of "the language definition", and somewhere around here there are probably comments that imply otherwise, but this is all beside the point, so...so? -- Doug

True, if adding the BCEL library to Java was necessary for Java to be homoiconic, then Java must not be homoiconic now. However, if Java with BCEL is homoiconic, and BCEL is just a convenience, embodying a set of techniques that I could apply myself in my own Java source code, then Java without BCEL would still be homoiconic. BCEL would just be a convenient tool that could be used to demonstrate that Java byte code can be interpreted and modified by standard Java programs. -- JeffGrigg

Well, but above I already answered that, I said (to paraphrase) that it is an untrue claim that "Java with BCEL is homoiconic". So the rest doesn't follow. -- Doug

How about MyNaiveAttemptAtUnderstandingHomoiconicity?

MetaDiscussion? moved to HomoiconicMetaDiscussion

JuneZeroFive

EditText of this page (last edited November 12, 2014) or FindPage with title or text search