The Source Code Is The Design
The Source Code Is The DesignA traditional manufacturing product cycle has a design phase and a manufacturing phase. If we want to draw an analogy between a manufacturing product cycle and a programming product cycle, the programming activity is analogous to the DesignPhase, while the compilation is analogous to the manufacturing phase. Note that compilation is practically free, as opposed to the manufacturing phase in a manufacturing product cycle.
Many have insisted on treating programming like a manufacturing activity. The main reason for the confusion is that TheSourceCodeIsTheProduct is an illusion caused by the expensiveness of producing source code, and the inexpensiveness of compiling executables (ManufacturingIsExpensiveButCompilingIsCheap?).
TheSourceCodeIsTheProduct is an invalid assumption. I think this is the big revelation, which might explain why people shout TheSourceCodeIsTheDesign (when maybe perhaps they should be shouting TheSourceCodeIsNotTheProduct?).
See WhatIsSoftwareDesign, which was the seed that started this page, and which discusses JackReeves' article of the same name.
This resonates with some thoughts about project management. It seems like a common error in software project management is in treating all the stuff that developers do as a manufacturing activity. This seems to be a driver behind a lot of poor management decision making, and unrealistic goal setting.
Maybe if software managers understood better (I could just leave it there ;-) that software product (executables) may indeed be manufactured in any quantity in a short time, flawlessly and cheaply by the machine, they wouldn't try so hard to map all the activities upstream from final build onto a production line.
I guess everyone from FredBrooks on has being saying these things, but MichaelFeathers' comment really crystallized this idea for me.
(If you are a software development manager reading this, be assured I'm talking about someone else.) -- KeithBraithwaite
Since programming is a design activity, not a manufacturing activity, you shouldn't automatically think you can treat programmers like assembly line workers. TheSourceCodeIsTheDesign might help explain why throwing more programmers at a project that is behind schedule will hurt it, not help it. -- francis
A software developer is more akin to an artist than an assembly line worker, since they are designing in writing source code. It takes time, consideration, and in some cases, a bit of skill. Creation can not be rushed. Throwing more people into the design mix can be counterproductive, unless one of them takes the reigns and becomes the architect. More people can be helpful to get the work done, but first you must determine what needs to be done in terms of the design. -- JasonBurkert
The position of the article was not that source code makes all other documentation obsolete, it is simply that the act of programming is designing. To be quite honest, I sometimes fear that people who read about XP superficially may get the impression that it is "let's go in like gangbusters and hack something up." Especially those who come from places with formal trails through a development process, and the managers chant "what do you mean you don't have a pile of documents??" You don't really hear the word design thrown around in XP. Yet, I do believe, from experience, that WardAndKent are right. You do just have to try things out in code. It is how you solidify and validate design. TheSourceCodeIsTheDesign.
The thing about TheSourceCodeIsTheDesign is that it recognizes that coding is part of designing. The difference between someone who can step up to the computer and program/design something well without having elaborately diagrammed it, and a hack who paints himself into corners is really one of experience, talent, and motivation to learn. OK, but what about the LessAbleProgrammer?
The artificial line between programming and design causes tremendous waste. Everyone thinks that software is intrinsically different. That it requires a completely different approach from anything else in engineering. It isn't different, we just misunderstood it. We believed that coding and testing were not part of design. They are just like prototyping on a breadboard. Part of design.
This is one of the TwoIrreparableMistakesOfTheSoftwareField, IMHO. -- MichaelFeathers
This just occurred to me, but if TheSourceCodeIsTheDesign, then are UML diagrams, etc., merely different views of the design?
Shouldn't the tools be used to extract the diagrams from the source code (I know most tools have this option, but using it is usually frowned upon)? Use Cases may be a problem, but isn't it also a problem when reading source code in determining what end user functions it can perform. Finally, should refactoring show up as cleaner diagrams?
Anyone else care to speculate on how TheSourceCodeIsTheDesign principle could reverse the relationship between UML and source code? [ consider that TheMapIsNotTheTerritory ]
-- WayneMack
ReverseEngineering is all about this reversed relationship, trying to find out how to distill design representations (potentially UML) from the source code.
I think the Reeves article touches on a very interesting and subtle phenomenon in human value systems in which intermediate effects come to be valued as much as their end effects, and it gets difficult to tell exactly where the value lies. When the translation of source code to machine code is highly predictable, it's natural to shift attention to the source, even though no machine can run it directly. But similarly, machine code has no value when the machine is broken. Because of the high availability of working machines, we don't have to focus there. Where we have to focus is all that matters.
If I look very carefully at anything I value, I see a complex chain of effects feeding it. Stable systems quickly become taken for granted, so we place our value tokens on the parts of the changing world that matter. If a design is something other than a product valued for itself, then it is merely some effect that is also a cause in a system stable enough for us to predict a value outcome.
Most computer systems are stable from the compiler down to the execution of binary instructions. Therefore, it's natural to think of "product" as the artifact just above that base. That's language source code. The system that produces that artifact is still quite unpredictable, so it's not likely we'll shift our orientation. Artifacts more abstract, be they whatever, will constitute a nebulous range of "designs" or "specifications". I think it's the lack of a direct and strictly repeatable translation of these artifacts that characterizes them. ("Specification" is interesting nomenclature, because it's really the lack of specific-ness that keeps it from being the product.) Nor is it possible to sort out all the possible forms in this category and give them names.
It seems wrong to say that any one of these tenuously connected artifacts is the design. Unless, of course, it's the only document you have <s>. So, while Reeves has caused me to think in more detail about the chain of effects present in software development, he hasn't convinced me that the source is the design and not the product, practically speaking. But then he didn't claim to have discovered something of practical value, right?
On Dichotomy
"Advocates ... hold that the definition of a concept specifies its meaning in its entirety. Objectivists hold that the units integrated by the concept constitute its meaning." -- From: http://enlightenment.supersaturated.com/essays/text/ioe1/09.html
Literate Programming tried to balance the equation between Design and Source Code by having a single document depicting both design and source code (TeX and Mathematica are good examples of this kind of programming). Even though this style was not prominently picked up by the mainstream software community, of late, the tool LEO (http://leo.sourceforge.net) has done a remarkable job in this arena. A single XML file contains both the document and the code with a GUI front end depicting the software structure as a tree. You can selectively extract source code or comments from this document. Unlike any CASE tool, Leo does not use any extra intelligence to generate its own source code. You get what you put in there in the first place. In any realistic development the code always evolves from the first cut before it is fully functional. The beauty with Leo is that all these changes can be imported back into the original Leo document.
-- Hemanth P.S.
I believe that there is an interesting comparison to be made with electronic engineering and chip design. Proponents of CASE tools often promise to make software engineering more like electronic engineering, with UML diagrams analogous to circuit diagrams. However the design for complex digital chips is actually done using a programming language - the two leading languages are Verilog, which is similar to C, and VHDL which is similar to Algol. To quote a manual on Verilog (found at http://www.eg.bucknell.edu/~cs320/1995-fall/verilog-manual.html):
Digital systems are highly complex. At their most detailed level, they may consists of millions of elements, i. e., transistors or logic gates. Therefore, for large digital systems, gate-level design is dead. For many decades, logic schematics served as the lingua franca of logic design, but not any more. Today, hardware complexity has grown to such a degree that a schematic with logic gates is almost useless as it shows only a web of connectivity and not the functionality of design. Since the 1970s, Computer engineers and electrical engineers have moved toward hardware description languages (HDLs). The most prominent modern HDLs in industry are Verilog and VHDL. Verilog is the top HDL used by over 10,000 designers at such hardware vendors as Sun Microsystems, Apple Computer and Motorola. Industrial designers like Verilog. It works.
So in chip design, The Source Code Really Is The Design.
-- DaveKirby
Except now Object-Oriented HDLs are coming out such as http://www.aracnet.com/~ptkwt/ruby_stuff/RHDL/ based on Ruby. A UML tool that could generate Ruby could then be used to control the design of the HDL code. So it comes full circle.
Agree in Principle, Disagree on Semantics
I propose that the real issue is that design is not really a beneficial activity in software development, and to say "The Source Code Is The Design" is trying to use semantics to gloss over the issue.
I feel this is an important distinction if the goal is to remove the "design" stage from the software development process. Rather than being afraid of being accused of "not doing design", we need to turn the debate around to be "Why should we do design?"
For some tasks, it may be much more cost effective to create a design and evaluate the design before building the actual product. For software, this is not the case. For years, software has struggled to come up with something to use for "design." We had flow charts, PDL, Data Flow Diagrams, prose descriptions, and now UML. With software, however, it takes as much time to create the design as the actual software; the design is more difficult to validate than the actual software; and the simplifying assumptions made in the design are often the critical issues to evaluate in the software. For these reasons, it simply is not cost effective to design, iteratively correct the design, then write the software (and then iteratively correct the software). It is better to start with the software and iteratively correct it.
I believe it is time to explicitly state the long held secret of software, we do not need to do design; design is ineffective and costly.
-- WayneMack
See http://www.martinfowler.com/articles/designDead.html for a very different take on the need for design.
I am talking to a VC. They want to see something that shows we can do the software part of the project. Should I say, "So sorry, the source code will be the design?" Of course not.
A VultureCapitalist isn't going to be interested in SW design documents - source code or reams of dead trees. VCs are interested in whether or not you have a good business plan and business model. Of course, there may be bad VCs that consider themselves able to pass judgement on software specifications, but that's just bad lending practice and has little to do with good engineering practice.
A set of UML designs is not going to convince a VC of anything. Working software is useful, but as the previous writer has noted, the VC wants to see proof that the software can be sold for a profit. Tell a VC that you've gone through your initial capital to create paper diagram of what you want to do and he will probably just walk out the door.
If the source code is the design, then what are the UML diagrams and the project specifications that are created before the coding to be called? A design is a plan. From the programmers view, the source code is the plan because with that you can determine how your program will behave. On the other hand, if the source code is the design, then it is very difficult to communicate the design to someone except for the involved programmers. Maybe one can say that the source code is a view of the design? Just like design documents, UML diagrams and text descriptions are other views of the design?
UML diagrams and other associated paperwork (if any) are documentation about the design. A documentation can have different views as appropriate to shed some light (if any) on various aspects of the object described. However, the source code has a privileged status: it is not just "documentation about", it is the object itself.
There are lots of partial points here... Some developers actually like to "design" and document - and there is value in that. Design is the sanity check and collaboration that happens on a white board, the reduction of logic from a large K-map(yes - these apply at times - does anyone remember what they are?) to a code approach, and the thing you do anytime you get out pencil and paper to work a problem because the IDE in front of you doesn't allow for it. There is also value in moving quickly to code for an experiment... experience has shown the need for both. And the reality is that there are still XP practitioners who avoid any design efforts or artifacts to a fault. There are Architecture classes where people are trying to choose the LDAP product and the App server before they have found out who the customer is... too much predisposition to action is too common... streamline steps, trash the unnecessary ones, but don't just skip them because they aren't interesting to you.
XP supposedly is about collaboration - SO why is it just about collaborating only with other developers? How do you describe your product to the customer's IT department? The question is more about limiting the artifacts and maintenance of them, and then the communication to other parties that NEED to know... support will not read code, sometimes the deliverables are more than just code, VCs actually do "technical due-diligence" - not just "business due-diligence" - and deciding you know enough about "vulture capitalists" to discount their use of consultants who actually might want to see design - I hope you aren't expecting to woo any investors or market to anyone with technical savvy.
I think too many developers have been ducking the communication skill problem - if you can't present your "design" in meaningful terms to those who would support it, use it, invest in it... you have come up short on the developer skillset. Jobs include the ability to communicate to the business. Knowing which artifacts matter, when and how to use them(minimally - absolutely; not at all - I don't think so!), when informal capture can be enough, what you can reverse out through tools and WHAT YOU CANNOT, how to convey to others, and how not to get bogged down by it... not simply evading that part of the job... comes from experience and collaboration and a predisposition for aggression+quality... that is what we should be talking about.
The binaries are actually the PRIMARY deliverable - not the source - but not necessarily the only one; Also it is not inconceivable that non-code models can be used to generate generate binaries. MDA is way young and perhaps not destined to produce much... but I have worked on projects where, thru code, we took further coding out of the loop! Code that produces code - is a trend - I wouldn't get completely tied to source listings as an excuse for documentation... just be sensible - and not evasive - about reality. If you think you know it all and that things are black and white - you are young.(Even Kent Beck on occasion (paraphrasing) has said you can be too extreme)
-Jeff Smith-
You make some nasty assumptions there by saying that the binaries are the primary deliverable, you're wrong. First, it's a mistake to assume there are binaries, you just excluded all the dynamic languages extremeo's love so much. Secondly, you're assuming that the product is where the value lies, but many products are produced for the purpose of being sold, not directly for a particular customer, and often the writers of that code are considering the company itself as the product, hoping to be bought out by a bigger company... who wants that source.
TheSourceCodeIsTheDesign also brings up interesting conclusions on what SoftwareDevelopment is about. Since we are continuously in design mode until release, aren't we always prototyping? Are we delivering RefinedPrototypes?, prototypes that have been deemed GoodEnough? May be, but don't let your VultureCapitalist know that!
An analogy that came to mind when reading the article was regarding OpenSource. OpenSource is about giving the right to use the design; if you have the tools (which in many cases are FreeSoftware) you can "build the product". A striking difference is what happens when a manufactured good's design is given away. In general, the public cannot use the design to "instantiate" the product without incurring considerably amounts of time/money.
Design supposed to be something having its own value and impact to success of project. From this point of view design is what we define it and what meaning is most valuable and efficient for development. But still definition of design should finally reflect its purpose and thus be one of starting points of design related discussions or one of its goals.
What if there is a system designer in a team who does UML diagram and sends them further to developers does it mean that developers are just coding monkeys who blindly implement diagrams and interfaces or they do something more? And what it is? Is it next more detailed stage of deign or it is something what should have. What developer should do for program design and what he can be allowed if he is for example senior developer and allowed to participate in many activities? How senior level developer supposed to design software?
In my personal opinion, it is much better to separate coding and thinking in time or as separate team roles. Coding supposed to be precise and fast. The quicker you do it the more time may be spent for other activities like prototyping, designing, writing unit tests etc.
-Denis S.
Sometimes an hour of thinking can save you a week of coding, but sometimes an hour of coding can save you a week of thinking. Rapid prototyping crosses all phases of implementation and design. In practice, with Applied R&D work of any sort (be it software or hardware engineering), you can't really separate the 'thinking' task from the 'implementation' task because each continuously feeds back into the other. In some circumstances you may be able to separate the 'requirements analysis' phase from the others, but even that often fails because the customers often only have a vague idea what they want and limited ability to communicate what they want.
Of course, sometimes one's hand is forced. When the project is too large for any one team to have the expertise to complete all of it, one will need to parcel out 'pieces' of a design to several different teams, then somehow integrate them. This is not an ideal condition for the project as a whole since feedback is slowed down quite a bit... and it will certainly require advanced thinking to predict and eliminate potential integration problems before they happen.
Last time this page was edited was 2006, is 2010 now and some foggy areas are polishing up. MDA has come some way through (only in certain areas) and software developers have not yet been made a commodity (although some managers tried hard). To me looking backward at the papers of years ago, there is a lot of truth still and I deeply believe in the code is the design, the essence is that there is a continuum of activities and competences that start from one corner of the zachman diagram and ends up in the opposite one.
There is a more and more fine grained detail of information about a system that start from a high level taxonomy, into a domain model, into an application architecture, into system design, into code. This expands with other items such as Vision, Planning, User Interface, Business processes, Business requirements and so on so forth. It's just the same stuff, is all about design, from the first visionary project statement down to the last line of code written 20 years later in a support call. The first one is called project initiation, from the build up to the change management of the software is called TOGAF, the last one ITIL.
The problem with this is that there is no single man (maybe a superman) that can span in breadth and depth all the possible competences needed to make a large scale piece of software. Worse than that managers play a strange role biasing and trying to force their pink glasses on the whole picture. It's like that we started from crafting and teaching from father to son and nowadays we are in a weird factory making drawings for a puch card machine that ends up with some fully fledged robot or even worse that we have tools where we can start from the assembly code and drill back to coverage of business requirements and vision.
The parallel with the continuum goes on thinking that there is a bit more creativity in what we call everyday the design part (i.e. the bubble diagrams) and there is a bit more concrete foots on ground (finalization) perspective in the software engineering part.
I live in a company where architects are elderly application architects, that have some management role, and when you refer to struts-config.xml they don't know what is it. So on one side architects can talk nonsense with "patterns" (i.e. the big name MVC) and then when the pattern becomes live, as a scary living tiger that can eat you in one go, then they say I'm just an architect. It's just when the salsa start, and struts-config.xml has been famous to be the best of MVC and the ground 0 of MVC altogether.
To me, in this crazy world, there are a lot of people hiding their incompetence behind complexity. Source code should produce design and design should produce source code, if we don't have yet proper tooling/languages is simply another movie.
What seriously worries me about this...amongst many things.. Source code does not have nearly enough metadata to describe design or intention or governance. Also leaving things to evolution is dangerous. I want to nurture smart organisms, not delve into panspermia - JonathanCrossland
So?
Disproving this assertion is fairly simple. The C language standard has implementation defined behaviors, unspecified behaviors and undefined behaviors. Source code won't convey on which of these it is dependent unless it is commented (generously assuming comments are code rather than simply embedded design documentation) to a level few are capable of producing, and therefore the source code alone is often not sufficient to predict behavior. Two classic examples include "x = x++;" and memory mapped data structures for IPv4 headers which use "unsigned long" for the addresses. A number of developers assert the first has defined behavior because the one compiler they used to test it didn't emit a warning and the resulting executable produced a particular result, and become quite unruly when a compiler upgrade results in different behavior. The second assumes an "unsigned long" will be 32-bits, which is obviously not true for a range of systems. Then there are bitfields...
To look at it another way, could anyone with no C experience derive the complete design of the C language (which defines a standard set of headers and library functions in addition to the base language) with nothing but the source for say linux/glibc/GCC/binutils? Would they be able to identify where C99 and POSIX are incompatible, and which behaviors belong to which standards? Could they differentiate these features from OS, library and compiler extensions? Could they know what behaviors are implementation defined, unspecified or undefined? If the whole of the design is not captured in the code, the code cannot be the design.
Rather than clog up this already cluttered page with more discussion, I'd like folks to go visit WhatIsArchitectureAnyway. Source code is not design, it is implementation. I describe why on that page. -- MartySchrader
References:
Code as Design: Three Essays by Jack W. Reeves http://www.developerdotstar.com/mag/articles/reeves_design_main.html
Also, for a complementary view look at Component Software by Clemens Szyperski ISBN: 0201745720, there is a very interesting exposition about what is and where the design of a software system resides.
TheFooIsTheBar: TheDesignIsTheSourceCode, TheSourceCodeIsTheSpecification, TheAtomIsTheDesign
More about documentation vs source code: ToNeedComments, SourceCodeAndModeling, TheWhatButNotTheWhy, SelfDocumentingCode, DocumentationBeyondTheSourceCode
See: WhatIsSourceCode, UnderstandingHotDraw, TheSourceCodeAndTheArchitecture, ProgrammingAintManufacturing, WhatIsSoftwareDesign, DesignOfTheDesign, ExtremeProgrammingAintPrototyping, SoftwareBlueprints
BeingRefactored?: http://donald.fed.wiki.org/view/the-source-code-is-the-design
EditText of this page
(last edited November 3, 2014)
or FindPage with title or text search