Lightweight Documentation

I've been following the discussions on ExtremeProgramming and TheSourceCodeIsTheDesign with a lot of interest and I've been trying to reconcile two (opposing?) points of view:

I like the idea of making system knowledge explicit - its one of the things that got me interested in patterns in the first place
I don't like traditional system documentation - its instantly out of date, often misleading and expensive to maintain

It seems to me that a lot of our problems with documentation come from the significance we invest in it. Culturally, we tend to believe what is written down must be the truth. Historically, documentation has played a large role in the specification and procurement of big systems and so must, for legal reasons, be heavyweight and high-maintenance.

In order to reconcile the need for capturing system knowledge that can't immediately be gleaned from the code with the simplicity of not writing any documentation, I'm trying to foster a new style of documentation in my project team. The guidelines for writing documentation are simple:

Whenever you feel there is a concept in your work that isn't obvious in the code, write a note about it (see TechnicalMemos)
A note is signed, dated and explains the concept and context very briefly ... overall should be less than a page
Place the note in a shared repository - maybe a database, a file-system directory or a project Wiki

These are the guidelines for using the documentation:

All documentation is to be distrusted as out of date and subjectively biased
Always favour talking over reading the documentation ... its out of date and subjective and talking is more fun anyway
If you don't know who to talk to, look at the documentation to see who writes the most notes about the bit of the system you're interested in ... then go and talk to them.
If you need to explain something you did and you can't quite remember how you got to where you are now (and you think it's important to know) skim through a few of the notes you wrote to jog your memory
If you absolutely can't talk to the relevant person, use a combination of the notes and the code to try and figure out what they were getting at ... but remember to distrust the notes as out of date and subjective.

This approach reduces the problem of out of date documentation, not by making the impossible demand that all documentation is kept up to date, or by just ignoring the need for it altogether, but by simply removing the assumption that it should be up to date. It also reduces maintenance - if you have a concept that can't be understood in the code you should write a note about it. You may be able to re-use an old note in some way but don't worry about scouring through lots of documentation and changing all references, just get it written - it's only a page long and no-one is going to trust it as the truth anyway.

This approach also encourages reflection and review - I often write short notes about stuff I've done just to get it clearer in my own mind.

Finally, this approach doesn't preclude the production of the 'traditional' documentation if it is needed. We are about to convert our system to Java at the request of a client. It is important we capture early knowledge about the conversion because the conversion team will grow rapidly ... we will make heavy use of 1-to-1 mentoring but there are going to be times when it isn't possible to have the most experienced convertors available to the less-experienced people. The intention is that we will produce a lot of notes during the early, exploratory phases of the conversion and then refactor the notes into a conversion handbook, for use in the later phases. -- PaulDyson

I really like the distrust notion being at the beginning of everything. Wish, though, there were some way to do something even easier than typing up some document. Anything much more distracting than scribbling onto a card can be too much. Good pointer! -- RonJeffries

Sadly enough, I've been using computers so long that I can no longer read what I've written - typed documents are a must :-) -- PaulDyson

Whenever you feel there is a concept in your work that isn't obvious in the code, write a note about it

I've taken to asking my coders to write the note from a generic viewpoint, asking themselves if that piece of documentation could be written up in a way so it becomes useful for other projects, and this project is just one example of that concept in use.

It follows from this that only those concepts that are worthwhile to carry away from the project are those that deserve documentation, and others are not so deserving. Write patterns, not explanations.

I think there's also a subtle sea change occurring due to this as well - my coders are more often checking online resources for existing documentations of patterns (etc), and in some places a page long documentation has been replaced with a one line "this is implemented as a insert name of concept". -- EricScheid

A few years back, I coined a related principle that I called LoRD: LocalityOfReferenceDocumentation. It tries to combat the "out of sight, out of mind" problem when documenting code and design. I initially wrote it up here, but decided it was large enough to move to its own Wiki page. -- BradAppleton

I found that LiterateProgramming (LP) is the best way to keep documentation in sync with code. Because both are in the same document, it it much easier to update either of them. Some people might consider LiterateProgramming the opposite of LightweightDocumentation. I believe that this really depends on how you use it; nobody is forced to write large amount of documentation just because s/he uses LP. I just wish it would be easier to include pictures into documentation. -- ChristianLindig?

My team has started using TwikiClone for documentation and other communications. Eventually, we'll have to produce heavyweight, hard-to-maintain "real documentation", but during the development process we're putting design decisions and review comments into the wiki. Has anyone else tried this? Did it work? -- KrisJohnson

I'm currently taking part in a seminar about advanced functional programming where questions, discussions, and results are recorded with a WikiClone. Often pages are updated during the seminar by the moderator who also projects them onto a screen. I think this is a very good way of recording decision processes. -- ChristianLindig?

moved here from DocumentationOutsideTheSourceCode

I really like this idea of LightweightDocumentation. I think a lot of the aversion to documentation comes from the cultural baggage associated with it: if it is written down it must/should be true. There is probably some historical baggage too: huge volumes of specification that no one could understand, were always out of date and were an absolute pain to maintain. But it does have its uses to communicate ideas that can't be communicated face-to-face (because the idea's originator is busy, in a different country, working for a different company, or dead ... whatever).

Summarizing some thoughts:

Documentation should...

Reflect the current state of the development/system
Not be overloaded with mostly bureaucratic content (it contains only the relevant technical details - whatever you consider that to be)
Be precise enough to be useful without being unmaintainably excessive (Developers should feel it's worth reading and maintaining.)

Basic Documentation

If you just stick to the basics, you are 80% there.

Database
- Describe each entity
- Describe each column if not self-describing (whether all columns should be self-describing is a contentious subject.)
Describe each file and directory in code base

CategoryDocumentation