Problems With Documentation Discussion

(See ProblemsWithDocumentation to get the ball rolling.)

Are there problems inherent to the documentation and not to the people dealing with documentation (either readers or writers)?

Expense. Time. Difficulty in addressing the needs of widely varying readers. Getting out of date. Relying too heavily on foresight (see objections on OnlyForesightMatters). Most people don't like to read it (even reasonable people, even with really good documentation).

All of the above problems can be overcome by people willing to do so. Most professional tech writers, for example, will be able to overcome all the problems enumerated so far that are on the writing side of things.

Including expense?

Actually, yes. A good technical writer can significantly decrease the cost of documentation.

An example: On one development team , all the documentation is written by the programmers, of which there are twenty. On average, each developer spends four hours a week writing documentation of various kinds (internal, external, etc.), and they're each paid USD $40,000 a year. Working out those numbers, the company is paying USD $80,000 a year for documentation. A good technical writer makes about half that, and the documentation will be of better quality (all other things being equal). -- BrentNewhall

Does this actually save money? If design documentation is being produced, the developers will be idle until it is produced, so they might as well do the writing. If the developers can proceed without the documentation, why produce it in the first place? It seems you have added in the cost of technical writer without reducing any developer expenses.

The developers will not be idle during the creation of design docs; they will be answering questions from the tech writer(s) and proofing the design docs as the docs are being produced. Developers are not writers, so let's not thrash this particular dead horse yet again.

The original proposal was concerning a cost savings, not differences in quality of writing. If the developers are spending just as much time when the technical writer is used as when one is not, it would appear to be a cost increase, not a cost savings. The numbers above state the cost of no technical writer is $80,000 a year versus $120,000 a year with a technical writer.

But when spending $120,000 a year, more work is being done; the technical writer is working simultaneously with the developers, rather than the developers writing both code and documentation. That's the overall cost savings; you may be spending more in the short term, but you're getting much more work done, and it's being done more productively (the developers aren't struggling with writing documents, a skill they often do not have). -- BrentNewhall

The developers can not write code because the document is not done yet. For example my company do CMM (we went to level 5 last month). The process specifies that the detail design document and test spec document be done - approved - before coding can begin.

Okay. How is this a problem? Your process requires this. If you guys think the process is overly restrictive then change it. Process creation isn't written in stone, you know. Petition management to review and ammend the process to include some ThrowOneAwayInPractice coding or whatever you think is necessary.

Assumption: The cost of the documentation far outstrips its value.

We may agree or disagree whether documentation problems can be overcome. This, however, does not address the expense issue. Rather than taking documentation as a given, one needs to evaluate its value. Keeping documentation in agreement with software implies duplication; why is this duplication valuable and does its value exceed its cost?

Questions about this assumption:

If the code reflects the design document, does this not say the information in the two is redundant? Why is design done in prose any more efficient than design done in a computer language? If the design is done in prose, then it still must be rewritten into a computer language. If it is initially done in a computer language, no additional step need be taken.

A design document implies duplication only in the same sense as an index or contents page of a book does. To follow the analogy, it is not possible in general to say indexing is necessary. One might say that it would be useful in this case or that.

Before saying, "no one is ever going to read it" or "all you need to know is in the code", it is worth while considering the value of a design document as a quick reference, a plan for the future (and no there are no laws against this yet) or even as a kind of sales item. The only design documents that are redundant are those which are written because ThatsWhatWeDo?.

Please continue. What is the value of a design document for any of the above cases?

A design document can be shown to someone who is incapable of reading the code. Prose can be understood by all, but code by a very few.

How can there be a problem with documentation that is not a problem with the readers or writers?

Well, tools can print the documents improperly, or format them incorrectly. And people can cause many unintentional problems, but recognizing that fact doesn't help solve any ProblemsWithDocumentation.

My experience - which is as a technical writer, so I'm admittedly biased - is that documentation is often a victim of the Expert Problem. Specifically, documentation should be written by a good technical writer, but most shops don't recognize this, feeling that anyone can write the documentation, and thus don't have technical writers write their documentation. This is somewhat analogous to saying that secretaries should write their own word processors. The average programmer simply doesn't have the skills to write superlative documentation (through no fault of his or her own; few people are good writers, and documentation is a specific kind of good writing). -- BrentNewhall

Concur. I actually went bak too skool too lern technical writing. Worked for Motorola as a technical writer for a year. Hated it; too much like work. Now I find myself doing tons of writing for every client I serve. This is because of the above stated problem: technical expertise in a particular area does not imply capability in expressing the understanding of that area. My bane is being able to rite well enuf to express the technicalities to any audience -- as long as I know who that audience is. Blaming the documentation or the process for the inept way it is implemented is simply wrong-headed. --MartySchrader

Documentation is often used in lieu of fixing the problem.

Instead of fixing a software problem, a workaround is incorporated into the docs, with the attempt of giving the impression that this is how it was _supposed_ to work in the first place...really.

I have seen too many instances where, rather than fixing a problem, it is merely documented, often as a readme file on the install disk. When users encounter the problem, we then kindly refer them to the readme file, and afterwards berate them for not reading the fine manual.

Again, concur. Remember where I come from, folks -- many of my industrial and medical development projects absolutely required documents to back up the development process. But even without that kind of requirement, I still view the generation of documents to be a reflection of the level of professional care and consideration applied to a project.

I disagree. You often have to release a less-than-perfect product, and documenting limitations is a lot better than not mentioning them at all.

I think we are talking about more than a "less-than-perfect" product (all products fit this description), we are talking about a recognized problem. The key points are, does the problem get fixed on the next release, and why are we getting ourselves into the situation where we are finding problems like these at the last minute in the first place. Errata documentation is like band-aids. Band-aids are good, but it is better not to skin your knees in the first place.

If you really know what you are doing, you can describe it to somebody else. If you can describe it, you can write it down. If you can write it down, you can change what you wrote. My form of development involves writing things down just so that I can prove to myself that I know what I'm talking about before I go to the keyboard. How much more valuable is this when we go to convey it to a stranger? There is so much here to discuss.

Why is writing something in prose in a word processor superior to writing something in a software language in a code editor? Doesn't operational code convey much more to a stranger than an abstract document trying to describe the same thing?

The "prose" that you refer to is a common language method for communicating a complete idea: a problem, within its domain, all its limitations and exclusions, and the solution for this problem. Code may or may not convey this stuff accurately or even adequately. The written word, properly used, does that to a much higher degree of accuracy and completeness.

Often in my design sessions with other engineers I will describe a component using phrases like, "So this guy sends this message out to space and expects any other guy who recognizes the chatter to respond." This isn't a complete or even very technical description of what is going on, but it does convey the idea. When it comes time to formalize this description I will say essentially the same thing but take out the informalities.

In a case like this the idea is conveyed through its description, which is a very potent form of communication. Source code is not a very good mechanism for conveying an idea; it is simply a way of casting one particular approach in stone. Comments in source code expand on the particular method used to accomplish this goal, but they do not address the original idea the same way that a conversational description can do.

Think about it this way: Written laws codify the legal limits of behavior between people in a functional society. These limits are the bounds set by others on our behavior, but do we not set much stricter limits on ourselves? After all, we have documents like the Bible, Koran, etc. to show us the complete description of how a life is to be lived. If we just had the written laws then we'd be in a pretty pickle.

Code may or may not convey this stuff accurately or even adequately

I don't understand this statement. If the code does not convey something, why is it necessary to write a document to convey it? Why should the document describe something that is not part of the code? If it isn't in the code, it does not exist.

Oh? My code makes no mention of stdin. Therefore, stdin does not exist?!?

Well, if your code makes no mention of stdin (or its equivalent in whatever language you're using), the program isn't going to be reading from standard input. (A truism, yes?) I would therefore argue that (certainly from a documentation point of view) stdin does not exist for *this* application. -- NeilWalker

That is temporizing. The point is that there may be significant concerns that aren't covered by the immediate code under scrutiny. How do you know they exist, much less what impact they have? A document. To claim that these concerns simply don't exist is to put one's head in the sand.

Prose is a common language method for communicating a complete idea.

So who is doing the communicating and what is being communicated?

The ideas of the designers and developers are extracted by a writing professional. These ideas take condensate form as writing.

For developers to communicate to end users, operational code is far more expressive and complete than any textual document. Too much detail gets lost, misinterpreted, ignored, or left out in the written document. With the operational code, what you see is what you got.

Yes, what you see in the code is what gets executed by the target machine, but contains none of the thinking process behind it. None of the decisions, compromises, TheRoadNotTraveled, etc., shows up in the final code. Just the end result of the sometimes painful and always experience-filled path followed by the development crew.

For end users to communicate to developers, direct communication is best. Immediate verbal feedback is the ideal, followed by telephone calls, e-mail, and had written notes. One of the worst ways is to pass the comments through to an intermediary, have them sanitized, written into a document and passed to the developers. The better developers will get the updated document, pick up the phone, call a user, and ask, "What does this really mean?" When the developer gets pushed for time, tired, frustrated, or plain burned out, he will just go ahead and do what is in the specification. [Some would argue this is a good thing; we beat up on the developers and then criticize the users as well.]

This is the typical RedHerring used to blame poorly constructed documents when a bad requirement or bad design gets put into code. The real mechanism here is the closing of the loop by having the originator of the knowledge proofread the docs to make sure his wisdom has been properly distilled into written form. If this doesn't happen then the docs are just so much paper kaka.

[NOTE: In my experience, without a written set of requirements to show a client I could have wasted a lot of expensive time. Instead, the client says, "Do this." I write it down and hand it back to him. "Is this really what you want?" I say. "No, change this," he says. I change it and shove it back at him. "Is this what you want?" "No, change this." Etc., etc., until he finally sees what he wants on paper. Then I do it. Without the feedback iteration this scenario would be a disaster. -- MartySchrader]

Care to expand on the above aside? What is saved? Is it schedule time? Is it manpower? Is this a micro-level (single feature) saving or a macro-level (project wide) saving?

It is the effort to build something that the client didn't want in the first place that is being saved. By reflecting the client's wishes back to him in a cohesive document he gets a chance to see the mistakes in his own instructions. Doing it repeatedly until it's hammered flat eliminates this sort of waste.

Documents a product unto themselves

There is not a whole lot of market for documents as a product, outside of the "XXXX for Dummies" or "NutShell?" series. Documentation is at most a supporting element, not the end product. The documentation only provides value in terms of how well it supports the related product.

Before starting a new project, we usually have to produce a Cost Benefit Analysis (I'll come clean, that person is not me, it is someone higher on the corporate food chain). Documentation does not have to meet this requirement and more is always considered better. For every release, we produce both a 1 - 5 page "quick guide" for the user and a 150 page user guide to meet the requirements of our QA department. We ship the quick guide with every release and we haven't sent the 150 page version to the field in years.

Yeah, well, we all have these horror stories, eh? Are the requirements for the docs themselves written by people who know how to write? Do these people know how to write requirements? That takes a certain kind of perception which is lacking in most engineering and marketing/sales staffers. Anybody can bludgeon a problem to death with 100K lines of Java code. Few can create a solution with <1K lines of C++. Anybody can write a requirement that answers all the questions (well, eventually). Few can write requirements that get to the point quickly.

Anybody can hack out documents that require lots of trees be felled; few can write docs that say exactly what needs to be said without leaving anything vital out. We can all talk about how the documents were a pile of dung on this project or that project, but a few of us can point to projects where the docs said only what needed to be said. I have seen it happen and I have done it myself (not to toot too loudly). -- MartySchrader

Documents are usually expected to say "everything that needs to be said" hence my comment that "more is always considered better." The documentation, especially a requirements document, is used to replace (not augment) direct communication. The developers don't need to interact with the users, the information is in the requirements document. The testers don't need to understand how the product is used; all of the information in in the requirements document. The project reviewers do not need to know the usefulness of the product; the information is in the requirements document. The document is expected to foresee every possible question and provide the answer, so the documentation becomes the focus of the effort rather than the product. It is impossible to write minimalist documentation that will meet every possible need.

Probably the point I am trying to make is that "A problem with documentation is that the documentation is used in lieu of knowledge, understanding, and research."

Yes, I take your point. This is far too often the case. My take is that the document is the distillation of the knowledge gained through understanding of the results of research. The document is only useful if it crystalizes that knowledge into an easily transmitted form.

The entire world is filled with examples of both good and bad documents. The idea book from Dremel that came with your new Moto-tool is vastly superior to a Handy Andy project construction guide because the Dremel book was well written and the Handy Andy book is just so much kaka. Both books may tell you how to use your tools to create nifty things, but the things you make after reading the Dremel book will be a boatload niftier than the scrap lumber you churn out under the direction of Handy Andy, that pinhead.

Properly constructed technical documents are a joy to behold. Just compare any decent book from Hungry Minds Press or QUE to the stuff published by Prentiss Hall. Yee, hah! So why can't our own documents be this good? Well, as has been said so many times before on this page, it has to be done by somebody who is a professional writer. Don't try to make software developers write good documents and I won't force tech writers to code up your heart rate monitoring system.

This discussion seems to suggest that a lot of problems with documentation are problems with the requirements for the documentation. Example from above: "Documents are usually expected to say 'everything that needs to be said.'" Can you imagine being told to write a book about, say, dog training, that covers "everything that needs to be said?" The resulting documentation will lack focus, and will thus either be so detailed that it will be impossible to digest, or so vague that it will be superfluous. Arguably, a lot of the poor documentation out there comes as a result of poor requirements for that documentation. (This is often the central problem with poor third-party technical books; the author wants to cover "everything that needs to be said," and lacks any coherent direction).

Concur. This is why I wrote technical documents aimed at a particular audience when I created manuals, etc. My engineering documents of various types are aimed at a specific audience, also, but the borderlines are sometimes a bit fuzzy. Nonetheless, good documents are useful to a wide variety of readers. Bad documents are useful to no one. Good documents are focused, concise, and accurate. Bad documents are bad in so many ways I can't even get my hands around them.

Clarification, please. What is your definition of "Good" and "Bad" documentation? Does "Good" imply "Useful?" Does "Not Useful" imply "Bad?"

"Good" and "bad" are, of course, relative terms. What may be okay for you may absolutely suck eggs for me. To repeat for the twenty-twelfth-third time,

DOCUMENTS HAVE TO BE AIMED AT A PARTICULAR AUDIENCE!!!

I hope we were all paying attention that time.

The question of useful versus not useful is a good starting point for comparison, but I think most readers here know the difference between good docs and bad docs the same way they can tell art from pornography -- one look oughta do it.

Again, what is the definition of "good" and "bad" documenation? If it is merely intuitive then there is no need for any further discussion; no valuation can be made for documentation.

How would you like "good" and "bad" to be defined? Do you want some sort of definitive measurement? Some metric that can be applied? If so, we'll need to come up with some metrics for a bunch of other things we discuss on this Wiki, since a lot of the comparisons of "good" and "bad" are just as subjective as this one is.

For developers to communicate to end users, operational code is far more expressive and complete than any textual document. If you want to find out how the program does something, don't read the manual, run it.

It is often more efficient, at least in the short term, to document difficult user interfaces than to make them easy to use. It may not even be possible to make the interface easy to use. One example is the user of man pages and -h options in UNIX.

Assume a company producing a device is required to produce documentation so that an auditor may verify that required processes are followed. ReplaceDocumentation is not an option.

How might said company reconcile HireATechWriter against SpecializationIsForInsects? (Note: that page contains its own debunking.)

-- ChrisFay

At a company or sometimes a project level, one can usually decide what auditable artifacts need to be produced. The auditor should merely verify that the stated items are produced. In practice, however, few (if any) companies know what documentation is necessary, they only know what is common. This results in teams producing lots of documentation that is only ever read by the auditors. A project manager needs to evaluate the documentation his team will be required to produce, determine what value is provided by the document, and allocate funding appropriately. This is where the hire a technical writer versus having developers write documentation comes into play. Unfortunately, most management believes that "Documenation Is Free" when in actuality, the document production effort is the critical path in delivery, not the software development effort.

See: DocumentationPatterns, ReplaceDocumentation, ProblemsWithDocumentation

CategoryDocumentation, CategoryDiscussion

EditText of this page (last edited November 8, 2005) or FindPage with title or text search