Programmers Notebook

A LogBook specialization.

"Just pretend that you've got a microphone... and that microphone is always on."

Alternatively, you can turn the knobs up to 11 (GoesToEleven) and actually use a microphone, as suggested in RecordProgrammingSessions.

BestPractices

Start slowly; some log is better than no log. (AnyXisBetterThanNone)
Don't postpone writing something down. Somebody else will come along, and you'll forget your bright idea.
Put a date on every entry.
Number the pages so that you can easily reference them.
Use a consistent format so that you can easily go back and find something.
Use a horizontal separator line and a project name to separate individual subjects (e.g., music, theorem proving, or language research).
Use secondary headings that include the date and the subject (for example: "980425, Ideas on extending Java with laziness").
Different kinds of entries, each with a special symbol in the margin: remarks, questions, definitions, to do items, MetaRemarks.
After every entry, leave at least a single empty line. This can be used for last-minute additions, the answer to a question, or a forward reference to more thoughts on the same subject.
Make it readable for yourself and others.
Log meetings with dates, attendees, topics and results.
Keep a separate ToDoList, or put it in a special place. Do not scatter ToDoItems? throughout your notebook.
LetYourLogsBecomeYourPlans

Disadvantages

One disadvantage of putting to do items in the notebook is that they are scattered, and also the priorities are not clear. I intend to try out a ToDoList on a separate piece of paper, and refresh that list weekly or bi-weekly or so.

An alternative would be a spider that finds <span class="todo"> and @todo items in files of the appropriate type. I've been tempted to write such a thing for some time, but it seems to fall into the YouAintGonnaNeedIt category. At least, I don't seem to be missing it.

Beware the PushDownGoalStack
DontLoseGoodIdeas (a "YouArentGonnaNeedIt anyway" warning)

Instantiations of

The best practices list above is clearly talking about a bound or spiral-bound book. See IsAnythingBetterThanPaper.
A text file in EmacsOrgMode (was: EmacsOutlineMode) has served me well. I could witter elsewhere.
Discussion of wiki moved to WikiAsProgrammersNotebook.

There are more below.

How to use

Starting to keep a ProgrammersNotebook was the one of the two best changes to my PersonalSoftwareProcess I made over the last year. (The other was to use contracts, as explained in DesignByContract.) If there is something I think of but cannot use right away, something I have to do, or a nice quotation, it goes in the notebook right away. It is also very useful for taking notes at a meeting. Sometimes, I also paste in a design or code sketch that I wrote down on a loose piece of paper.

I currently use a folio-sized notebook (slightly larger than A4). This is big, but it gives me the freedom to sketch designs etc. that would not - or barely - fit in an A5 booklet. But this is personal preference.

At the end of the notebook I keep a time log, so that I can always find out what I was doing when. But this is only a secondary use.

I learned about a programmers' (or engineering) notebook from a colleague. WattsHumphrey's PersonalSoftwareProcess book also mentioned it, and that made me decide to try it. -- MarnixKlooster

I conditioned myself to initiate logging in the moment normally wasted debating "Shall I write something in my scribbler?" (This is a variation on my personal philosophy of "Don't hesitate to tell your wife you love her.") This sometimes took the form of a near-dooble, but over time my entries would break down into 3 basic types; sketches, prose blocks, and point notes (including calendar entries) -- BenTremblay

Some comparisons with what Marnix does:

I'm nowhere near as orderly as Humphrey would suggest, or as Marnix is. My notebooks are for fixing things in my mind, communicating with a quick drawing once in a while, and for having materials to refer to later. Most references back are to the same notebook; fairly often one book back; very rarely further.

I use spiral-bound 8.5x11 notebooks, 5x5 quadrille-ruled. I've got them going back well over ten years. Sometimes, I even look back at them to find something.

At the front of the notebook, I paste in one-page month-at-a-glance calendars for the few months I expect the notebook to last. Write my appointments and hours right on there.

A trick I use for the ToDoList: I put an array of those little yellow sticky notes on a page near the front. I write ToDoList items on them, and remove each when it's done.

For all kinds of specialized pages, try printing a little form and sticking it into the book.

I start a new page for almost everything, although sometimes I can't resist reusing a page with only a couple of lines. For formal meetings and such, I write the date, meeting name, attendees at the top. Otherwise, I don't use much standard formatting. My notebooks are just for me, so I don't worry about format as much. I do use special symbols to mark action items and questions, and I use multiple colors of ink for direct meeting notes and meta-notes written then or at another time.

-- RonJeffries

I do this also. I have smaller notebooks: 9.5"x6." I use them for blurbs of at most five lines. It is a great way to organizing writing ideas. Periodically, I make new blurbs that reference previous blurbs to consider whether certain ideas will work together. For programming or just things in general, I use to-do lists in Notepad and splatter them across my Windows NT desktop.

For finely grained work, I'll often liberally spread my code with one-line comments prefaced by "MF" to bookmark things to revisit, often optimizations. I DoTheSimplestThingThatCouldPossiblyWork (often brute force) and then refactor as things come to life (in C++ no less). When I get rid of all the "MFs", I know I've done all that I set out to do.

Invariably, ideas come to me when I don't have my notebook, so at the end of the day I pull little scraps of paper from my pockets. I'm thinking about getting a PalmPilot, but not seriously. I know next to nothing about them and it is hard to believe that anything is better than paper.

-- MichaelFeathers

IsAnythingBetterThanPaper?

I am deeply convinced that not using paper, i.e., an ElectronicLogBook is an AntiPattern, or at least a GreyPattern. -- GeraldoXexeo

I also believe that you should use bound notebooks. Using any kind of loose paper is also an AntiPattern, because you can lose order and pages. -- GeraldoXexeo

I am never without my Day-Timer (R). (a brand of paper pocket calendar.)

With the "two-pages-per-day" format, I have a whole page to write things that would normally be on random scraps of paper.

Typical entries: Start/stop times for billable hours (a religion with me ;-); counts of things I just did -- for status reporting; names, phone numbers and web addresses people mention; quick organizing notes for routine meetings; etc.

One big thing I do with my 8 1/2 by 11 inch spiral bound (read: cheap!) notebook, is to draw a line from each "ToDo" item to the right margin, where I draw a box. When I do it, I check off the box.

-- JeffGrigg

I tend to use an ASCII [EmacsOutlineMode file] rather than a paper book, but the concept is the same. I rarely remove from the file, and usually append rather than editing. See also DebugByDescribing.

-- MartinPool

For dissociated, wish-I-had-time-to-do-that ideas, I write my thoughts on a Post-It note and stick it on my office wall. It amuses me to look up there and see them all while I'm thinking about something else, or want to get some inspiration for a current problem. Right now my amoeba-like Post-It note organism is about 1 square meter and slowly growing around my desk.

-- RusHeywood

I've been using IndexCards lately. They're hard to leaf through but very easy to reorganize as new information is added. I collect them into little decks by subject and put rubber bands around them. I don't use them to keep track of time, except when I have a bunch of immediate ToDos?. Then, I put them one per card, put the cards in my pocket and periodically throw away the ones that are already done. I do this for open items between meetings too.

-- PhilGoodwin

I too do this a lot now. -- LaurentBossavit

I like using IndexCards (one of the many paper-based technologies) as a ToDoList because, as you say, they're easy to reorganize. Or, rather, you aren't forced to organize them when you're writing them.

With competing paper-based technologies, like the bound notebook or loose sheet of paper, you're required to impose an order as you write items down. But when you write items on IndexCards and throw them on a pile (or put them in your back pocket, if you're using the IndexCards release for the 3"x5" platform, upon which I've standardized), you're not required to look at them in the entry order the next time. Thus you can defer ordering and grouping. This makes it much easier to focus on content now, and worry about priority and categories later. Not to mention time estimates and risk and feasibility and all that good stuff. (This separation into a "loose flowing" phase (of entering items) and a later "refactoring" or "editing" phase is also characteristic of the WritingWithoutTeachers or FowlerWritingMethod writing process, and of the "make it run, make it right, make it fast" programming concept, where you make things work first, and then refactor them.)

This point about ToDoList technologies doesn't necessarily apply to maintaining a ProgrammersNotebook, however. Perhaps a bound paper book is better for that.

-- ApoorvaMuralidhara

Not that paper doesn't have its place, but...

When I started running wikis, I got hold of Apache32 for NT and ActivePerl, and set up a local server on my home NT system (notebook) that could serve a wiki. The thought was that it would form a decent "notebook" for random notes, with built-in searching and easy updates. Works fine. I mean these days one almost always has a browser running, and now with ActiveDesktop, any folder window is a browser.

The neat feature from my point of view is that my local notebook is just an ftp away from being accessed from any location or machine if I place it on one of my web sites. I especially appreciate the hyperlinking between pages, from pages to files, from pages to Internet URLs... Not quite seamless, but close.

-- BoLeuf

I too have found a personal wiki valuable for project notes. I've found that when recording thoughts on paper, I often get bogged down in the details, sometimes losing thoughts forever in the effort to fit them into the linear notebook medium. In wiki, details are delayed by encoding them as AbstractLinks?. Usually the link name can be descriptive enough to record the flavor of my thoughts at the time. I've found myself drilling down on subsequent passes and filling in levels of detail in a way that would have been much more difficult (or impossible) on paper. -- TimVoght

That is a cool idea. I wish it were feasible for me to try. What I did was take a guestbook perl script, and change the text of it to indicate ideabook. I then placed the whole thing in a password protected section of my web site (CGI access, but no real database support). Now, I can get to it from home, work, the labs at school, walking down a hallway at school (they have terminals scattered around for users to check email and web with), and other places. I have a script on my home linux machine that back the files up to my home computer nightly (that script also backs up other dynamic areas of my site, as well as doing mail processing and downloading commonly read news groups). I'm working on a procmail script that will take emails to a certain account and post them to my ideabook form. -- JoshuaBoyd?

Followed up in GroupWare.

LionKimbro wrote a book on a notebook system for tracking all your thoughts and making a map of your brain.

http://www.speakeasy.org/~lion/nb/html/

I'm trying it nowadays, it's pretty interesting. He said he'd be making a wiki about it shortly, I think there could be a lot of cross-linking between it and these pages. -- EmileKroeger

Here's a trick for keeping two books in one (say, to separate musing and personal observations from operational notes and lists): flip the spiral notebook vertically, turn it around, open it up, and start the second "book" on the upside-down pages. When your two books meet in the middle, start a new notebook.

It's one of those tricks that is obvious after you've seen it once. -- DaveSmith

I have used that trick. I learned it in high school when a friend showed me a paperback book of jokes that were insulting to a certain ethnic group. When flipped over, it was a book of jokes insulting to another ethnic group.

Here's one way to do a ToDoList in your notebook. Reserve the second page or so, blank, for the list. Pave the page with blank yellow sticky-notes, the small 1.5x2" size. Write todos on the notes. Remove them when done. Rearrange to reflect priorities.

I use the second page because having the first page on top of it keeps the notes from getting deranged as you open and close the book.

NextList is more useful than normal ToDoList. I use this method with yellow sticky notes. But some question: Can ProgrammersNotebook work for TimeManagement? I think it's difficult...

For those of you who use emacs a lot, there is a mode for keeping a TimeLog? by JohnWiegley at http://www.gci-net.com/users/j/johnw/EmacsResources.html. There is also a mode below that URL to schedule tasks. I print the TimeLog? and glue it into my ProgrammersNotebook.

-- MarkoSchuetz?

Is there anything Emacs can't do?

No.

Yes! BootStrap. The moment Stallman realizes this, EmacsOS will be born, and Linux will go down. -- JuanPabloNunnezRojas

You give me a coffee machine with a networkable brewing control interface, I'll make Emacs control it :-) -- PaulHudson

Well, it seems that someone isn't keeping up on his RFCs! ;-> I suggest you take a look at ftp://ftp.nordu.net/rfc/rfc2324.txt, where you will find the Hyper Text Coffee Pot Control Protocol (HTCPCP/1.0). Good luck with the CoffeEmacs? project! -- IsidorRuderfer? (note: rfc2324.txt is dated April 1, 1998. That should indicate the seriousness of this tongue-in-cheek emacs application)

As with most good ideas, someone has already done it. http://www.chez.com/emarsden/downloads/

Emacs can't be vi enough (ie effective and exact enough) for a vi person. :)

I use my Handspring Visor for this. I have a separate paper notebook that I use for mostly visual thoughts that I wouldn't be able to grep anyway. It's a small sketch book that I picked up cheap somewhere. I used to worry about categorizing stuff a lot, but now I use a few ad hoc categories and rely on the visor's find to track down my ideas.

-- LoganCox

Ideas and the organization of ideas and information have been an ongoing pursuit of mine for at least 20 years. Almost all of the suggestions and approaches mentioned above have been and continue to be utilized in my PersonalInformationSystem through a technique I call PaperPorting.

-- DonaldNoyes

Every few hours I write down (with a hand-held analog graphics tool on non-virtual hardcopy) what I'm currently working on.

Seems like a BestPractice to me, but some here might not require it of their minions, or might require more than just a text description. (The e-mail and version control logs tell all, you know.) -- PhlIp

It's good. I wish I had written down everything that happened on C3 [ChryslerComprehensiveCompensation]. I would be able to fill the Internet with stuff I learned and how I learned it. Instead of just Wiki. But what's email? -- RonJeffries

After reading this comment several times, I think I now understand why you ask, "What's E-mail?" You're pointing out that on an ExtremeProgramming project (like ChryslerComprehensiveCompensation), all communication is face-to-face (which I agree is a good thing), so there is no "E-mail log". (Although there is a version control log.) Am I right? The comment probably wouldn't make sense to someone looking at this page who didn't know much about ExtremeProgramming. -- ApoorvaMuralidhara

"I use my Handspring Visor for this. ..." -- LoganCox

Funny. My friend added an RFC 2324 implementation to his and uses it to run his coffeemaker (which runs Emacs, of course, even though EdIsTheStandardEditor?). -- JoeOsborn

There is a new piece of software from Microsoft called OneNote that might prove useful as a ProgrammersNotebook. It has many of the requisite features: include text and images, links between pages, runs on PDAs, etc.

http://office.microsoft.com/onenote/

Except I just realized the trial version is 80 Mb. So maybe this isn't an ideal solution... Looks like a neat piece of software, though.

I am just half-way through the trial period using this software, and I am convinced that it will soon belong to my toolset. It is versatile, comprehensive and easy to use. Its use is limited only by the user's imagination and curiosity. I have already included it in my FirstStep? and SuccessOriented? schemas. While it is useful in capturing pages from the internet, it is not limited to that. It has two ways you can capture, each of which having strengths and weaknesses, but when taken together, will cover most capture and recover efforts. The first is like a camera, taking a picture of what you have included in a user-defined rectangle, the second being a capture of the complete document. The second method captures not only text, but links and images as well. It has an additional feature I haven't used much where you can capture and store sound from your computer sound system and store in a page as a link. There are a host of other features I haven't touched yet, including handwriting recognition, etc. You'll have to try it to find out for yourself. No cost for trial, which is for an adequate period to allow you to decide whether you want to purchase it or not. I will probably do so near the end of the trial period. -- DonaldNoyes

I've maintained an all-electronic ProgrammersNotebook since November, 1986, and have never needed paper. Of course, I'm what you might call a WordPerson rather than a PicturePerson, so all-electronic works better for me. Can you imagine trying to search thousands of paper pages without an index? - BillZimmerly, Trees love me. :)

Who says you're without an index? Just make one. Fill it as a part of the review that you (hopefully) do on your notes.

The process of using a LogBook for just about everything was drummed into me at college back in the 70s. My prof regularly stated that unless an observation was written down, it never happened. With a few lapses, I have stuck to this mantra since then and now have a wonderful repository of personal history to look back on.

For years now (25 plus) I have followed basically the same method of logging.

I use A4 case bound ruled notebooks (occasionally spiral bound if I can't get case bound). I reserve the first 5 pages as an index and then number all other sheets (i.e. two pages on a single sheet of paper) sequentially (if not already done so). I start each topic/project/client/whatever on the right hand page of a new sheet. I then continue to use that sheet for that topic until the sheet (double page) is full. I then find the next empty sheet and continue. To link the sheets together, the right hand page of the sheet has the number of the previous sheet relating to that topic (or '-' if it is the first in the topic), the topic name and the number of the next sheet (filled in when a new one is started). This sounds familiar. Wasn't at least one early file system implemented this way, as a bunch of linked lists of blocks in a BAM of some sort?

The index sheets contain a sequential contents list of the topics, plus a list of sheets containing the narrative of that topic.

One variant that I used when I worked in Germany was to use A4 duplicate books. This allowed me to get the best of two otherwise competing access methods: a NarrativeStyle? and a topic specific style. In addition to the logbook, I also had TopicFolders? in my FilingCabinet?. As each page was completed, the carbon copy was removed and stored in the topic's folder, leaving the master in the LogBook. By this means I had the complete narrative of a topic, together with any other materials, in one place. Unfortunately I can't get the A4 duplicate books in the UK, so I gave up on this approach.

I have toyed with the notion of using an ElectronicLogBook many times, but I always come back to paper in the end.

-- GarethHowell

I highly recommend NoteLens (google on it) for this. It is missing some minor usability improvements (like remembering grid sizing in registry), but satisfies almost all your needs in a simple way. It has great searching and categorization and does not get in the way of you, nearly as fast as notepad.

I used to use a PersonalWiki for this but switched to a personal PasteBin