Personal Software Process

See:

Introduction to the Personal Software Process (SEI Series in Software Engineering) by WattsHumphrey ISBN 0201548097 (for non-developers)

A Discipline for Software Engineering by WattsHumphrey ISBN 0201546108 (for developers)

http://www.sei.cmu.edu/tsp/psp.html

http://www.sei.cmu.edu/tsp/building-teams.html

See also:

ProgrammersNotebook, which is one of the development practices WattsHumphrey recommends
TeamSoftwareProcess, also created by WattsHumphrey
CollaborativeSoftwareProcess, created by LaurieWilliams - extends PSP with PairProgramming
AgileAndTspDiscussion, a general overview of blending agile approaches with PSP and TSP

'A Discipline for Software Engineering' is a daunting book. At least, that was my feeling when I first saw it. Having worked my way through it during a PSP course, let me give you all a short summary of the book.

The book describes several things:

A general framework for software development processes: the importance of a process, metrics, how to plan.
An 'example' development process, which is essentially the waterfall model: separate phases for planning, design, design review, code, code review, compile, test, post mortem (= results analysis). This process is used in all explanations and process scripts, and the PSP student is supposed to use it during the exercises.
A number of planning practices (create conceptual design, count lines of code, use historical data and statistics)
A number of development practices (logical / functional / operational / state templates, verification techniques, coding standard, design and code reviews).
A lot of mathematical formulas and algorithms which are needed to do the statistics.
A collection of 20 programming exercises that should be performed using the PSP. (These exercises are all about implementing the statistical formulas and algorithms needed for the process.)

I'd like to say some more on the process here. As I said, WattsHumphrey's process is essentially a waterfall process. For larger projects, the advice is to divide it into 100-line parts, and implement each part using a design...test cycle.

Although WattsHumphrey emphasizes time and time again that it is important that you adapt the process so that it works for you (the Personal Software Process), my experience is that the book and also the course block that. All process scripts, forms, and explanations assume that you use this process. Also, the spreadsheet that the SEI uses for their course is tightly coupled to this process. This means that the student is not given much freedom in changing the process around.

As a final remark, judging from the development practices, and from his programming examples, it seems to me that WattsHumphrey does not have much programming experience. For an example, read section 12.5, 'Program Tracing', and then compare this to coding 'clearspaces' yourself and testing it using a few UnitTests. I must say that I trust the advice from people with a lot of development experience (such as the extremists) a lot more than WattsHumphrey's.

AuthorsDontCode?

Of course WattsHumphrey says a lot of valid things about the need for a process and metrics, but I found out that for me at least, the ExtremeProcess works better than his and - very importantly - it is more fun.

See ExtremePspExperience for an experiment in blending PSP with ExtremeProgramming.

-- MarnixKlooster

There was a great study/article in the Communications of the ACM once (I wish I had the reference) that showed that programmers spent more time analyzing and crunching statistics with the PersonalSoftwareProcess than producing software.

-- sg

I wonder which activities qualify as "producing software".

I think it's pretty clear that they spent all their time analyzing their stupid statistics and not designing and writing code, which is what the whole point is. Even the ACM found the PSP (well you fill in the words). Please read the article by searching the ACM's site.

-- sg

Don't mix three unrelated concepts. Competent handling of feedback on a process you care about can include statistical analysis, especially if the effects of interest are subtle. At the same time, taking one technique from an overall competence in one context and over-applying it in a different context is a common way to undermine effectiveness. PSP is first a training course, and second a set of habits you converge on for doing your best work in the given context. Learning behaviors are not production behaviors.

I think that "sg" might be referring to Beyond the Personal Software Process: metrics collection and analysis for the differently disciplined by Johnson et al., currently at http://portal.acm.org/citation.cfm?id=776907 but behind a paywall. That's an IEEE CS proceedings paper, though. I found no CACM article. -- AnonymousDonor 2011jan21

I will be Back to add more detail later. For now, let me say I been to PSP land and done the process. Yes it's restrictive. Yes you cannot really tailor it to your process. Beware, I have looked at the statistics and I do not like the methodology at all. I do not trust the answers that the stats give. The stats as described measure the wrong things. -- AlanChristiansen

Just wondering whether you learned nothing from the experiment, though. For instance, do you know what "your process" is, and did PSP help you discern its boundaries? For another instance, what are the "right things" to measure, and did you maybe figure out some of that while you were struggling with PSP? Is there some reason why you can't measure the right things for you, or were you already doing that before PSP? -- WaldenMathews (never done it)

I learn things all the time so it is hard to say whether or not I learnt anything about programming as a result of the process. If the choice is between programming without thinking about how you do it and doing PSP sometimes, I advocate PSP as less worse, if you manage to remember not to believe the stats.

PSP measures all errors and simply counts them. There are no matched pairs, no control groups, no randomizing of the order in which experimental conditions are applied. It is simply not an experiment and the stats thus measure nothing. The text asserts that because the total number of errors goes down things are better. Yes, that happened; my errors went down when I did it too. I stopped making the simple to find, easy to fix errors, and started making a few serious, hard-to-fix ones.

The distraction of attempting to perform PSP interrupted my normal programming flow. All my mental alarms kept going off, and saying bug alert, because PSP had distracted me, and I know from conditioning that being interrupted results in bugs.

I have a much better (for me) reflective process. Whenever I find a hard to fix/find bug, try and work out how it happened. Current number one cause, I got interrupted. Current number two cause, cut and paste. Current number two[sic] cause, I am mortal.

PSP exists to make XP look better. :) -- WyattGreene

I bought Introduction to the Personal Software Process on some sort of theory that it might actually be a worthwhile thing to learn, but so far (I've read Chapters 1-8), I've been unimpressed. The chapter on estimating project sizes, in particular, seems pretty bad. As an experienced programmer, I find the notion that I can estimate the size of a program by counting the number of while loops and compute statements in it insulting and useless.

On the other hand, section 7.8, "Prioritizing your time", was very informative. He says, "The discretionary activities are...: eating, sleeping, ...." Dang, I'll be so much more productive if I quit spending all that time sleeping!

Anyone know of a school in the PortlandOregon area that uses the PSP? I'd love to dump this loser on some poor student who's going to have to actually live this. -- BillTrost

Bill, learning PSP just by reading this book would definitely be a challenge - it's meant to be a textbook for a hands-on course taught by a PspInstructor?(either in an academic setting or a workshop setting), not a self-study guide. Whether counting the statements in your program is a useful proxy for estimation or not is, quite simply, a hypothesis which may or may not work in all cases or for all people. The point of the PSP course is to learn how to find a proxy that does work for estimating your effort. That's why writing the programs, and collecting and analyzing your own data, is a mandatory part of learning PSP. -- KarenSmiley

For a while, I was quite impressed by the rationality and logic of the PSP. The idea is to track and measure your performance in order to better predict and optimize your future performance. Just as world-class athletes record their best times, world-class programmers should do the same. The benefits of the PSP are tremendous: you can get more work done, with fewer errors, and more accurate schedules. The recommended practices of the PSP are all buttressed by clear empirical evidence, in contrast to the anecdotes and buzzwords that drive the development practices of many programmers.

Yet, the emotional cost of the PSP is the one that is probably hardest for the average programmer to swallow. If you follow the PSP, you follow it in addition to any other process that your project is following. The drive to continue to use and improve the PSP must come entirely from you. If you are in a programming environment where programmers do whatever they please as long as they get results, then it will take a lot of self-discipline to spend the extra hours you'll need to spend every week to fill out your forms and tabulate data. The average programmer will probably wonder why they are filling out all these forms, while their colleagues - who do more programming and less paperwork - seemingly do good work and maintain reasonable productivity.

What the PSP assumes is that programmers are driven mainly by the desire to complete high-quality final products in a timely fashion. At almost any cost. The PSP insists that programmers must shape themselves to fit the process, not the other way around (as in XP).

The nail in the coffin of the PSP is that I don't know of anyone who uses it. I've heard from many people who have tried to use it, but they almost invariably say the personal cost is too high - too much paperwork, too much overhead. Without an active user community, no process as ambitious as the PSP can really take off.

The drive to continue to use and improve the PSP must come entirely from you.

Without an active user community, no process as ambitious as the PSP can really take off.

PSP is definitely a HighDisciplineMethodology that requires coaching and support to really be successful in the long term. That's exactly why the TeamSoftwareProcess and the role of TspLaunchCoach were created. There are also newer, lower-overhead tools nowadays. I personally think that combining TSP with PairProgramming would be a good way to lower the impact of data gathering on sustaining MentalStateCalledFlow ... that is a theory I am hoping to be able to test out in the near future. See AgileAndTspDiscussion and the research by LaurieWilliams on the CollaborativeSoftwareProcess.

-- KarenSmiley, 8/14/2003

The PSP insists that programmers must shape themselves to fit the process, not the other way around (as in XP).

Although WattsHumphrey emphasizes time and time again that it is important that you adapt the process so that it works for you (the Personal Software Process), my experience is that the book and also the course block that. All process scripts, forms, and explanations assume that you use this process. Also, the spreadsheet that the SEI uses for their course is tightly coupled to this process. This means that the student is not given much freedom in changing the process around.

True - but as mentioned in MartialArtsAsSoftwareDevelopmentMetaphor, it's pretty common when you are first learning a thing to exaggerate it, and to perhaps even over-do the discipline a bit. Learning PSP is rather like most other endeavors ... tailoring the techniques works better after you've become proficient in the basics. Real-life experience with PSP as part of a TSP (TeamSoftwareProcess) project team, using the SEI's TSP spreadsheet or another TSP tool, is a better indicator of its flexibility and how PSP can be 'personalized'.

-- KarenSmiley, 8/14/2003

I also think it's clear that the specific scripts described in the book are starting points. At this time, I not familiar with the whole process (OK, I've only done Program 1A), to figure out how to define scripts that look more like the "quicker" cycle of XP over the waterfall cycle. At the very least I'm able to show that (at work) I've spent 4.5 hours programming over the course of three days. That's helpful in getting some of my extraneous activities pushed onto someone else. ;) --TerryLorber

planning practices ([...] count lines of code, [...])

Does this methodology provide a way to travel through time so that you can count the lines of code that haven't been written yet? Or does it advocate not beginning planning until after the project is complete?

Neither. Rather, you perform planning to record how you anticipate spending your time for the current cycle (think iteration) or task, and you perform design to basically arrive at a set of engineering tasks for the current cycle/task. These are both acknowledged in the text to be, basically, wrong up front. While the goal is to improve your skills in these elements of the process, it's never said you must be perfect, and the expectation is that you'll fluctuate around a mean (average) level of proficiency. Please note that this isn't much different than what happens, in practice, with XP or other agile processes, only without the anticipatory record keeping.

CategoryBook CategoryMetrics CategoryStatistics