Fully Parallel Programming

I'd like to investigate the matter of why simply subdividing a program and handing it out to multiple solo programmers is or isn't (should be / shouldn't be) more time efficient than PairProgramming. This discussion is started on PairProgrammingCostsBenefits, which ends up in this thought experiment.

"it sounds like you are saying that you can get a job done more quickly if you put more people on it. ...that's not a particularly stunning result (except for how often it manages to not work in real life)."

"pair programming results in 10-15% longer lead times in development for similarly sized teams" [...But only *if* a non-pair team of two is twice as effective as a solo programmer. Your mileage may vary.]

"Program A needs to be demoed next Friday. If software were perfectly divisible, then two people could each be given half of Program A to develop, achieving maximum parallelism, and so they should be given separate parts of Program A. Repeat this subdivision, and you get the strategy that any number of people could be applied to make any project arbitrarily fast. This is the "more the merrier" strategy... However, software is not perfectly divisible. At some point the interfaces between the pieces starts costing time."

"Suppose a task took a single person 40 hours. A pair-programming team might complete the task in 23 hours (46 person-hours). How long would it take an ordinary non-pair team of two people to complete the task? I would guess 25-30 hours (50-60 person-hours), depending on the task divisibility."

: [This is slightly out-of-context--see below.] Putting on my PointyHairedBoss hat, this would mean that assigning two programmers as a non-pair would result in faster delivery. A 40-hour solo project would be doable by two people in 20 hours. Of course this fails badly for larger numbers of programmers, but it seems more reasonable for two people.

Is the PointyHairedBoss right? Why (not)?

The pointy haired boss is right that the actual cutting code part takes a little bit longer. But cutting code is not half of what we do. Quality and delivery are the other half, and pair programming has very strong benefits for that other half. See: http://members.aol.com/humansandt/papers/pairprogrammingcostbene/pairprogrammingcostbene.htm

Something about the old "make a baby in one month by impregnating nine women"?

Yes, but PhilGoodwin and the CliffordAdams in PairProgrammingCostsBenefits point out that 15% longer is 15% longer. Why can't we just take every program, split it into its maximally parallel pieces and hand out to solo programmers, and reap the 15% time-to-market benefit, and cover the defect repair costs under the market share gains?

: Because to do that, you'd have to have precise and granular enough specifications to allow these folks to work independently. XP/pairing doesn't assume that--the mixing allows the specification to evolve from a framework. So, the question implicitly compares apples and oranges.

In some cases you may be able to do that. You may even be able to force your customers to pay you more for having shipped them buggy software, by calling bug fixes "upgrades" and charging for them, and by charging for support calls.

This mostly works for external customers ("shrink wrap") who have few choices. External customers who have choices will go elsewhere. [I believe not. WorseIsBetter and check the history of the software industry over the last 50 years.] Internal customers, for whom the majority of all software is written, usually don't get charged for calling the help desk. You may be able to charge them for the bug fixes, though.

You are going to have to charge a lot to cover the increased defect rate, perhaps more than you'll get from the 15% time to market. Once you've shipped, the cost to fix bugs does increase. [You mean, things will run the way they run now...]

Finally, there is the risk of harming your good will with the customer. If they have a choice, they may leave. If they don't have a choice now , they'll leave as soon as they do get a choice. [Don't forget, they competitor lost the time window opportunity, and may be out of business.]

There are several forces involved in the decision to use PairProgramming:

Is it useful to deliver the software sooner than a single person could write it? (If delivering early is not useful, then solo programming will probably create more code for a given number of person-hours.)
Is the project easily partitioned into separate tasks? Is the interface between sub-projects simple and easily understood?
Are the programmers willing to work as pairs? If working separately, will they communicate effectively?
How important is the quality of the code? Are bugs major disasters or minor annoyances?

Some of the replies are missing the point of my PointyHairedBoss example. The PHB response was in reply to the assertion that programmers don't collaborate, so the solo-programming results are applicable. The PHB then assumes that a group-of-two will produce the same results as an individual, but in half the time. (Hopefully the MythicalManMonth would correct this delusion.)

I think the solo results are only applicable for easily separated subprojects that require almost no collaboration. Even a moderate interdependency may slow down a "group of two" enough that a pair will be faster. Other benefits of PairProgramming like better code, faster detection of integration problems, and fewer people risks would just be a bonus.

Perhaps most programs or subprojects are written by a single programmer simply because of the problems of conventional group programming. If (non-pair) group programming doesn't help much, and it sometimes fails spectacularly, it may be less risky to do solo projects. PairProgramming could allow more companies to effectively use more programmers for early delivery. --CliffordAdams

It's interesting that there seems to be 2 kinds of parallelism here. One kind is dependent on the code to hand, and one isn't.

The Program A example is mostly about the first kind. It is about whether we are lucky enough to have a problem which divides neatly into two independent, equally sized halves which can be given to separate teams (of whatever size).

PairProgramming is mostly about the second kind. It is about, eg, how thinking interacts with typing. All programming jobs need you to think and type, and individual programmers have limited ability to do both at the same time. So all programming tasks get at least some benefit from PairProgramming. This helps offset the cost of having half as many teams.

I think division of each story into "real code" and "unit tests" may serve to create more opportunities for this second kind of parallelism. These are subtasks that interact a lot, so can't efficiently be split between 2 independent programmers, but they give the thinking programmer something to think about when the typing programmer is typing. Hence PairProgramming will suffer unless it includes an XP-like approach to unit tests.

I hope productivity studies allow for this. It might be useful to identify all such tightly coupled yet parallisable subtasks, to help convince PointyHairedBoss. Can we structure our development process in ways analogous to unit-test-first, to expose more? -- DaveHarris

When DoIt was first adopting PairProgramming, CarlParziale made the observation that "The problem on our projects has never been that we didn't have enough developers typing in enough lines of code. The problem has been that we have millions of lines of code that don't do what we want." As JamesCollins summarized, "Programming is not, at its heart, a typing problem."

Once you admit that programming is an exercise in problem solving, pairing makes lots of sense. -- BillBarnett

Another factor in considering PairProgramming is that time estimates will have some error range, and this error can have a greater effect on non-pair teams. I hope the following scenarios will clarify this idea. --CliffordAdams

Background:

A company wants a client-server system built in less than 30 days. It is estimated to take 200 hours for a solo programmer, which is too close for comfort. The system splits easily into separate client and server halves, and each is believed to have the same difficulty. The company has plenty of spare programmers, so they decide to experiment with PairProgramming.

The first group is a non-pair team of Alice and Bob. Alice writes the client, Bob writes the server. Estimated delivery in 100 business hours (200 people hours), presuming the same effectiveness as a solo programmer.

The second group is a PairProgramming team of Charles and Debbie. Charles and Debbie work together for both the client and the server. Estimated delivery in 115 business hours (230 people hours), using a 15% estimated slowdown in people-hours vs. a solo programmer.

First Scenario:

Suppose the client was easier than expected, saving 10 people-hours of programming, and the server was more difficult, requiring 10 extra hours. (This could also happen if the two sides are not split evenly.)

In the non-pair group, Alice finishes the client at 90 hours. Bob is still working on the server at this time, and probably knows he isn't going to finish in the remaining 10 hours scheduled. Bob "doesn't have time" to train Alice on the server architecture (or Alice tries to help but is not effective). Bob finally delivers at 110 hours. There are no integration problems, so the project is delivered at 110 hours (10 hours after scheduled).

In the pair group, Charles and Debbie save 10 people-hours of work on the client half, but spend 10 extra people-hours on the server. The pair doesn't have integration problems, and delivery is at 115 hours as scheduled.

When the project is finished, Alice is the client expert, and Bob is the server expert. Charles and Debbie each have knowledge of both client and server.

Second Scenario:

Unexpected problems with the server require significantly more work for the server half (50 people-hours). The client required exactly 100 people-hours.

The non-pair group may be in trouble. After Alice finishes the client, she may or may not be able to help with the server. The results could be any of the following:

Best case: Alice instantly understands the server code, and the remaining work splits perfectly evenly. Alice contributes 25 fully effective people-hours. The project is delivered at 125 hours, 25 hours later than scheduled.
Intermediate case: Alice finds a 15-hour part of the server code which can be written separately. Bob spends 5 hours training Alice on the server code, and answering Alice's questions. Bob also writes 35 hours of the remaining server code. Delivery is at 140 hours, 40 hours later than scheduled.
Bad case: Alice cannot contribute more time to the project. (She may have been called away to another project.) Bob writes all the server code. Delivery is at 150 hours, 50 later than scheduled.
Worst case: "Adding people to a late project makes it later." Alice and Bob each go to 5 extra hours of status meetings and damage control. Bob spends 15 more hours training Alice on the server and answering questions. Alice attempts to implement a large portion of the server, but her code does not work. (Perhaps Bob forgot to tell her about an important assumption.) Bob spends 5 more hours trying to fix Alice's code, before giving up. Bob finally implements all 50 extra hours of the server. Delivery is at 175 hours, 75 later than scheduled.

The pair group (Charles and Debbie) simply keep working together on the project, both contributing to the client and the server. They deliver in about 144 hours ((250 * 1.15) / 2), which is 29 hours later than scheduled. This is 4 hours later than the "intermediate" case, but the intermediate case has 11 more unscheduled hours.

Any other scenarios? Some real-life failures would be interesting.

In my college "Software Construction" class, the project was doomed when a critical person dropped out late and their code was unusable. Nobody could write the missing code unless they ignored their assigned part, and it was too late to reassign projects. (Time was already tight since we had spent half the class writing a huge feature/wishlist document.) In the end, only the Help and Exit functions of the application worked. (A successful compile of the whole application was a major victory in our integration phase!)

My code worked, but neither my callers nor the functions I called were implemented. I had to write testing programs for each side to convince people my code was OK. At the time I thought "my code works, so it isn't my fault", but I'm less certain now. --CliffordAdams

Also see BrooksLaw and AmdahlsLaw.