Ats Diary Week Three

Part of the AtsDiary.

13 March 2000

Sent out my second AtsStatusReport today. See the link for more details. (Note: The authentication risk was resolved on 14 March.)

14 March 2000

Today, we were supposed to complete our first story. At 144 estimated hours, it was our largest story: add basic authentication to the application. It consisted of a number of big tasks:

Figure out how this company's authentication system works, and integrate with it so people can use their existing passwords.
Create and track user sessions on the client and server sides so we're not constantly transmitting username/password information over the link, or re-authenticating all the time.
Modify all server-side code to check the session before doing anything significant. Modify all client-side code to pass the session through the distribution layer.
Create a login/logout GUI and dialogs for when there's a security problem.

Well, we didn't make our milestone. At best estimate, we're 72% done. Actually, most of our tasks have finished about an hour under their estimate. But authentication was under-estimated significantly, taking 12 ideal hours instead of four. Passing the session information through the distribution layer looks like it was underestimated too, as it's now at 12.5 hours out of 12.

A big part of the overrun appears to be due to setting up AtsUnitTests?. Authentication, in particular, required a lot of difficult database-manipulating tests before it was done. The session-handling modification was hard to get started too. I'm hoping that these overruns are one-time hits and that we won't see them on other tasks.

Our AtsLoadFactor hit 2.9 today (our schedule is based on a LoadFactor of 3.0) and shows every indication that it will continue to drop. We've identified some techniques to better meet our estimates. Since we're only one-third of the way through this iteration, our load factor is going down, and (hopefully!) our overruns were due to one-time hits, I think we'll still deliver all of the scheduled stories by the end of the iteration.

I'm not counting on that, of course. Here's some specific steps we're going to take to meet our schedule:

As a task gets closer to the wire, I'll do less mentoring and more direct coding.
If a test is going to be hard to implement, we'll check if a simpler approach that wasn't quite as complete would be good enough.
When we estimate the tasks for our remaining stories this iteration, we'll use our experience on this first story to create more accurate estimates.

ExtremeProgramming has definitely helped me in this minor crisis. Without ExtremeProgramming's IdealProgrammingTime-based metrics and LoadFactor, I wouldn't have known there was a real problem, nor any specific plans for how to fix it even if I did. As it is, I feel that I've identified the problems early enough and have enough information about them that I can take the appropriate steps. In particular, the IdealProgrammingTime approach has really allowed me to zero in on the tasks that are having problems. In earlier days, I would have been tempted to write off the problems to overhead, assuming that we wouldn't have as much in the future. (This is particularly true since we had a lot of expected but unavoidable overhead right at the start of the iteration.)

As it is now, our overhead is less than estimated, so I know that we have to take stronger steps to meet our estimates and to make better estimates in the future. I feel that I've identified the problems early enough and have enough information about them that I can take the appropriate steps. In particular, the IdealProgrammingTime approach has really allowed me to zero in on the tasks that are having problems. The best part is that the metrics are extremely light-weight and probably don't take more than a minute in a week to track for the entire team. That's because everybody just keeps track of their times on their EngineeringTask cards as they work.

15 March 2000

Passing session info through the distribution layer was definitely under-estimated. We originally estimated 12 ideal hours. We finally finished it with the clock at 19 hours, but that doesn't include the amount of time needed to integrate and get all the tests working again. Other tasks are going fairly well, though, and the AtsLoadFactor is now down to 2.8.

I'm looking forward to finishing this first story and estimating the remaining AtsEngineeringTasks for this iteration. I know we're over schedule for this one story - the biggest one - but I'm still hoping that we'll come in on time overall. Once we get the remaining estimates, I can figure out what our load factor would have to be in order to meet our schedule, and then determine if we're likely to be under that load. If we are, great! If not, I'll have some unpleasant news for the stakeholders. At this point, I'm not sure. It could go either way.

On the positive side, the programmers haven't had nearly as many interruptions recently as before, which is why our AtsLoadFactor keeps dropping. Perhaps not coincidently, the interruptions stopped right around the time I posted "Time lost to other projects: 21 hrs" on the AtsTrackingWhiteboard (two days ago).

Tomorrow, we should finally be done with implementing our first story. There's only two engineering tasks left, and they're wrapping up. But I also have to implement functional tests for the story, which could be a problem. Our deployment script broke down and we haven't had a chance to fix it, so we haven't been running our tests on the distributed version of our app. Thus we could run into problems with distribution. I'll also need to modify the existing tests so that they work okay running distributed as well as local. That could be a huge task, so I might just short-circuit it and only run the functional tests distributed. The problem is that distribution bugs are the most difficult to find and are the ones that really need good test coverage.

Interesting times ahead. I'm not ready to panic, but there is cause for concern.

16 March 2000

The news today isn't so good. We created and estimated AtsEngineeringTasks for the rest of this story's iterations today. Unfortunately, it looks like we're going to be way over the mark. We originally estimated 62 ideal hours for the remaining stories. When we created AtsEngineeringTasks for the stories, the total estimate came to 105 hours. Ouch.

Some of this may be backlash for underestimating tasks on the first AtsUserStory. We estimated them at 72% of the time they actually took. (In fairness, though, most of the estimates were up to an hour higher than their actual times. But two of the seven estimates were very low, and were the reason for our overrun.) I think some of the current estimates are too high, but it's hard to say.

We're going to hold off on reporting any bad news to the stakeholders for a few days so we can calibrate the estimates. If the estimates appear to be correct, then on Monday (two working days from now) I'll spill the beans in my regular AtsStatusReport: We have TooMuchToDo. I'm hoping not to have to make that report, since it will almost certainly damage my credibility.

So, what went wrong? To be honest, I think we just underestimated our initial stories. I based my numbers on how long it took my original team to do things, and they were extremely familiar with the application. This new team is not and needs a lot of guidance. I was aware that the estimates might be a little low, but I bumped up the AtsLoadFactor to make up for it. As it turned out, that bumped-up load factor was a lot closer to the actual load factor than I expected. (Our load factor is currently 2.8; my "bumped-up" estimate for the load factor was 3.0.)

The other thing that has cost us is the AtsUnitTests?. We didn't fully account for how long they would take in the initial estimates. One of the developers has suggested that we significantly cut back on unit testing. His argument is that, since this is a very short project, we won't see much benefit from them, and that the GoldOwners care about OnTimeDelivery? much more than they care about HighQualitySoftware?. Sad to say, I think he may be right about the GoldOwners. My gut feel, though, is that shortchanging the unit tests will cause delays due to more bug hunts, even on a short project like this one. I'm not sure what I'm going to do about the unit tests at this point.

Overall, though, I don't think I can't point to ExtremeProgramming as the cause of our problems. Our problem is caused by one simple thing: poor estimates. The controversial areas of ExtremeProgramming that we're using, such as YouArentGonnaNeedIt and PairProgramming, haven't hurt us at all. There haven't been any design problems and the PairProgramming is absolutely essential because of the amount of mentoring that's required. We're going at a breakneck speed and working as fast as we can... there's just TooMuchToDo.

As I said, I'm not ready to panic yet. But things are looking increasingly bleak. We're going to spend the next two days making sure our estimates are properly calibrated. We'll probably cut down on difficult tests, and I'll take over the keyboard rather than simply mentor when I pair with someone. On Monday, we'll see where we stand.

One thing that probably will hurt us: developer morale is not good. The developers know we're behind and that our current estimates are way over. We've been doing the most difficult stories first in order to offset risk, but in order to try to raise morale, I've bumped a simple yet interesting story up in the queue. We'll do that one next and hopefully the shared success will increase team morale and help the team jell.

Oh, and one more potential minefield: we haven't been testing our code in its distributed state. Our tests aren't set up to run distributed. I did a manual test and I know for a fact that the application currently doesn't work distributed. Resolving problems like this was one of the most painful problems in the first phase of the project, so I consider this problem to be very high risk. I'm not too worried, though, because I've got tomorrow scheduled to resolve the problems with the deployment script, fix the unit tests, and make it run distributed. The good news is that one of our tasks (the one that was scheduled at 12 hours and came in at 21) resulted in complete distribution test coverage. All of our distributed methods are hit. So once I get those tests running over the wire, we'll have a fantastic distribution smoke test. I believe that will completely mitigate the distribution risk.

20 March 2000

Well, our new estimates look like they're right about on target. The estimates currently stand at about 95% of actual. Which is great, except that it meant that our original estimates were definitely too low. So I got to write up a lovely AtsStatusReport explaining the situation (see the page for a copy). Actually, I think the report turned out fairly well - no screaming, wailing, or gnashing of teeth - but I won't know for sure until I hear back from the GoldOwners. Tomorrow morning, I'm guessing.

I also rearranged the AtsTrackingWhiteboard today (see the page for details). With the newly-calibrated estimates, things are looking pretty good. I'm "cheating" a bit by saying that our commitment schedule is based on an AtsLoadFactor of 2.6 (our current LoadFactor), when actually it's still based on 3.0. But that small indiscretion, while going against the ExtremeValue? of communication, gives us a bit of wiggle room for things like AtsSpitAndPolish and doing the release (which has always been a headache, and will probably continue to be a headache despite our attempts at automation).

In other news, the AtsUnitTests? are really starting to pay off. Although they tend not to find any unexpected bugs when we write them, I've personally had old unit tests catch about five problems in new code. In my mind, that's the real value of unit tests; it's relatively easy to write new code that works as planned, since you're thinking pretty hard about the new code. The trick is to not break any old code in the process. The unit tests have definitely saved my bacon in that regard.

The unit tests also made debugging the distribution process a piece of cake when I worked on that Friday. They caught 20-30 bugs in distribution that were due to another developer's misunderstanding of the brokering process. And the bugs they caught only would have occurred when an exception was thrown, which means these bugs almost certainly wouldn't have been caught if it weren't for the tests. I liked unit tests before, but now ImaBeliever?. ;)

On a less positive note, AtsEngineeringTasks have been giving me headaches. It's hard to come up with tasks that don't have interdependencies or that work well with DoTheSimplestThingThatCouldPossiblyWork. I don't think I've got the knack of breaking the stories down into EngineeringTasks yet. In my opinion, this is the part of ExtremeProgramming that's the most difficult; the rest of it may be controversial, but it's fairly easy to do if you want to. Breaking UserStories down into Engineering Tasks, on the other hand, is really hard (for me, anyway) and also very important if you want the other implementation practices to work.

21 March 2000

There's been absolutely no fallout from the estimate changes. The status report email has been received and opened, but there's been no responses. Apparently, we're still well under the original budget, so maybe that's the reason. I'm not going to complain. ;)

Morale is visibly higher now that the pressure is off.