Three Week Project Turnaround

(A shorter description: http://www.charliepoole.org/article.php?p=turnaround)

In July, 2001 I spent three weeks trying to turn a project around.

The project had troubles: big requirements that said nothing, schedule slippages that had already extended it to twice it's original estimate, no integrated version of the app we were developing, etc.

I had worked on this project for about a year, first as developer and then as a team lead. This was far beyond my original commitment and I was planning to leave. At the same time, the project manager was leaving, and a new one was not available for three weeks. I agreed to take over for that short time with the understanding that I could try to make some changes during the period.

This page summarizes my experiences over the three weeks. It was originally structured as a sort of diary but that didn't work as well as I hoped, so I've refactored it in terms of what I tried to do, what I think went well and what could have gone better.

-- CharliePoole

Postscript

In early October, about two months after the experiences described here, I spoke with one of the team leaders on the project to see how much of what was done in the three weeks had "stuck" and what the results were. I found that the project has continued to produce weekly drops that pass their test. After adjusting the project velocity, completion of stories has proceeded pretty much as estimated. Customer is happy, and pre-sales are up.

Considering that there had been no integrated version of the app in the first year of the project, I feel pretty happy about this result. The biggest factor in our success was the willingness of the team members to give practices that were new to them a serious try.

Post Postscript

Recently (December) I had lunch with the technology VP from the company to see how things were going and try to drum up a new contract now that I'm ready for work again. The project has been regularly meeting its schedule each iteration and they seem ready to try more of the same medicine on further projects. Cool!

General Thoughts

At the start, I asked myself "If I only have three weeks: how does that empower me?" My answers, which mostly turned out to be true, were:

With only three weeks, I won't waste time
With only three weeks, I'll have to quickly turn tasks over to other people
With only three weeks, I'm forced to recognize what parts of the project have most value
With only three weeks, I'll do the simplest thing that works
With only three weeks, people may be more willing to try out a new approach

The time frame presented some challenges too:

Three weeks isn't much time
People didn't completely believe that any changes would stick after I left
The work style forced by a short time frame wasn't for everyone

Overall Approach

I had been reading a lot about XP, but had never done a project that way. My main approach to changing this project was to borrow ideas from XP but try to convert the entire project to it midstream. I added in a few adhoc ideas from my past experience with projects.

I used only 4 of the 12 ExtremeProgrammingCorePractices:

Why only 4? I don't think the team could have handled more change in the three weeks. If I were staying longer, I'd be introducing some other practices and I have hopes that some of the other team members who stayed behind will be continuing to make changes.

Here's what I decided I would try to leave behind after three weeks:

a workable project structure
a set of stories
one integration cycle complete

Trivial Changes

It's hard to get people to believe that long-standing ways of doing things can change - even (or maybe especially) when they have been complaining about the old approaches for a long time. I made a few "trivial changes" right at the start in an attempt to lead people to believe that the more serious changes I planned would also come about. In fact, I think some of these were only superficially trivial and had significant payback.

We used to have a weekly status meeting where a manager asked each member of the team why they were late. Well, in fairness, that wasn't the intent, but most team members saw it that way. We dropped that meeting immediately. The three leads (see next section) and I would provide status reports at a meeting with my manager instead. For a whole team meeting, we set up a weekly breakfast on Mondays where people could talk about accomplishments, look ahead to the coming week and ask for help where it was needed.

Another change that we made was to acquire a "war room" for the team. It has a table for meeting and spreading out cards, a white board and we are waiting for a couple of integration machines to be set up.

These changes were small, but they seemed to have a lot of impact on how people felt about the project. I'd definitely recommend starting out any major restructuring with a few easy changes like this.

Project Structure

Our team of up to 12 C++ and VB programmers seemed too big and we had already experienced coordination problems. The traditional style of leadership in this company has one project manager directly supervising the entire development team. Programmers worked together only when necessary to solve particular problems. Introducing PairProgramming might have helped but it seemed to me that it would take much longer than three weeks to introduce such a change.

The company culture also went against putting one developer "above" any others. Nevertheless, I set up three teams of three to four programers and named one developer as a "team lead". I defined this role as one developer coming forward to help the team by coordinating its work. This seemed to work. The leads adapted to their new roles at varying rates but all three of them were doing well by the end of the period and I was able to deal, for the most part, with three people rather than 10 or 12.

I was lucky in that I got to pick the leads myself after working with most of the group for about a year. If I had been coming in cold to the project, I don't think I'd do the same thing.

Requirements clarification

We had continual discussions about what our requirements actually meant. I set up a Requirements Coordination Group to handle these questions with a customer (marketing) and two developers. The customer figures out the actual requirements. The developers communicate with team members who need to understand the requirement.

Answers to requirements questions are posted on our team web site. Some people have posed questions about implementation - i.e. where should this requirement be implemented? These get resolved as well.

At first this took a bit of time. Some people wanted to add a lot of documentation and record-keeping to the process but we managed to avoid that. I think this worked out really well and the key to its success was that the group focused on communicating rather than documenting.

Planning

Most of the changes already mentioned were in place the first week. However, we still didn't have a plan for completing the project. The requirements clarification process was good for individual questions but we needed to review all the requirements of the app.

We used stories for that purpose. I gave the entire team a little talk on writing stories and had them break into groups. The customer and I circulated through the groups. Having developers write stories is not the usual thing but since they have actually been coding this app for up to a year it seemed to work. They captured what they thought the stories were and then our customer corrected them. This led to lots of "Why didn't anyone tell us this before?" comments. The customer added a lot of stories we missed as well as prioritizing all of them. In the end we had about 160 prioritized stories.

Due to some unexpected circumstances - described next - we didn't develop estimates for all the stories until the very end of my three week stint. When I left the project, the final content of the release was still being debated.

First Iteration

While our organizational changes and story-writing activities were going on, we still continued to work toward our first integrated build. Basically, this involved stopping work on any new features and attempting to integrate as much of the prior work as we could. Once we had a set of stories, and even before planning the release, we identified those which we wanted to include in this first iteration - due at the end of week two.

Work was progressing but on Monday of week two a bombshell was dropped: a version of the product has to be shown to a customer in another state on Thursday. Of course, if we had been building regular integrated versions of the app all along, this would be easy. We hadn't been - this was to be the first. But the need was real and immediate. The three leads and I decided to press forward and complete our week's integrated build by Wednesday. Stories we didn't think would make it were dropped. We worked overtime for three days. The first integrated build in the life of the application was completed, installed and shown to the customer. It had no "demo" code - everything was stuff we needed for the iteration - just less of it than we had originally planned. For the rest of the week, we took it easy.

Of course, this isn't the way things are supposed to work. We should have done a release plan first and then defined the first iteration. A few developers were totally discouraged by this turn of events, seeing it as evidence that our planning was for nothing. I don't think that's true. It was only our planning that made it possible for us to adjust so quickly to an unexpected demand.

Release Plan

One negative side effect of our successful first iteration was that management's expectations were raised. In fact, one manager told the CEO that the entire project would be done in a month! When we actually did the release plan, it showed us that we needed about four times that time. As of my last day on the project, the final "official" plan was still under discussion.

Once our manager learned how we did the estimating, he immediately came up with the idea of speeding up the project by increasing the velocity used in the estimates. We convinced him that he should wait an iteration and see what the actual velocity turned out to be. In the end, it was decreased rather than increased.

What Was Accomplished

Identified the stories for the application and prioritized them
Established a pattern of regular integration and created an integrated build for the first time
Gained acceptance for a process involving fixed iterations with variable story content.
Developed a more workable project structure
A few people started taking testing more seriously - this would have been my next focus if I had been staying on the project.

Some of what I hoped to accomplish was done, some not. The fact that we dealt successfully with a sudden (and fairly unreasonable) demand made us all feel pretty successful.

Advice Received

I received some very useful advice on this page while I was doing the project. I'm preserving - for as long as folks want it - some of it here with comments as a way of thanking those who contributed.

Do not work longer than your official hours. You want to stay sharp for the whole of the three weeks.
View the requirements clarification group as a resource which the various teams draw on, rather than as being a central co-ordinating quango. ''
Focus on enabling the team members to genuinely take pride in their work again. let them turn the project round for you
Don't overload other people with memos or e-mails. (I did that to my manager in a similar situation)
Be as honest as you possibly can.

My experience of this was with a much smaller team (5 people). My main mistake was going into a rather manic mode, trying to do too much too fast, and looking too far ahead (beyond the three week horizon) - with the excess memos as the most visible symptom. The way I worked was not effective and did not turn the project around.

-- JamesCrook (initially)

Good stuff - I tried too much in the first week and then when the crisis hit, I couldn't ease back

One good way to get people to accomplish a drastic change in a short amount of time is a cheap but relevant measurement that you can track progress on. Instruct the team that they have to track and reduce/increase one or more (but not too many) of

Time to integrate new features
Regression tests
User stories
Or something else that you think is relevant.

Set the goal specifically, like time to integrate should be less than half an hour; and/or (my favourite) that the project should always be in a shippable state. This is ManagementByObjective?, which can be evil if not done holistically, but a definitely useful way to get things done quickly. Remember that MBO only works if you get buy in from the person(s) you are setting the objective for. -- SunirShah

[see Sunir's ReleaseEasyReleaseOften page]

P.S. Best, best, best of luck, man. Thanks for sharing this with us.

We weren't terribly formal about it, but the main metric for my three weeks was to always be integrated and never lose any functionality once we integrated it.

Charlie, thanks for the great topic and good luck with your project.

Here are some ideas I have about some more simple things that might possibly work.

(See VisualizingRisk)

I hope this is useful to you.

-- ChrisBaugh

I had never seen risk-reduction dealt with so specifically before I read this. I actually read it half way through my story-writing session. While I ate lunch, I made some adjustments to my process to focus on risk-reduction where it was needed. Our customer tends to focus on what he can see, so some back-end stuff was not getting dealt with till we talked about risk.

Charlie, from what I've read and experienced with XP, it's difficult to do ContinuousIntegration and SmallReleases without support of the other practices. Do you feel nervous about leaving a customer who expects SmallReleases and a team that has a hard time delivering on those expectations? -- JoeArnold

I plan to refactor and write more about this issue, but here's a quick take on the question.

For three weeks, the project accelerated. If we had continued to build on the practices, I believe that acceleration would have continued. Once I left, there was some attempt by individuals to do more: UnitTesting and Refactoring in particular. But, there was no real drive behind it. So the acceleration stopped and the project coasted - but at a greatly improved velocity. After some months, they began to get a little off even the four practices and actually slowed down. But they were still doing so much better than before that I'd call it a win.

Doing 4/12 of XP is far from ideal, but I'm thoroughly convinced that it was better than doing nothing. I'm now talking to the same company about doing more.

See "Charlie Poole, Lead Developer, NUnit", "Test Driven Development, XP, NUnit and more (March 14, 2005)" interview (two files) at http://teamagile.com/mainpages/Interviews.html

CategoryStory

Also see DeathMarchProject