Change Your Organization Diary Part Two March
Change Your Organization Diary Part Two MarchStart with ChangeYourOrganizationDiary. See also ChangeYourOrganizationTactics.
Next and previous parts:
4 March. Two days of chaos. I spent most of Monday (yesterday) working on getting people set up in their new location. We've hijacked an executive conference room that wasn't being used. It's a nice place. Big, well lit, good view. Well... almost big. With eight people on the team (seven programmers and one customer), it's getting a little crowded. We're probably going to have to set up just four pairing stations and find another way for other people to have a personal computer.
Yesterday and today were both pretty stressful because the programmers all have existing commitments to meetings. Also, there was an emergency need to estimate stories so a PM could create a budget. As a result, we didn't finish the iteration plan until the end of the day today. So it's felt a little out of control, from my perspective.
We have an iteration plan now, with task cards estimated and assigned. Unless the estimates are pessimistic, I think we have too much to do -- we've got an estimated load factor of 1.7, and given the number of meetings and fires they have here, I expect it to be more like 4.5. So we're probably not going to be able to get everything done.
I'm keeping my mouth shut for now, though. A lot of the estimates seem very high to me. (But I always think they're high, and I'm always wrong.) At the end of the day Friday, I'll know more. We've got another release planning meeting on Monday to account for adjusted velocity. I hope I don't have to deliver any bad news, but I'm not counting on it.
One thing that's surprised me about the situation here is that everyone is extremely specialized. I've never seen it so extreme. Everyone is very competent, but there's hardly any cross-project knowledge. In many cases, there's only one person who's qualified to estimate a task. I knew there was specialization, but this is out of control.
On what dimension are they specialized? Specific old technologies? --BillSeitz
6 March. JimHighsmith pointed out the SatirChangeModel to me. It describes the psychological impact of change and how individuals and organizations react to it. One of the interesting things about the model is that it shows performance increasing over time (after a new process has been introduced), but with jagged ups and downs, not smoothly. JimHighsmith mentioned that each low point is a potential time for abandoning the change.
I don't know much about the SatirChangeModel, although I did look it up online, but I have been experiencing those jagged ups and downs myself. Yesterday and today were both good examples. I alternate between feeling like things are going well to feeling like there's no chance this could work.
On the good side, the combined workspace is working well. I'm seeing SerendipitousCommunication already, and quite a bit of it. People are pairing, mostly, and they're communicating while they do it. Work is coming in at roughly 1/4 to 1/2 of estimate.
On the down side, last-minute tasks keep showing up on the board. Integrating with non-construction stories is proving to be a pain. There's a lot of questions about how to manage budget and bill customers. I haven't tried to introduce TestDrivenDevelopment, refactoring, or other code-centric practices.
One big problem that's looming on the horizon is that the project managers keep wanting to add people to the project. Faced with schedule overruns, they're adding people. It's good that they're manipulating the variable that they can, but it's starting to worry me. We're already at seven programmers (including myself). I'm concerned that if many more programmers are added, I won't be able to bring them all up to speed. It's already a struggle.
Right now, though, I think that our biggest source of problems is that there isn't one project. There's many. The customer isn't speaking with one voice, and I don't know how to make that happen. The person with the knowledge and clout to succeed in that role is far too busy to be available on the level that I need... maybe. I should pursue that further. We have an OnsiteCustomer, but he's come from QA and doesn't have authority over project managers. He's going to be great, I think, but he's a little reluctant (or maybe just unsure of his role) and I haven't had the time to work with him to work out the kinks.
I've also seen my first major negative reaction. One of the programmers called a meeting this morning with his PM and myself and tried to split his stories off of the team. His first objection was that with pairing, he didn't have enough time for his tasks. His second objection -- and I think this was the real objection -- was that he didn't like pairing and that the noise of having people constantly chatting around him was too distracting. (There might be a hidden third objection, which is that he was a team lead and now is one programmer of many.)
I'm not surprised about the resistance. Of all the XP practices, programmers resist PairProgramming the most, in my experience. I even mentioned this on Monday when I told everybody about what we'd be doing. Today I reminded him of that, asked him to be patient (it's only been three days) and to give it two to four weeks.
This afternoon, he was still pairing, so I think it's going to be okay. He's a good guy: solid, reliable, takes the lead. Last time I saw resistance like his, it was because the programmer had a personal commitment to get a project done. I think something similar may be going on here. He was / is tech lead on a specific project and he feels ownership of that project. He's concerned that all of this XP stuff will get in the way of getting that done. In a way, he's right. Yet another thing for me to figure out.
My personal approach to introducing XP is very hands-on. I pair with each programmer, I understand their issues and concerns, I make jokes, we have fun. In the course of doing this, I fix little problems. I talk about test-driven development. I tweak code so it fits a standard coding style. I push the keyboard over so my partner isn't reaching halfway across the table.
It's a personal, involved approach. It's never failed me... yet (Really! Never! Four projects, so far.), but it takes a lot of time. On this project, I've been too busy solving problems with integrating XP to the existing processes. I haven't had the time to work one-on-one with the programmers like I want to. In addition, I don't really understand the technology. There's a lot of old, non-object oriented languages here that I don't know, and a custom back-end product I haven't yet integrated with.
On the other hand, it takes about six weeks for my approach to really bring people around. It's only been three days so far. I pick up new technologies quickly, too. So what I need to do is focus on pairing as much as I can. That will resolve most of the issues I was just talking about.
The problem with that solution is that it leaves the PMs hanging out to dry. I've got to keep them happy, too. It's a tough balancing act.
"Be careful what you wish for?" I got what I was wishing for. Complete control over the combined team. Back in December, I boasted in this diary, "I know that, given the authority, I could turn this project around. In six months, I could have the whole organization dancing to a happy and productive little two-week beat."
Whew. Big claims. Now I have to follow through.
7 March. I'm still seeing passive-agressive resistance to pairing across the board. Not a big surprise. I'm not going to confront it yet.
Today was the end of the first week of our first iteration. At the end of the day, I spent several hours analyzing the data and putting together a nice status report for the project managers. I think they'll really like this report, because it gives them visibility that they never had before.
It's not strict XP, but I've got the team writing their time down on the task card whenever they work on a task. That allowed me to calculate how many ideal hours the team has spent working, and also allowed me to calculate estimate accuracy on the few tasks that have been completed. With that information in hand, I could figure out the team's progress on the iteration.
In normal XP, you don't do this. It takes several hours, it's error prone, and it's not very useful. But it gives me the opportunity to write a status report that will impress the PMs, so I did it. I've done status reports like this once before and gotten a very positive response.
The status report is concise but informative. There are four short paragraphs in the beginning followed by two tables. Each paragraph has a bold header:
The team is xx% billable.
There's a lot of unplanned tasks.
We're slightly ahead of schedule.
Our biggest risk is poor release planning.
Then the two tables. The first table has work broken down into planned billable work, unplanned billable work, and unbillable work. The second table has each story, planned or unplanned, that is included in the iteration. Each story has the number of hours completed, the calibrated estimate for the story, and the percent complete.
Again, this isn't standard XP and I'd rather not do it. But it's necessary that I impress the PMs by showing them how much visibility and control they have now that they didn't have before. Over time, I'm hoping to phase this status report out.
***How did I calculate all this, you may have wondered? (If you were smart, you didn't wonder. Skip past this, it's boring, and not XP at all. I only did it so I could have an impressive mid-iteration status report for the PMs.)
During iteration planning, programmers estimated tasks in terms of ideal days. As they worked on tasks, they tracked actual hours worked. So for each task that's complete, I can calculate estimate accuracy.
estimate_accuracy = (estimated_ideal_days * 8) / actual_ideal_hours E.g., estimated_ideal_days = 10; actual_ideal_hours = 100; therefore estimate_accuracy = (10 * 8) / 100 = 0.8 = 80%Given estimate accuracy, I can calibrate the estimates so the estimates are more accurate. If I've predicted estimate accuracy correctly, my calibrated estimates should equal actual work completed at the end of the iteration.
calibrated_estimated_ideal_days = estimated_ideal_days / estimate_accuracy. calibrated_estimated_ideal_days ~= actual_ideal_hours / 8 E.g., estimated_ideal_days = 10; estimate_accuracy = 80%; therefore calibrated_estimated_ideal_days = 10 / 0.8 = 12.5I can also measure how much ideal time is being done every day.
ideal_hours_per_day = actual_ideal_hours / calendar_days E.g., actual_ideal_hours = 100; calendar_days = 4; ideal_hours_per_day = 100 / 4 = 25Which allows me to predict how much ideal time I'll accomplish next week.
predicted_ideal_hours_next_week = ideal_hours_per_day * number_of_days_next_week E.g., ideal_hours_per_day = 25; number_of_days_next_week = 5; predicted_ideal_hours_next_week = 125Combine that with the hours already worked and I have my prediction for the total number of ideal hours I'll work this iteration.
predicted_total_ideal_hours = actual_ideal_hours + predicted_ideal_hours_next_week E.g., actual_ideal_hours = 100; predicted_ideal_hours_next_week = 125; predicted_total_ideal_hours = 225Now I can look at my total estimate for the iteration and calibrate it:
calibrated_total_estimate = total_estimate_in_hours / estimate_accuracy E.g., total_estimate_in_hours = 150; estimate_accuracy = 0.8; calibrated_total_estimate = 187.5Now I can tell if I'm going to finish the iteration on time or not by comparing predicted total ideal hours to the total estimate (in hours). If the total estimate is larger than the predicted hours, I won't make it. If it's smaller, I will.
E.g., predicted_total_ideal_hours = 225; calibrated_total_estimate = 187.5; 225 > 187.5... predicted work > estimated work... I win!I can also calculate percent done on any individual story, task, or project.
percent_done = actual_ideal_hours / calibrated_total_estimate_in_hoursThese last two equations provided the data I needed to answer the twin questions of "are we on track" and "how far along is my story?"
Again, don't do this! It's not actually that useful, except as a parlor trick, and it takes hours to set up.
What constitutes unbillable work? Is this actually non-work like chatting? Or is it something else? -- BrianRobinson
11 March. People are starting to adjust. Pairing is becoming more natural. PMs are coming into the bullpen to do release planning, almost without prompting. I'm even finding more time to pair. There's a long way to go, but at least the basic process is moving past the "you want us to do what?" stage. One thing that I haven't even started to introduce is testing and refactoring.
There one's holdout who still isn't pairing. He's working on a documentation story and more recently has been fighting a fire for a different project. So it's possible that he's not actively resisting pairing... but if it's not active, I think at least it's subconscious. Probably not coincidently, his PM is considering taking him off the project to finish up some documentation. I'm not going to stand in his way -- I've told him I want him on the team, but if he's set on going, that's fine. It would be unfortunate if he left: he would be a good foil for me.
As things start to settle down with the basic process, I find myself focusing on the next big problem. In this case, it's interaction with the customer. I had two separate conversations about what we couldn't say to the customer. In both cases, the thrust was that we couldn't tell the customer about problems. We couldn't tell them that there was a risk or that we couldn't meet a date. Blech.
I don't really know this for sure, but I think the problem is with my organization, not with the customer's. The customer is a big company, sure, but even big organizations are made up of people. Talk to the right people in the right way and problems can be raised. My experience is that customers love being made aware of potential problems... if they're told about them in time to resolve them.
Here's an example of something that came up today. The customer pushed a date forward for a requirements document. The PM was stressed that we wouldn't be able to get it done on time due to historically slow customer feedback. I said that we just needed to make them aware of the tradeoffs and the need for cooperation. The PM said we couldn't do that, that we just had to do whatever it took to meet our commitments.
I'm afraid I lost my temper at that point. Not in a bad way, fortunately. But I did start speaking very directly about the need to make the customer aware of risks, not to just bend over and take whatever they dished out. (Um... those were my exact words. Okay, so I wasn't exactly Mr. GQ.) I grabbed a couple of cards and asked the programmer what the risks to getting the work done were. Each one involved not getting something from the customer in time. I wrote each one on a card and said, "So if these risks are resolved, can we get the work done?" He said yes. I turned to the PM and said, "Tell them that we can do the work, but that we need responses by such and such date and such and such involvement after that. Tell them that the work will slip if that doesn't happen. Then remind them regularly up until the deadline. At the deadline, remind them again and slip the work."
I'm not sure how that went over. This is the exact story that's potentially being taken out of my purview, so I may not be tracking it. But I think I should. This company needs practice in working with customers to resolve problems before they turn into expensive firefighting nightmares.
What's ironic is that the customer is practically begging us for an agile process. They want the ability to change their mind and they want shorter development cycles. Hmm. Sounds familiar.
My next goal is pretty clear: I need to start understanding the people and structure of the client company and get to the point where I can talk to them directly.
When are you going to intrduce testing and refactoring??? I've only used parts of XP on personal projects, but I have found that if I don't make tests as I go, adding tests later is hard work. And if I don't have tests, refactoring is almost impossible.
(Not based on experience). If you are retro-testing, one thing you might do is throw some test stories into your iterations. These explicit stories could implement the important tests, while building up the test framework (e.g. side effect testing), until it is easier to use for the other developers. And your velocity wouldn't go down :). Anyway, I'm just guessing. Good luck whatever you do!
13 March. One more day until the end of the iteration. I've been saying all along that a story wasn't done until it was 100% done, but the "100%" part only really sunk in today. A bunch of risks have come to light as a result. This is a good thing, even though it means we might not get all of the stories done.
Another interesting thing that happened today was another release planning meeting. There's clearly issues with having all of the PM's coordinate their work into a single release plan. I think everybody agrees that it's valuable, but it's been very challenging.
Something that's particularly unusual about this place is that they're not resource constrained. I've never been in an environment like that. Here, when PMs say that scope and time are fixed, they really can fix scope and time! They get to add people.
This is such a foreign situation for me that I'm not really sure how to deal with it. Adding resources has a non-linear effect on velocity, as we all know. Previously, PMs got around this by sacrificing quality: they would add people to a project willy-nilly and then those people would do "whatever it takes" to get features done on time. I haven't seen the resulting code, but I would be surprised if it was any good. But then again, the code here is fairly low quality anyway.
During today's release planning meeting, the PMs were complaining about the velocity. They said they couldn't adjust time or scope. In the past, they would just request N resources and those people would be assigned to their project. Now they wanted to request N ideal weeks and have those assigned to their project.
I've been resisting this approach up until now, but it's because I've been trapped in the resource constrained mode of thinking. If the projects really aren't resource constrained, then we can add people so that the total velocity equals what the PMs collectively need.
This is going to take some creativity. After all, Mr. Brooks does have a few things to say about adding people to a project. And my way of introducing people to XP is hands-on and requires a lot of personal attention.
Up until now, I've been focused on efficiency. What if I gave that up? What this company really needs is reliability and quality, at least for now. I've been doing things that improve efficiency as well, because I could, and because it's usually a priority. But here, maybe I can sacrifice some efficiency in favor of meeting those all-important time and scope deadlines.
Of course, no project is really resource unconstrained. Not even these projects. So whatever I come up with, I'll have to make the tradeoffs visible.
Who knows. It's going to be interesting.
15 March. In TheSecretsOfConsulting, GeraldWeinberg said,
"If you stay in the brine long enough, you become a pickle."
I'm slowly starting to turn back into a cucumber. Phew. What a relief. I've been trapped in this world of MatrixManagement, blind obedience, and mandatory success so long that I'd almost forgotten that you could be a human being at work. (It's not just this company, either. I've been pickling for over a year.) Take a look at my early attempt at leadership tactics... ewww, briny. I'm fond of my latest ones, though. (Uh oh. Note to self: review KillYourDarlings.)
Today was the last day of the first iteration. I'm feeling particularly empowered because of our seven stories, we only delivered three. Our velocity was four ideal weeks and we planned seven.
I feel great, because in my weekly status report today, I admitted it. I didn't whitewash, sugarcoat, or hide the tiniest sliver of this fact. In fact, I took responsibility for the problem. I laid out the issues and I described how we were going to fix them. My attitude was straight-forward and honest, without the slightest hint of apology or defensiveness. Essentially, I said, "Several stories in this first iteration failed because I didn't manage risks. We've learned a lot, the team did good work, and I'll be proactive about risk management next iteration. Here are the major problems we're seeing and some likely solutions. [...]"
It feels so good to be honest.
17 March. It's a tough job, balancing honesty with tact. I, unfortunately, tend to error on the side of honesty. That's what I did today.
It's the first day of our second iteration. As usual, we spent most of the day planning. At the beginning of the day, we had a release planning meeting. Then we had a break, and a post-mortem for iteration 1. Then lunch, and iteration planning. Iteration planning took the rest of the day.
I made the mistake of being too honest during release planning. My onsite customer, who at first was detached, is really starting to step up to the plate. He recognized that we weren't doing adequate long-term planning last week and has, nearly single-handedly, taken responsibility for addressing it. At today's release planning meeting, he presented his long-term forecast to the PMs.
It showed that we needed a minimum velocity of 11 ideal weeks, peaking at 14, in order to deliver all the currently planned projects on time. Not on his plan were the several projects not yet scheduled. Our actual measured velocity for iteration 1 was four ideal weeks. That number was skewed a little low by some iteration 1 bumps, so call it six in reality. Regardless, it's half of what we need.
This long-term forecast is a wonderful document. I heartily recommend it if you're ever in my situation. Here's how it works. First, take the release date and estimates for each project. Figure out how many iterations are between the start of programming and the release date. Divide the estimates by iterations, repeat for each project, sum, and voila... minimum average velocity. Note the word minimum.
The good thing about the long-term forecast is that it's a reality check. It doesn't take the place of detailed release planning, but it puts a stake in the ground, tells you if you're nuts.
In our case, it told us we were nuts.
The PMs took this in stride. They're good folks, just excessively nervous about delivery dates. We started talking about ways to increase the velocity. Naturally, the PMs talked about adding people.
This is the part where I was excessively honest. Rather than saying, "yes, that's a good idea," I said, "yes, that's a good idea, but I'm nervous about my ability to handle too many people."
Bzzzt! Bad idea! We ended up in a long, unproductive, tension-inducing conversation about risks, why we were doing XP in the first place, the point of diminishing returns, yadda yadda yadda. It didn't do anyone any good. Eventually it was decided that I would take responsibility for increasing the velocity, however it was done.
Which is where we would have been in the first place, if I had just kept my fears to myself. Raising risks is important, being honest is very important, but I should not raise potential risks. Risks are the possibility of problems. That's fine. What I did was talk about the possibility of a possibility. Too abstract.
The rest of the day went well, though, and nobody got out the pitchforks and flaming torches. The PMs don't participate in the rest of the meetings, just our on-site customer, which is probably for the best.
One item of interest remaining: during our post-mortem, we identified one thing to do and one thing not to do in the upcoming iteration. The answers were wonderful. I didn't prompt them at all. Well, for the first one I did, a little bit.
19 March. It's been very hard, but things are starting to come together. People are relaxing into the routine. They're laughing, joking, enjoying themselves. (The turning point was the discussion about having fun. They just needed to know it was okay to have fun while working hard.) We've got solid strategies for dealing with our two biggest problems: "emergency" work and external dependencies.
I'm still spending the vast majority of my time dealing with process issues. I paired half an hour today. Half an hour! That's much worse than I recall from my last big XP transition project... but I know more this time, so I'm probably doing more. It's also a much more complicated environment.
Even though it's been bumpy, things are firming up very quickly. It's only been two and a half weeks. That may be because I've focused solely on process. Test-driven development and refactoring are still in the future. I'll introduce those once I'm comfortable with codebase, and that won't happen until I can pair nearly full-time. Most of it I don't know at all, and it's written in a screwy (but popular) language that's new to me.
I can't get too confident, though. There's still some very difficult issues yet to be exposed. The next big one I see lurking in the sand is the client relationship. Making the external dependency problem visible will probably pull that one out as well.
27 March. I was right about the client relationship. It's now the top agenda item for just about everybody involved in this project other than the tech team. It's on everyone's lips. Let's hear it for leverage!
Now that I have an XP team, I'm finding organizational change to be much easier than in ChangeYourOrganizationDiaryPartOne. Goals are clear and easy to identify: everyone wants more velocity. It's easy to use that as a lever to cause change. My current process for change looks like this:
When they didn't get resolved -- and not a single one did -- I talked with the PM who owned the affected stories. We removed his stories from the board and scheduled a new story. I sent out a status email explaining that nearly all of the work done in the first week was cancelled due to external dependencies. I started mentioning casually that our velocity would be four, not eight, due to lost time on cancelled work. Then in our release planning meeting yesterday, I took a firm line. I said that we couldn't accept more than four ideal weeks of work because that's what our trend was showing, and that we couldn't assume anything better due to the external dependency problem.
Voila. It's not a surprise, so no argument. The developers have been feeling pain on this for months, but now the low velocity causes the PMs to feel that pain too. Suddenly, the issue is on everyone's mind. PMs are agreeing not to schedule stories that have unresolved dependencies. Higher-ups are escalating the issue with the client. Stuff is happening.
Whether or not it actually resolve the problem is an unanswered question. I'll learn the answer next week.
EditText of this page
(last edited February 7, 2004)
or FindPage with title or text search