Change Your Organization Diary Part One
Change Your Organization Diary Part OneStart with ChangeYourOrganizationDiary. See also ChangeYourOrganizationTactics.
My organization is like a lot of organizations. It has good points and bad points. I think the good points outweigh the bad, which is one of the reasons I'm working so hard at changing things: I'd like to stay here for a while. The main reason, though, is that I get grumpy easily. I don't like working in environments which make it hard to produce quality software.
On the good side, this organization is willing to change. There's quiet interest in agile development, but not a lot of knowledge of what that means. There's acknowledgement of problems and desire to improve. The people in positions of authority are generally easy to talk to and are open-minded.
On the not-so-good side, there are a lot of barriers to creating quality software. People specialize in technologies and are composed into project teams. (This is called MatrixManagement.) WaterfallProcess? is the primary methodology. There's some bullpens, but quite a few cubicles. It's nearly impossible to PairProgram at the desks.
The worst problem, though, is communication. I think it may be a result of MatrixManagement: the UI people work on the UI stuff, and the database people work on the database stuff. They go to separate meetings and have separate managers. It gets worse: project managers are a separate group that sits on the opposite side of the building. QA is on a whole 'nother floor and typically doesn't get involved until a project is coded. There's no real concept of team: a project brings together a UI person, a database person, a project manager, and a QA person. They work away, emailing back and forth and going to lots of meetings, until the project is done. Then each person goes his separate way. Worse, everybody's assigned to multiple projects simultaneously. There's really obnoxious time tracking requirements to go along with this.
As I look back on this list, I'm encouraged. All the good things are personality related: it would be impossible for me to change them if they weren't good. All the not-so-good things are culturally related: they're aspects of the way the company's used to doing business. Hard to change, but not impossible. There's hope.
I started this diary on Nov 27th (2002), so the following is from memory.
October 1st. I get hired. I talk with the person I'm replacing for half a day, and then he leaves. Today was his last day. I have a desk and a computer. Time to get to work.
October 4th. One of the other people here who's into XP is working on an automated build script. There's hope.
~ October 7th. It doesn't take me long to identify the communication problems. I don't feel like I understand what I'm working on or how it relates to anything else. I address the immediate problem by picking up my computer and moving into the bullpen with the other programmers on my project. Fortunately, I'm only assigned to one project right now. With the benefit of hindsight, I can say that this was a great move. I've picked up a lot of information about the project just by being in place to overhear conversations.
~ October 21st. The project I'm on is in its final stages. We're trying to fix defects so we can release to production. We have an hour-long meeting to discuss defects every day. I try to resolve the problem by writing my defects on cards and taping them to a whiteboard. I say, "If you want to know my status, just look at the whiteboard. Everything I'm working on is on there." I estimate costs for my defects as well.
~ October 24th. No one else is writing their tasks on cards. I stop estimating.
~ November 4th. I keep talking about process improvement and agile methodologies. I'm trying not to be a ChronicComplainer but I fear that I slip into that role occasionally. On the other hand, whenever I do complain about something, it's in the context of how we could fix it. Still, I think I'm probably being at least a little obnoxious. It's hard to curb my tendencies, though. Maybe writing this diary will allow me to get it out without irritating my coworkers.
On the other hand, my comments are being noticed. Or maybe it's because I was able to start fixing defects within the first week. Either way, I'm being invited to more architectural-level meetings.
~ November 7th. I take down my cards. The cards got noticed by other people, in a sort of "What's that odd collection of cards on the white board" sense, but they weren't really used by the project manager. The other two people on the team weren't following my example, so there wasn't really a point. I keep using the cards, but I just keep them on my desk.
~ November 12th. This organization, like many out-source organizations, wants to increase reuse by building a framework. I had a discussion about process with the architect of the framework yesterday and we had a meeting about it today. My points were well received, and when I suggested that we make them official policy, I got permission. So as to not lose momentum, I wrote up a one-page "official framework policy" document! We now have an agile process for framework development.
The process is very, very simple. It goes like this: Anybody can work with the framework, but
The not-so-good part is that the "official policy" is not so official, as yet. I got permission that this be the official policy, but then I drafted it and sent it out. Nobody has officially anointed it as official. So to speak.
~ November 18th. I sent an email to the director a few weeks ago saying that I have experience with process improvement and describing some risks I saw with specialization. Today we had a meeting. It was a great meeting in that we got along well and discussed a huge variety of topics. I think I established my credibility. It was not such a great meeting in that there was no call to action. There's an interest in change, but in cautious change. I can't really blame him. I left feeling that I had implicit permission to make minor changes. That's not what he said, but now I feel like I'll be given some slack if I do. I don't have any real authority, but I've always been good at talking people into trying new techniques.
~ November 20th. The task-tracking whiteboard has spontaneously come back to life! No index cards, just markers. That's okay. The cool thing was that I didn't do anything.
~ November 22nd. I've converted the whiteboard from markers to index cards. Not much resistance... erasing and moving around marked notes was getting messy. Today, Nov 26th, the board looks great.
November 25th. The other XP guy has installed a Wiki. It's being adopted enthusiastically by several people. It looks like it will fly.
November 26th. I've introduced the idea of prioritization on the tracking whiteboard. I keep asking the project manager to order my cards by priority... and I think it's starting to catch on. I've also gotten the QA guy to submit tasks through the project manager for prioritization rather than just emailing me all the time.
This brings us to the present. Future diary entries should be a little more detailed, and will talk about the problems I face in more depth. The brief entries up to now have made things look straightforward and easy, I think.
27 November. Today was a very good day. For the last two weeks, we've been having a growing problem with FireFighting. We're in the very last stages of releasing a product, and QA keeps discovering defects. Defects are scheduled according to what the customer feels is critical. Everything critical has to be included in the next drop. Drops take two hours because our builds aren't automated. We drop twice a week.
As we've gotten closer and closer to the 'final' drop, the pressure has mounted. Yesterday , we got stuff that "absolutely has to go in!" the morning of the drop. Today, my last day before the next drop, I got a few defects that "absolutely has to go in!" an hour before the end of the day.
Up until now, I haven't been reacting to this insanity well. The project manager's approach to the problem has been to ask us (actually, to tell us in the guise of asking us) to stay later, come in earlier, work on Thanksgiving, etc. Yesterday I snapped and refused. I told the project manager that we needed to solve the firefighting problem, not to ask programmers to work extra hours. No, I wouldn't be coming in early.
True, yes. Smart, no.
I realized my mistake right away and called the PM to ApologizeUnconditionally. I'm glad I did: things were friendly today.
So why was today a very good day? Because the same situation came up today, and I handled it much better. I think my approach got us one step closer to an agile process.
As I mentioned earlier, an hour before the end of the day, the PM brought in two new defects. I dutifully wrote them on cards, got him to prioritize them, and taped them to the whiteboard. For the first time, I also estimated them, at two hours each, and told the PM that I could probably only get one done before I left.
The PM went back to his cube (on the other side of the building). A few minutes later, I got a call:
Question: If your build system is such a nightmare (two hours for a "drop", with a complicated list of instructions), why don't you fix it? On my previous assignment, we had build instructions that involved hopping from this machine to that machine to run the various commands necessary to build the system -- clumsy, fragile, and annoying. You also can run into problems where building a version of the system out of version control doesn't give you the same system, because the build rules have changed. I changed it into "push the button and it goes" system -- still a bit clumsy, perhaps, but now it's revision controlled and, most importantly, doesn't eat up a programmer hour just to compile the system.
My answer to this question is clear in my own mind, but I'm having trouble expressing it. Everybody in my organization already knows that fixing the build system is a good idea, but they're not making it a priority. I hate to drag out a hoary old cliche, but think of it this way: fixing the build system would be like giving the organization a fish. Instead, I'd like to teach the organization to want to fish.
So I'll be successful when the organization says, "We're wasting too much time on manual builds. Hey, programmers, how much time would it take to automate this?" (Programmers answer.) "Okay, do it. We should recognize return on that investment by the end of this month, so let's tentatively add these two items into the release plan for the next month." I'd even be happy if they said, "No, that's too much time. Keep doing it manually." Any answer would be fine -- I just want conscious consideration of issues like these.
To achieve this goal, I'm doing several things:
Well said. Note the use of the term priority: The build system may be horrific, but other things may be even more important. This is an extremely important principle, and is often forgotten in the rush to do the RightThing. -- BrentNewhall
2 December. Every day, I make a tiny bit of progress, but I'm also reminded of how far I have to go. It gets depressing. If I were offered a coaching job, I'd take it in a heartbeat. On the other hand, I'm not actively looking for other opportunities. That either says that the situation is hopeful, or that I'm lazy. (Or stupid. My efforts could well get me fired.)
What gets depressing is not that the organization is backwards. That's pretty common. What's depressing is that I have so little control. I know that, given the authority, I could turn this project around. In six months, I could have the whole organization dancing to a happy and productive little two-week beat. I just don't have the authority.
There were a few things that have been working in my favor. The last two weeks have been a debacle for my project. Quality problems up the yin-yang. People working until dawn over the Thanksgiving holiday. Stuff like that.
It's too bad for the project, but it's good for me, and it's good for the project's long-term health. This was bound to happen with the way things were being run. I'm glad it happened sooner rather than later. People are a little more willing to change, and I can make my suggestions with a little more authority. When I made suggestions today, there was more understanding and no disagreement. I didn't try to implement any radical changes, but all the suggestions I made were implemented.
Starting tomorrow:
This actually gets us pretty close to where I want us to be. I want:
Some agile ideas are in place, but they're very fragile. I've gotten them to sprout, but if I were to leave, they would wither away:
3 December. Today was a surprisingly boring day. It shouldn't have been -- we had a major issue crop up, and today was the last day we could do a drop before releasing to production. But everything proceeded calmly. I don't think I can take all of the credit... but maybe I deserve it. The same scenario in the past has been a nightmare. All of the initiatives I've pushed for played a role in our success:
When I arrived today, a number of issues were sketched out on the whiteboard. I talked to the PM, wrote the issues down on tasks cards, got him to prioritize, and estimated with the help of the other on-site developer. There was a major issue, so I paired with another developer that we occasionally work with, and we worked through the day.
At three o'clock, I overheard the PM say something about doing a drop. This was the first I'd heard of it. When he got off the phone, I asked him whether we were doing a drop today. When he said yes, I told him that I wanted to get home at a reasonable hour tonight, which meant that we needed to start the drop by 5:00. Did the new issue on the board (previously prioritized as "least important") need to go in?
He answered yes, so I told him that since I had estimated it at two hours, I would have to stop working on the major issue that I had been working on all day. Fortunately, my pairing partner was completely up to date on all the issues and could continue working on it without me.
Because we had calibrated our estimates by that point, the PM knew my logic was sound. He didn't argue. I started working on the last-minute task, but did so with a relaxed feeling, confident that I had enough time. My pairing partner continued working on the major issue alone.
I finished the last-minute task around 5:00. I confirmed with everybody that I was going to do a build, and then printed out the build checklist I had put on Wiki. The PM had updated it earlier to correct a problem with the previous night's build.
The build went smoothly and I left right on time!
***Looking back at what I've written, it reads like one of those oh-so-annoying made-up examples of a new process in action. But it's all true. I know there's going to be lots of problems ahead, but I think we may have turned the corner. As I said in my introduction, everything I've put into place was used today:
4 December. Code freeze today. Nothing interesting happening. This sort of feast/famine cycle has been common. We go into production in a few days, and we'll all be standing by, prepared to support that effort.
11 December. Code finally went live. Yep, it took a week. I spent most of the time fighting fires, but I did have time to write an essay about improving communication by collocating programmers, QA, and project manager. I was careful to propose a very small change: just collocating people. I didn't mention unit testing, pair programming, or iterations. I see collocation as a necessary first step, so that's what I focused on. And, if I could only choose one thing, collocation would be it. Communication is a big problem here.
The other XP guy here thinks I'm being too cautious. We'll see. Previous attempts at top-down change have resulted in disinterested responses and comments about changes being too big. The XP guy thinks my timing was just off... now that we're finishing a product cycle, now is the time to change the process. Also, the fires have been so bad, they're impossible to ignore. He called a meeting yesterday (that I couldn't go to, darn it) and process issues were discussed for an hour and a half.
No one else has given me any feedback on the essay yet, but that's because there hasn't been time.
12 December. Reaction to the essay has been universally positive, but only mildly. Everybody has said they liked it, but I've had no real feedback. It must have made an impression on someone, though, because I saw one of the managers that I didn't give a copy to carrying it.
15 December. Lots of good things happened today, but I'm short on time and can't discuss them in depth. But basically, I'm starting to see my labors bear some fruit. On two separate occasions today, I had the opportunity to describe my ideal process to managers. The whole nine yards: bullpen, pairing, test-driven development, me coaching, my past coaching experience.
Why haven't I done this before? I was never asked. The last thing I want to do is shove big ideas down someone's throat. I've been waiting for the right moment ever since I started working there. It's interesting that it arrived twice on the same day. Oh, and both times, the response was active interest and talk about needing a new project to come along so we can do it.
The other good thing that happened today was on the bottom-up front. My current team had a planning game! It took nearly four hours, which is typical in my experience. It started out a little rocky, as we were figuring out which tasks to include, but then went very smoothly once people started volunteering for and estimating tasks. I also had everybody estimate their own velocity, for use over the next week. Next week, we plan on having another planning game to check velocities and update estimates.
I got the planning game to occur by talking about it over the last week or so. I kept saying to anyone who would listen that I wanted to have everyone sit down before we started the next version to look at the tasks and estimate them. Then today, my comments resulted in an architect here asking the team lead to schedule a meeting to discuss features to be included in the next version. Before that meeting, I talked to everybody I could, saying that I wanted to do something called a planning game, and here's what it was. I got buy-in from the team lead to do so and lead the meeting. As people were gathering for the meeting, I told the project manager and architect that I had been talking to people about doing a planning game, here's what it was, did they mind. It was hard work setting it all up, but it paid off.
I feel like things are building up to a head here. I wish I could say why. I've effected some change on my local project, but it's been very sporadic and labor intensive: the traditional "two steps forward, one step back." But I'm also seeing wide interest in process improvement that isn't directly related to what I've been doing. I'm not even certain that it's a result of my efforts. If it is, it's because I've been continually talking about process improvement to anyone who will listen. Maybe I've raised people's awareness and interest without them directly realizing it. I don't think anyone would point to me if asked why the organization is suddenly so interested in process improvement.
16 December. Remember how I said it's been "two steps forward, one step back?" Today was a step back. The project manager and QA lead are no longer sitting in the bullpen. The PM is ignoring the cards we prepared Friday and has gone back to handing out spreadsheets with "commitment dates" on them. I'm pretty sure he's added tasks to the list as well, or at least shuffled owners around. I asked the PM if would mind sitting in the bullpen with us again, since it was so helpful last time... he was clearly reluctant, and said something about needing to make sure space was available.
Days like today are depressing.
But now that I've been off work for a while, I can apply a bit of perspective. Why is the PM resisting? I think he knows, deep down, that the cards and bullpen are effective. So what's going on? Well...
(I wish I could just apply a RolledUpNewspaper. I feel like I'm trying to prop up a soggy mattress: the part I'm holding on to stays up, but everything else droops.)
17 December. Struggling back up... today I got the PM to prioritize the task cards we created Friday. It wasn't easy... I had to gently prod several times. All the programmers were very happy to have them prioritized, which was gratifying. I would say that the programmers are very happy with the index card approach. The PM is not actively against the idea, but is being passive aggressive about it: basically, he ignores them and does his own thing. I think he's definitely uncomfortable with the task cards, but I'm not sure why or what to do about it. If you're reading this and have an idea, I would like your input.
...what are your personal goals in this situation? (long term, medium term, short term?) -- AnonymousDonor
Hmm... just ask. How novel! ;) I'm reluctant to do that, though, and I'm not sure why. Maybe because I want to avoid confrontation? Maybe because I'm afraid of what the answer would be? Something for me to ponder.
DigitalStoryCards: I've never liked them, to be honest. My answer to "IsAnythingBetterThanPaper?" is "Almost never!" Obviously I'm biased. But my rational answer is that I want to avoid technologies that give people an excuse not to sit together. I'll keep them in mind for the future.
18 December. Today I learned that the organization boss has acceded to a real bullpen! The details aren't settled yet, but it's probably going to combine several small, related teams into one, and include QA and the project manager. Just what I've been wanting! Now I can emphasize the next change more: a bullpen doing mostly-XP with me as coach. The boss is going to want a detailed plan for migrating to the bullpen environment, so my next step will be to create that plan. It's going to be a big project, so I'll probably do it over my Christmas vacation. Hopefully they won't proceed while I'm gone.
So, why did this happen? I asked my champion the same question. (Uh... champion? It's a word I just sort-of invented: somebody I work with who has more clout than me and thinks along generally the same lines I do.) He said that he wasn't sure, but that the boss was pretty open to the idea. He thought it was just because lots of people were talking about the idea.
I can't prove it, and it's oh-so-unhumble of me to claim this, but I'm pretty sure that was my doing. I've been talking about bullpens a lot recently.
Hmm... a side note. When I say I've been talking about something a lot, I mean a few times a day. I actually don't talk about it that much, relative to the other stuff I do. The majority of my day is spent doing my actual work. Every so often, a legitimate opportunity to say something change-related comes up. By "talking about X a lot," I actually mean that the majority of my change-related comments are about X.
What does a legitimate opportunity to say something change-related look like? Here's some. I've put the ones that I think result in the most change at the top.
Every so often, I'm asked a direct question about process. These occasions are rare but wonderful. I get to spew all kinds of stuff. Impact-wise, it's probably about the same level as "something vaguely related comes up." But since I get to explain my viewpoint in more detail, it probably raises my credibility a bit more than the few-sentence approach.
I also write papers. So far, I think I've averaged one every few weeks. The papers are targeted at people who are higher in the organization and generally have a direct call to action. So far, I've written them after I've already seeded the change pretty thoroughly via one of the above methods. No one has ever responded to one of the papers and said "Yes, let's do what's in this paper." But so far, the main actions recommended in all of my papers have been followed. I don't know if that's because of the word-of-mouth, the paper, a combination of the two, or just blind luck. So far I've written three papers:
19 December. Feeling particularly burned-out today. No real reason.
(vacation)
7 January. I'm approaching a critical moment in my time here. Since I've started, I've been uncertain about what form of "ChangeYourOrganization" this diary would end up demonstrating. I think the next few weeks will tell me the answer. Coincidentally, other contract options are starting to open up.
I returned from my vacation yesterday. My normal duties have been light, so I've been working on a document that describes a specific XP-like process and reasons for switching. I've hinted at this before, but this is the first time I've been this blunt. The timing is perfect: we're about to start several new projects that need this process and the project I'm currently on is winding down. I've been here long enough to earn respect and to gain an understanding of the company and its customers. If I can't make a significant change now, it's unlikely I can do so.
The above sounds a little strident. That's not the attitude I have at work, or at least I try not to. I think this is the best possible time for me to make a change, so I'm doing everything I can to make sure it's successful.
I finished the first draft of my paper today. I passed out a copy to a few people I trust and tomorrow I'll go over it with my champion. I plan to be clear that I consider this paper to be the culmination of my efforts and that it's important to make sure everything's correct. I'm hoping that, with his help, we'll fine-tune the paper and then (somehow) present it to management.
I'm not really sure what will happen or how I expect this paper to help. Previous papers have been well received, particularly the last one, so I'm continuing in that vein. I did a pretty good job with this paper, I think, so hopefully it will have an effect. One thing I know is that there is a genuine need for the changes I'm suggesting. If the organization doesn't change, it will have difficulty dealing with the projects coming up.
So... what happens if this paper doesn't succeed? I have a few options left. First, the most likely result is that there's a lukewarm to positive reaction, but no actual change. I've never gotten a genuinely negative reaction. (Well, once. But that was a reaction to the way I did something, not the basic suggestion.) The reactions are generally of the "good idea in theory, but you don't really understand what we're facing here" nature. Not that anyone's actually said that. I have never actually gotten into a serious discussion with anyone with authority about what kinds of change are necessary, and what is feasible given the needs of the organization.
As I was saying, if there's no real response to this paper, then I have a few options left. I could talk to my main manager about my lack of responsibility and influence. I could tell him that I feel I'm being underutilized. We've had a brief discussion along these lines already, but I could be more pointed about it. Another option would be to talk to the director, the person with the power to make these sorts of changes, and take my case directly to him. I've talked with him once before (18 November) and it was positive.
I don't really want to take either of these steps. I'm not sure why, but I feel like these are last-ditch efforts. If I make my case and the answer is "no," then that's it. I've taken it as far as I can go.
Eventually, that conversation may need to happen. But when it does, I'd like the decision to already be clear... and in my favor.
8 January. Went over my paper with my champion today. Got the reaction I expected: agreement with the basic concepts, disagreement that it was possible in this particular organization. Ironically, he talked about the difficulty of change and how some people resisted change without knowing it. :)
I kept at it, though, and eventually he agreed to run it up the proverbial flagpole with management. I could have done that myself, and maybe I should have, but I don't have a day-to-day relationship with the other members of management yet. Maybe I should start working on that.
I did learn one interesting thing: my champion, along with his counterpart in project management, has the authority to make the changes I'm recommending.
I'm going to let it sit and come back to it next week. In the meantime, I still have a lot of spare time. Maybe I'll start establishing a unit testing beachhead.
14 January. I've been going through a lot of ups and downs lately. I alternate between extreme burn-out and calm patience. The paper is making some rounds... I'll be talking to someone with influence about it tomorrow. This is actually the first person who has questioned the things I've said, which I take to be a good sign: it means he's actually reading it, not just smiling and nodding.
16 January. As I mentioned in my entry from the 14th, I talked to someone with influence over process about my paper yesterday. The upshot was that he and I have the same basic perspective on process, although I'm a bit more radical when it comes to agile development. We agree on what the major problems facing the organization are. We even agree, more or less, on the first step.
Unfortunately, that's the point where I hit a stone wall. I don't know what it is about this company: everybody happily agrees with me, but that's as far as it goes. Nothing happens. I pushed and pushed for action... gently... and the response was essentially this: "There's nothing you can do to help at this time. Please wait an indefinite time for an indeterminate result. At some point we'll let you know, and you'll have 'the important conversation' with the guy in charge."
At first, I was very disappointed with the lack of results. I still go through times every day where I feel completely helpless and burned-out. But now I think I'm just going to sit, wait, and see what happens. Communication failures are already starting to occur in a big way. We had some problems today with people accidently overwriting each others' files... exactly the kind of thing I predicted in my paper. (Unfortunately, I just predicted communication problems, not this specific issue. I wish I had!)
I hope people will connect my predictions with what's happened, but I get the impression that people think I'm writing these papers from some sort of idealistic viewpoint, not a practical one. I might say "I told you so!" eventually, but not just yet. I've bruised enough egos with my recent push. Now it's time to just put my head down and observe. I'm going to focus on low-level, tech-oriented change for a while. I've started introducing unit tests for our UI components.
I may be petty, but I'm going to enjoy watching things fall down for a while. I did everything I could to stop it, but I was ignored. I don't feel the slightest bit guilty about my enjoyment.
Maybe I do feel a bit bitter. Any advice, O Readers? (Assuming there's any left! Add your thoughts to the bottom of this entry, please!)
A familiar frustration. See e.g. http://www.moebius.nl/inch/index.php/TheDoomedProject. (BrokenLink 2004-06-28) Feel free to get in touch privately if you want to compare notes. -- LaurentBossavit
It might help to document the failures as they occur, noting why each failure would not have happened if your suggestions were adopted. It would give you something concrete you could point to when people ask "how would your changes help us?" (Disclaimer: I haven't been in this situation.)
17 January. A couple of notable successes today. First, and most enjoyable: I successfully created some unit tests for the UI portion of our app. Although this may not have much impact on the organization, it felt good to write some quality code for a change.
Second, management has just done a re-org and is moving people around. This involves shuffling cube walls. I got word today that the new floor plan includes the bullpen I've been pushing for. I'm not as thrilled as I could be, though, because the project managers also moved... to a different floor. And the new bullpen won't have room for all of the programmers on the project. And QA won't be included. Sigh. It's actually not much different from what we already have.
But I can keep writing quality code.
22 January. The bullpen idea is forging ahead. Today, I talked with the person in charge of the new floorplan, who is also the manager for the DB guys here. (We're split into UI programmers and DB programmers here.) It was a great conversation: constructive, to the point, and resulting in action. It was wonderful. I've never had a conversation like that with management here.
The overall floorplan is excellent: DB and UI guys are going to be put together into little pods scattered across a single floor. Although this might not sound like a good idea, it's actually perfect: the isolation will encourage the formation of teams. Each pod has about ten people in it, which is just the right size. You see, one of our biggest problems here is MatrixManagement: part of that is the idea that when you were Created, you were Created as a UI Programmer or a Database Programmer... and Thou Shalt Not Fraternize with the Other Side.
This is a wonderfully subtle attack on that problem. I wish I had been responsible for it.
So I went to talk with the guy in charge of this today about PairProgrammingFacilities. First thing I did was tell him how much I liked the new layout. I did that partly because I figured that most people would be reacting with complaints about how their new area wasn't as good as the old one. Then I told him that with the current situation, it was hard to get two people in front of a computer for collaboration or pair programming. I gave a couple of examples of when I had been trying to work with someone but couldn't do so comfortably.
He agreed with me, and asked for some alternatives. So I sketched out some ideas that focused on getting rid of the interior walls in the bullpen and using big rectangular tables. He raised some objections, I agreed, we batted around a few more ideas, and finally came to a conclusion that should work. Meanwhile, we talked about the advantages of pair programming, ConvectionCurrentsOfInformation, and how people would react to the reduction of personal space.
I came out of that meeting feeling like I had actually made a difference. That's a first for this organization. I hope this conversation results in actual action. I'm going to follow up on it next week.
23 January.
"They want us to reduce development cycles and respond more quickly to changes," he said.
I smiled, knowing what was coming next. That's a textbook reason for instituting an agile process.
He continued. "That's why it's important that we get the framework right."
I was floored. I've been talking about agile software development for three or four months now. Along comes a classic customer request -- the exact request agile development was invented for -- and this is the conclusion?
I've clearly screwed up. My approach has been fundamentally flawed. Since he got here (which was before me), my "champion" has been advocating a framework as the solution to the company's problems. The framework's been developed. (It's not solving the problems.) I come along and advocate agile development and people-centric processes.
Of course I'm not getting anywhere. I'm not saying anything that can be heard.
Frameworks are making my life more difficult too - I'm also trying to introduce XP (see OptimisticProgrammingSkunkworks), and the biggest problems we have are with complex software like WebSphere, WSAD (which puts proprietary patches over the open source Eclipse IDE) and STRUTS. I'm lucky we're not getting pressured to use EJB. These are SilverBullets as conceived in the image of a bureaucracy. -- TomRossen
29 January. Remember the manager I had lunch with on the 15th? He's the one who told me to sit tight and be patient. Today, I ran into him in the hall and we talked for a few minutes. One of the things I asked him about was whether my efforts were making any difference. He said that they were, but at a level I can't see. He said it was "aligning the conversation." Come to think of it, I'm not sure what that means.
His statement didn't really make me feel better or worse. I have seen things change, and in the direction I've proposed. They've changed glacially slowly, and only partly in the ways I've suggested. But they're changing. What's less clear is that it's as a result of my efforts. Probably, yeah... I can see that. But the manager may have just been sweet-talking me. He knows I've been unhappy, and he's recently described his job as "talking people off the ledge."
Actually, I'm not sure what that means, either.
31 January. From conversations with a fewer higher-ups today, I got the strong impression that people think my ideas are not evolutionary enough. They think I'm trying to do too much at once. I had two people today convey basically the same opinion, so I think it's something that's going around.
It's true. I have been advocating fairly significant changes. Maybe it's impatience. I see so many problems that could be fixed if I were just handed the reins of a team and told, "go." (Judging from that sentence, there's probably some hubris to go along with the impatience. Now all I need to be great is a healthy dose of laziness.)
On the other hand, it's hard for me to be evolutionary. I've suggested evolutionary changes, and some of them have gotten implemented. For example, on November 12th, I successfully put a very minimalistic agile-ish process in place for development of the framework. Problem is, it was too minimal. There was no coach. People didn't know how to do test-driven development. It didn't work.
Another evolutionary change that I rammed through by sheer force of will is the process for the project I'm currently working on. The project is winding down, so a lot of the process has faded away. That made a difference for me personally, which was nice, but it didn't have any lasting effect on the organization.
Back to my point: evolutionary change is difficult because there's so many interdependencies among agile practices.
I think if I were more patient, I'd be satisfied with evolutionary change. I am making a difference, but it's imperceptibly slow. There's some sort of weird political shit going on. People keep hinting at it: it's gotten to the point where everybody I talk to agrees that my ideas are the way to go, but that even the director doesn't have the ability to make the changes required.
Maybe that's a just a way of getting me to shut up, but if it's true, it's a serious problem. The director isn't doing his job. Okay, okay, that's harsh... I don't understand all the intricacies of the situation, etc. But let me compare this organization to another I know of. At this other organization, the director saw that there were problems with development. He considered it and concluded that agile development was the solution, but he got resistance from executives. He worked at it for a while, but eventually declared, "I'm the director of development. We need to switch if we're to be successful. It's my decision to make, and if you won't let me do my job, I need to step down."
Guess what? That company is now several iterations into an agile development process. The pilot project has expanded into a normal project due to overwhelming success. Everybody is happy.
My current company has an organizational disfunction which is preventing it from making decisions. Everybody, from the director on down, recognizes that there are problems. The director himself told me there were problems the day I got there. Decisive action is needed, but nothing's happening. I don't just mean in terms of agile development -- I mean anything.
People in positions of authority need to stand up and say, "We need to do this, this, and this, or we're going to fail. Trust my judgement and let me do my job, or accept my resignation."
It may be getting close to time for me to do the same thing. Today I was formally put in charge of a small project. It's due in May (but requires a full waterfall lifecycle), and is the first of seven related projects that require close communication. I'm worried about the risks on these projects. They require close communication, but don't have it. I formally raised the risk this evening. I said, "These projects have a real chance of failure with our current development approach." We talked about the difficulty of making change. I continued, "Failure wouldn't necessarily be a bad thing. It would serve as a wake-up call. The worst thing that could happen is that these projects succeed, but at great personal sacrifice."
Too extreme? Too close to an ultimatum? Maybe. Last time I got in this mood (7 Jan), I wrote my comprehensive process proposal. That document was a failure: too aggressive.
3 February. Whew. Where to begin?
Last Friday (previous entry), I formally raised the risk of failure on seven linked projects due this year. This morning, I had my bi-weekly meeting with my boss, and I raised the issue with him as well. I told him that the projects needed a single leader: someone who could co-ordinate all of the different people involved. He asked me if I wanted to do that. I told him I was concerned about the organization not being set up for the projects to succeed, and that I didn't want to lead a project unless it could be a successful one.
So... this afternoon, the architect in charge of those projects approached me. He told me that had re-read my process proposal over the weekend, and thought it could be done. (The one I just recently dismissed as "too aggressive.") He asked me to lead the seven linked projects. I said I was interested, but I was concerned that I wouldn't have the organizational support I needed. I asked him some questions: would we have a bullpen; could I do iterative development; would developers be assigned to me full-time; what kind of leeway would I have; and then told him I would think about it and let him know more tomorrow. But before he left, I told him how much I appreciated his considering my ideas.
Later that day, he introduced me as the tech lead for the seven projects. Well, okay then. :)
I should be thrilled about this -- I have the ability to make a much bigger impact than I could before. Now I can demonstrate my ideas rather than just talk about them. But I really am concerned about whether this project can succeed. I'm concerned about organizational support.
On the other hand, it's a golden opportunity, so I'm going to embrace it whole-heartedly. I'll start moving in an agile direction immediately. I'm not going to be aggressive about it, but I will be firm. Why? Because I think these projects will succeed or fail based on communication, refactoring, and schedule. I need to establish practices that enable these three things right away... not because we need to get started with them right away (although we do), but because I need to know if those practices can even be put into place. Basically, I'm trying to find out what the resistance is going to be to the practices I consider critical.
One thing's for sure. Work just got a lot more interesting.
Some notes about why I think I got offered this job:
One thing to say: CONGRATULATIONS! :) Well, it beats beating your head against a wall, don't it? Remember this moment, and what you said ("I didn't ask for the job"), if (and when?) you start to feel that you're in over your head; you didn't ask for this, they asked you. You didn't misjudge the situation and get yourself in too deep, they judged the situation and determined that they want you in charge. So take charge :) --WilliamUnderwood
Sounds incredibly familiar to the situation that we have been facing over the past few months. We are a bit ahead of your timeline here, in the midst of our second XP project. Unfortunately, we have started to see the walls shrinking in on us.....our lack of Executive support has begun forcing us back into the Waterfall planning approach and status reporting structure. Old habits are hard to break, and due to the cross-functional nature of our project, we are being slowed by external stakeholders/resources necessary for our need to "Go Fast". Best of luck moving forward! -- ScottFerrell
5 February. I've done it. I've changed my organization.
I've been looking forward to saying those words ever since I got back from vacation. The funny thing is, I was planning on following up with, "and I like my new job much better!"
But I haven't found another job. It looks like I may have actually succeeded in changing my current organization.
Okay, I have to admit: This declaration of victory may be premature. But things have never been so good. I spent most of the day talking to management and going over my comprehensive process proposal item by item, getting buy-in on each point.
Yes, we'll have a bullpen. Yes, we'll have two-week iterations. Planning game. Test-driven development. Collective code ownership. Pair programming. Continuous design. Yes. Yes. Yes. Yes. Yes.
Wow.
I was this >< close to leaving. I had gotten all the way through the interview process with one company, and was getting overtures from two others. I've been privately committed to leaving ever since January 15th. I was fed up, burned out, pissed off.
Some of that came out. I let it slip to a few people that I was on my way out the door. Maybe that's part of why things changed. I'm not really sure. I'll ask about that once things settle down.
Today, I formally agreed to run the linked projects. I called the other companies and told them I was no longer available. Tomorrow, I'll go find all the people I'll be working with, explain the new process, and find a place for us to sit.
It's going to be hard: the projects I'm running are the highest-visibility in the company. The client is difficult to work with. The programmers have a lot to absorb. The development environment is bizarre. This is a huge challenge.
I can't wait.
Yeah!
Continued in ChangeYourOrganizationDiaryPartTwoFebruary.
EditText of this page
(last edited June 29, 2004)
or FindPage with title or text search