No Process Discussion
No Process Discussion[[This page contains the original discussion from NoProcess,which is undergoing some refactoring. -- CostinCozianu]]
Why does anybody need ExtremeProgramming, RationalUnifiedProcess and the like?
I mean why do you need a defined process with all kinds of prescriptions, superstitions, discipline and myths and all the goodies?
(Assuming that is a real question, there is a real answer, or six at least. From AgileSoftwareDevelopment, p 170: 1. Introducing new people to the way people work around here, 2. Substituting in new people for previous ones, 3. Delineating responsibilities, 4. Impressing sponsors, 5. Demonstrating visible progress, 6. Curriculum for ongoing education.)
For every recipe given in all kind of methods ranging from XP to RUP to WaterFall there's plenty of examples where it just doesn't make enough sense to even try it in practice, and there's plenty of practical examples of projects succeeding following a path contrary to what is preached in any particular approach.
Of course it doesn't work for everybody. Some people are more productive if they follow any of these processes. But some people are a lot more productive if they do whatever they see fit for each specific context:
But to really be effective, you've got to avoid thinking as much as possible: this gives you time to think about the really important things. The standard examples are things like driving a car or even simple walking. MuscleMemory lets you do the basic steps without thinking, leaving you to worry about more important things ("skid marks on the road - someone had a problem here, lets be careful"). A process is an organization's MuscleMemory! -- DaveWhipp
Hasn't anybody around here worked on successful projects with NoProcess at all? At least everybody must have seen it in practice somewhere, regardless whether he was directly involved and whether the effort was successful or not.
I know I did. I just love it.
It works like this: you gotta do what you gotta do.
Requirements, architecture (only if an architect exists and it so pleases him to do architecture), coding, a bit of design, a bit of database, a bit of testing, QA, documentation, a bit of politics with the management only when it's needed. All these activities are science and art in their own right and you need a lot of knowledge, flexibility and common sense to see them through. What you don't need is a preachy approach: you gotta do it this way. You need is to have a bunch of people that are good at what they're doing, have fun together, and take pride and responsibility in their work.
And forget about ProcessImprovementTools. The only things that needs to improve are the skills and knowledge of each individual and the team as a whole. -- CostinCozianu
Concerning the preachy approach, while there are some who preach their approach as though there is no other way on earth, most just offer helpfully what worked for them (and yes, sometimes what they think will work the next time as though they had already tried it - watch out for those!). However, many software professionals I have worked with cannot understand the concept of a guideline. They think in terms of mandates, and so misinterpret the spirit of helpful advice.
I've also noticed - I swear this is true in every case - that the people who make the biggest stink about creeping bureaucracy as lowly programmers are the first to levy rigid mandates to their way, once "in office". No wonder they were so scared back then. They fantasized that they were working for themselves.
No process, no bullshit, just getting the job done. Sounds good, let's do it for as far as it takes us. But when the individual skills and talents fail to automatically coalesce into the ideal project performance, you will find programmers and managers alike discussing process, perhaps without using that word. It's a natural transition. So be careful to distinguish between "no process" (an unlikely state of affairs under any circumstances) and "no awareness of process" which seems to be the current state of the practice for many groups.
I'm not saying that it works everywhere. I'm saying that I know for a fact that it works, from my own experience and the experience of others. Since everybody talks nowadays of just in time design, refactoring, the source code is the design and the likes, why wouldn't all these be applicable - in certain situations - to the process. The source code is the process (if the product is good who cares about the process), just in time process. Now, of course when things doesn't work you have a problem. But the bulk of the problems in SoftwareDevelopment, are not about the process - any particular approach can prove successful, you can pick one or mix and match at will - , the bulk of the problems are with the crappy technologies, the generalized lack of education on ComputerScience fundamentals, and with the personal responsibility of the people involved at all levels.
The problem is people are less likely to admit their own fault, and instead will try to focus on process difficulties and challenges, supposed technological complexity, and play around with all kinds of fancy words. That's why having a NoProcess process is going to help. It gets the crap out of the way, and focus on personal responsibility, knowledge and mastering of the craft. If you don't do what you have to do or you do it unreasonably slow, or you do it crappy, all the processes except NoProcess offer plenty of opportunity for excuses, for example:
Just try to see how many UnitTests were written by DonaldKnuth and how many UML diagrams were drawn by the same author to see how much he has in common with either XP or RUP, the current fads of the day. Compare that with the quality and importance of his programs. Yes, of course, he is a bloody genius, and we don't have teams made of geniuses in everyday life; we don't have to work on projects like TeX or CWEB either. But then what are we supposed to do? Try to focus on individual improvement and learn a lot more on TheArtOfComputerProgramming, or should we hope that following processes like TestFirst, PairProgramming (or the equivalent RUP, agile, whatever else) is going to make up for our inherent deficiencies and our natural inclinations to do crappy work? -- CostinCozianu
Knuth invented LiterateProgramming. I'm pretty certain he was trying to say something about how software development should be done. In particular, that we should "write the code for the benefit of the human reader, not the compiler". In the 1983 article (http://www.literateprogramming.com/knuthweb.pdf) he explicitly bills LiterateProgramming as a "methodology", offered as a complement and furtherment to the latest advances in "methodology" - StructuredProgramming. That last paragraph strikes me as analogous to dismissing QuantumMechanics as nonsense on the grounds that Isaac Newton never dabbled in them, and he was a bloody genius, now wasn't he?
No, I think the last paragraphs was not dismissing anything in particular, but was trying to say: it can be done in thousands of ways, as long as you know what you are doing. And the focus should not be on practicing (or preaching ) of any particular process, but on evolving that necessary knowledge and art. Within this knowledge we might want to include several things from very different methodology, and apply them accordingly to each particular task at hand, and according to the personality of each individual.
NoProcess might work fine for small projects of two or three people who work well together. But once you have more people than that, or have to bring a new person in, then you need to have some "rules" and "responsibilities", and then you have a process.
"Do what you've gotta do" is fine, but how do you know what you've gotta do? Are you asking the customer what they want, and figuring out how to implement it? Then you have a process. Are you keeping track of bugs, prioritizing them, and assigning them to people to fix? Then you have a process. Do you always check code into the source code repository before doing a production build? Then you have a process. If you are out of the office, is it possible for someone else to fix a bug in your code? Then you have a process.
The benefit of formal process models is the sharing of information about things that work, and things that don't work. Everyone who decides to "buy in" to a process needs to accept the responsibility of evaluating that process in relation to their own work. It's certainly true that following the rules of a formal process without understanding it or thinking about it can lead to bad outcomes. But that is your fault (or your bosses' fault), not the fault of the authors or proponents of the process. There's only one thing you always gotta do, you gotta think.
-- KrisJohnson
I think we can agree that the only certain thing we gotta do is we gotta think. Where I see it differently is that I don't think that you have to think one time only in advance, and buy-in for a process. And to try to explain more clearly, I see processes as packaging of totally unrelated techniques that address different concerns for the purpose of selling the whole enchilada.
Let's see what someone usually gotta do:
And they don't tell you where their choices work, where they don't and why is so. It's only packaging. Like a salesman for a Dell server who goes to the CTO of a telecommunication company and says "Here it is, designed specially for the telecommunication industry", while it is only a stupid x86 architecture. Or designed for "small businesses". Bullshit. There's no technical or economic reason that if you're running a small business you should suffer with WindowsMe? and 128MB of RAM. The salesman is selling you a label, lying to you that the thing under the label has been carefully designed to fit your needs.
Likewise, "formal" processes are packaging of a very narrow set of ready made choices under an attractive label. Obviously, the ready made choices will fit to some problems, and will suck at others. Where processes and methodologists fail by a huge margin is to acknowledge this very basic thing. Where the process is going to work, and where it is not the best choice. Every one pretends his favorite scheme applies everywhere, nobody will tell you where it doesn't and, surprisingly, they are all sincere. They honestly don't know, and it's not in their interest to find out. The lack of knowledge suits them better, like a lawyer who doesn't want to really find out whether his client did or did not commit murder. Not knowing, suits their purpose better. -- CostinCozianu
RUP is a commercial process that is packaged and pushed hard by its authors and sales force. But the same doesn't to XP or many other processes. There are a few books out there on other processes, but few of them make a claim that they are all things to everyone, or that there are no possible variants.
[[Well, XP, by the voice of KentBeck, tells us just that, see the quotation above that I got it from ExtremeProgramming.]] The preface to Kent's book starts with "This is a book about Extreme Programming (XP). XP is a lightweight methodology for small-to-medium-sized teams developing software in the face of vague or rapidly changing requirements. This book is intended to help you decide if XP is right for you."
And because I'm no methodologist, I think in my ignorance that the cool thing to do is to have NoProcess. That is you assume no predetermined recipe. No XP, no RUP, no nonsense. You make the best choices, for each task that you need to get done to finish the project, as opposed to flipping a coin at the start of it to see if it's XP or RUP (or add your favorite here).
So you're saying that NoProcess comes from ignorance, or from people who know nothing about methodology? No-one will argue with that. And if you are flipping coins to choose your processes and methodologies, it's little wonder that you've been unimpressed with the results.
No, I'm not flipping any coin, I'm not a project manager, nor am I able to influence such a decision. The only thing that I can tell is that NoProcess worked wonderfully in some projects that I've been with. I was trying to suggest that the means one can use to decide on whatever process are no better than flipping a coin.
NoProcess comes not from ignorance but from people who refuse to "buy-in" to one size fits all solutions. It is the lousy work of methodologists that makes me think about NoProcess. Their work is always orthogonal to the science supporting every particular engineering task, but they have the latitude to decide how every task should be done. I'd rather rely on more substantial things, like requirements engineering, data modeling, ComputerScience, each of which offer enough support in every engineering task rather than follow methodologists.
The thing you can be sure of, is that there is no "one size fits all" solution. There are only "one size fits all" processes/methodologies/hypes. So to respond to your essential question: but how do you know what you've gotta do?, well there are very good books on requirements, design, coding, databases and all the topics that I need, then I make my best shots, keep a critical eye on myself, learn from mistakes, build experience, challenge some presupposition and the cycle goes on and on. But if I want to end up with the least knowledge on how, all I have to do is to follow a process.
That doesn't answer the essential question. The essential question is how do you know what you need to do, and how do all the other people in your organization know what they need to do? Whenever multiple people are working together, some common rules need to be followed - you can't be asking each other every few seconds who is doing what and how they are doing it - and there needs to be a policy for assigning work and for letting everyone know enough about what everyone else is doing that no one does anything counterproductive. There is no process that is intended to tell you how to design systems, write code, use databases, etc.: the point of a process is to try to make the activity more monitorable and the outcome more predictable.
Why is it that you want to read books to learn "software engineering" tasks, but want to make up your own process without any study whatsoever?
Because when I read book on processes, I get bored and waste my time, and I think I'm better off without it. Or I haven't seen the right books and the right methods, please give me some suggestions.
I would totally agree with you if what you asserted here was true: There is no process that is intended to tell you how to design systems, write code, use databases, etc.: the point of a process is to try to make the activity more monitorable and the outcome more predictable.. Are you sure about that?
But unless I am terribly wrong, or I don't interpret English correctly, all of them tells you how to do requirements, how to do design, how to do every little bloody thing (and equally how not to do it). Do I need to go into details to convince you? I enumerated some earlier on this page. Every little process that I know of limits my ability to choose what is the best fit for each engineering task. Requirements, Design, Coding, you name it.
Let's take an example. Let's say I am in charge of requirements and database design for a project. And let's say that in my judgement given the problem at hand, the best way to do it is to approach the requirements from a business rules perspective, produce a little bit of documentation on that, do some conceptual modeling and derive the database design.
If I was in a RUP environment, this already would be outside the process. I would have to worry about Use Cases, Use Case Models, Business Object Models, Business Use Case Model and a bunch of others, none of which would fit the problem at hand. It's well known that object models and use cases are not all there is to it with regards to requirements (actually in RUP it would be "Business Modeling" and "Requirements"). With regards to conceptual modeling and database design, I would be outside the RUP as well, after all I had to do object models first, using their bloody UML, which is anything but useful for the mentioned tasks.
If I was to be in an XP environment, I wouldn't have any more luck at all. I'd have to do user stories, code test first, DTSTTCPW, evolving design, refactor mercilessly, obviously my approach wouldn't fit in.
The RUP and XP processes don't tell you how to determine requirements or how to design the system. Only skilled technical people can do that. The processes determine how you document the requirements and design, and ensure that they are communicated to the appropriate people, reviewed appropriately, and that some reasonable estimates of costs and risks can be made. Every system has use cases, object models, and all those other things you don't like, but if you have NoProcess then you don't write them down or discuss them with the customers and other developers (and may not even think about them). Sometimes everything works out fine when you do that, but often it doesn't.
Well, they don't tell you in the ultimate details, but they do tell you a particular way to follow, and that's fixed and immutable. If I can't convince you on this one, I can't convince you of anything. Not every project has object models because nobody established that OO is good for everything, not every situation you meet in requirements can be solved by use cases alone, not every QA problem can be solved by unit tests. It's as simple as that.
NoProcess doesn't mean no responsibility, on the contrary, it requires more responsibility and more trust within the team. It doesn't say that I won't write the requirements down, it means I don't start off with the prejudice that I have to do it the XP way or the RUP way, or any other pre-determined way. I don't know where you derived that NoProcess means CowboyCoding from.
Maybe you should have named the topic NoPredefinedProcess?, NoOneElsesProcess?, DefineYourOwnProcess?, or something like that. The title and the introduction imply that no process at all is something to strive for. And statements like "A man's gotta do what a man's gotta do", "just get the job done", and "UML is for girls" strongly suggest a cowboy mentality.
I was suspecting that my disregard on actual processes were only my ignorant rants, until recently I read the opinions of TomDeMarco and MichaelJackson, who at least are not as ignorant as I am with regards to processes, methodologies, organization and the whole enchilada.
So TomDeMarco says: As I get older and crankier, I find myself more and more exasperated with the great inflexible set of rules that many companies try to pour into concrete and sanctify as methods. The idea that a single method should govern even two different projects is highly suspect: the differences between projects are more important than the similarities.
Also MichaelJackson:
Failure to focus on problems has harmed many projects. But it has cause even more harm to the evolution of development METHOD. Because we don't talk about problems, we don't analyse them or classify them. So we slip into the childish belief that there can be universal development methods, suitable for solving all development problems. We expect methods to be panaceas - medicines that cure all diseases. This cannot be. It's a good rule of thumb that the value of a development method is inversely proportional to its generality. A method for solving all problems can give you very little help with any particular problem.
But have you ever read a book on a development method that said: This method is only good for class X?
[[Yes, actually. AlistairCockburn says, in SurvivingObjectOrientedProjects, "I present the following methodology outline for a medium-sized Production project in an industrial setting. The characteristics of such a project are the following: 20-40 people, 1-2 years duration, time-to-market is important, not life-critical, need to keep costs down and to communicate with present and future staff." That seems pretty explicit to me. In AgileSoftwareDevelopment, he goes further, and gives a name for each of several different methodologies whose target domain he describes. So the answer is Yes.]]
{{ It is already mentioned somewhere on the page, that AlistairCockburn might come closest in this regard. The question is whether the characterization of the method's domain is accurate and/or meaningful, because it describes secondary aspects of the project and not the problem. So the answer was not quite, MJ was referring to class of problems. It doesn't necessarily means he has to be right, maybe processes are better if they fit the characteristics of the project and not of the problem, but maybe not. }}
Thank you for the vote of half-confidence ;-). Whether you fully approve of my particular method of characterization, the fact that I do characterize the appropriate domain for a particular process negates any claim on this page that no methodologist at all characterizes the domain for their preferred processes. The fact that you insist on processes being characterized for problems, not project teams only means that you disagree with the characterization - the existence of a characterization. And the ensuing discussion should go onto a page called NamingDomainsForProcesses?, where we can argue whether the problem or the project is the proper basis for a process' characterization. The point being, I am a methodologist, and I do characterize the domains for methodologies, and RobertGlass is another who does the same, and MichaelJackson is another who does the same, and JimHighsmith is yet another. With four names on the table, I think you can drop the extremist "no methodologist" position at this point. -- AlistairCockburn
Both quotations are from MichaelJacksons book "Software Requirements And Specification - a lexicon of practice, principles and prejudices", which, by the way, can be read in one afternoon, and can be re-read with great pleasure many times after the first lecture, unlike any process book.
Now putting aside the two personalities, what they say is a matter of common sense. It would take strong arguments and maybe hard data to affirm the contrary. And although the time they said those is way before the XP and RUP and all the modern cool stuff, I'm afraid that there hasn't been much progress, if any. What process en vogue right now is closely focused on [class of] problems? Is it something that I'm missing here?
The fellow with the lovely rant above is responding to pre-canned silver bullet off-the-shelf process solutions, and has obviously never spent enough time with the CMM to understand what useful process discipline means. Meanwhile, notice the glaring waterfall offered as the basic stuff you gotta do. Lifecycle models were the predecessors to the modern notions of software process. Our friend needs to catch up by a few decades, but he's a process guy, no doubt about that! And a closet methodologist, too.
Well, if you think it's waterfall, so be it. I won't argue with you to see the difference, nor will I respond to your off the target rants. But I'd be curious to see what is your idea of modern notions of software process?
I haven't spent much time on the CMM, because I frankly don't have that time. I'm no methodologist, neither am I project manager to justify spending time to seriously study CMM and the likes. There's so much to study to be a better software engineer, that the last thing you want to do is waste time on processes and meta-philosophy on processes, like the CMM.
My superficial opinion (rant if you like, but I do have strong arguments to back it up) is that all the processes en vogue right now are exactly what you said: pre-canned silver bullet off-the shelf solution, or are they not? When I'll see anything at all that doesn't smell like that from a distance, I'll probably sacrifice some time to study it. So, considering that as long as one lives, one learns, I'm always looking for suggestions from people who know more than I - and that's what wiki's all about. Do you have any lead/advice for me in this area?
No, but if I did it would be to read the overview of the CMM in the time you would have spent creating your next version of this page.
It was presumptuous for you to assume that I haven't read the CMM in the first place. I feel like deleting this whole section. And what is the CMM, after all, The Holy Bible or something? But that's another story altogether. Have a happy new year, Walden.
It couldn't have been very presumptuous. There was enough evidence to draw the conclusion, but if it was the wrong conclusion then it was wrong. The CMM is a little like the Holy Bible in the sense that many more talk about it than actually read it. As for being authoritative, no. It's just a good starting point for constructive thinking on software process, precisely what you appeared to be asking for above, but of course were not. Naturally, I played it safe by declining to offer advice! Otherwise I might have fallen victim to your little game.
Let's not bicker after midnight tonight, okay? You have a nice page here. Look at how big it's gotten. You are expressing a very important and common sentiment about a glut of stuff that's supposed to help with software development but does not. I just wish you weren't throwing the baby out with the bathwater. Observing what you do without undue bias, and observing the results you achieve also without undue bias leads to helpful insight. That's the core of "software process" technology. There's much unconscious software process improvement going on all the time. Making that stuff conscious is a DoubleEdgedSword: (1) it competes with more visible work and subtracts from immediate visible progress, and (2) it lets you take control of learning curves and get where you're going much faster than the unconscious. Keep it real, my friend. Yes, reject all the shrink-wrap process garbage. But also, temper your "no process" mindset. The cure for indigestion is not to have your alimentary canal removed. The cure for mismanagement of software development is, likewise, not "no process".
I'll second that. We've all had that feeling of "Why don't they just let me do my damn job, instead of writing these pointless documents?!?" I've never followed any cookie-cutter methodology, and don't think I ever will (RUP seems particularly annoying). But studying those process models, documenting my own software development process and those of the teams I've worked on, and discussing them with others has been valuable. -- KrisJohnson
I think you got my intentions wrong. NoProcess is not against documents, it is against MandatoryDocumentTemplates. I hope now you'll see what I was trying to say. -- CC
I take the liberty to assume that KJ and WM brought here the perspective of project managers on this. I brought in the perspective of the software engineer, who feels much more comfortable when he knows that no predefined recipes prevents him from doing his work the best way he can. I'd rather have a project manager who trusts me more than the process. I understand it's pretty uncomfortable for a project manager to have NoProcess. But is it true that a predetermined process gives you real reasons for being confident? I assert that if you have a predetermined process, you should have more reasons to worry about, unless you're reimplementing almost the exact same thing over and over again.
I assert that the probability of a predetermined process fitting closely to the problem domain is very low. There's lots of different things in life and only a handful of processes. And each process has a fixed choice on how to do every engineering activity. Let's approximate that we have on average 4 core activities in each project (analysis, design, coding and QA - no order or dependencies assumed ), maybe add 1 to account for variations in different processes. For each of these there's lots of choices on how to do it, because you have different techniques and you usually use a combination of techniques not a single one. Of course the choices vary with each engineering task, but let's assert that you have 10 choices on how to do each task. Now here it comes your favorite process and pretty much fixes every choice. Would you care to estimate the chance of a good fit?
Well, you might say that the above extrapolation is exaggerated. Surely there's a bit of a correlation between the optimal choices for different tasks, you don't need to make the absolute best choice only a choice that is close to optimal, the optimality is also influenced by the team's personality and "cultural background". But the thing is, no methodologist will care to explain why his choice is optimal for exactly what type of problems, and where his process doesn't apply (I think AlistairCockburn is the closest to this ideal, but that's a different discussion). And going into a practical example to prove my point, both XP and RUP (and I don't know of anything positive in this regard) will give you sub-optimal choices for OLTP kind of applications with relational databases. Quite a big chunk of misfits. I'm talking here from practical experience, because in most projects that I've worked on I was trusted with data modeling and database design, and more important I've invested a lot of time in studying this subject extensively.
So what would make you comfortable, trusting that RUP and XP have a more or less predictable way of solving a specific task in a specific context, or having a software engineer whom you know you can trust and who would tell you: "Boss, I know how to do this and I'll do it by Friday."?
For example, if you had a guy like me on the team you would risk being told that UML is for girls, that is reasonable to draw UML for the upper management only, if required and only post factum, RefactorMercilessly doesn't apply to databases, UnitTests are cool but there's a bunch of situations where they're pretty much useless, there isn't anything useful at all on requirements engineering in either the XP series of books or the books with the Rational stamp on them, that there's a lot more to TheArtOfComputerProgramming than ObjectOrientation and lots and lots of other personal opinions that would make me undesirable with either an XP team or an RUP team. I'd back my opinions with plenty of high quality scientific literature on all of the mentioned subjects and with my personal experience.
Now the question is if you would trust a professional who will try to do his best as to his knowledge and applied to a specific problem, or you'd rather trust a process, and why?
The other thing that I'm curious if you can share from a manager's perspective is what's your process of "buying-in" to a process? How do you decide that RUP or XP or whatever else is good for you?
When it comes to this point, from my angle I see that RUP tries to infiltrate the mindset of managers, chief architects and in general people with larger influence and responsibility, while XP tries to get the mindset of programmers. And it all ends up being about politics, the developers will say "we want XP" (and politically they have very good reasons for that), while the upper management, architects, chief architects are likely to incline to RUP (and again, politically they have good reasons to want that).
But let's get back to MichaelJackson and ask where is the link between the specificity of the next project you want to get done and the process you want to buy-in. How do you know that the process is not a total misfit for your next project? If you have figured that out, which is easier, how do you know that is close to optimal? -- CC
The notion of NoProcess reminds me of those companies where there are no OrgChart?s or titles given to anyone, supposedly because this keeps the organization flexible and avoids political games between different departments. In reality, such an company has a hidden, byzantine organization that requires all of its members to be skilled politicians to get anything done, and the organization changes depending upon who happens to be in the office at the time. The real reason for such an organization is that it provides the company's owner absolute authority over everything, as no authority has been officially delegated to anyone else. The same goes for a software development organization where the managers insist that there be no formal process: it just lets the managers order people around without giving those people any way to determine how their work will be judged, nor any way to predict what the managers are going to ask for next, nor any way to discuss how the organization's performance can be improved.
Beautiful observation.
JerryWeinberg has (as usual) helpful and insightful things to say about this, in vol. 3 of the QualitySoftwareManagement series in particular (I've just finished it).
It comes down to the notion of steering. In a software team we have a system that functions in a certain way, and we want (and expect) it to produce a certain result. When reality turns out to fall short of our expectations (it always does), we may want to take into account how the system turned out to actually work, make some adjustments, and accordingly, revise our plans for producing the desired result.
We will not be able to make these observations or these adjustments unless we have a "theory" of how the system works, even an approximate one. So we invent a "process model", or a process for short. As with all theories, it is only in very rough correspondence with reality, but so what? It only has to be close enough to be useful in getting us where we want to go. -- LaurentBossavit
Practical men, who believe themselves to be quite exempt from any intellectual influences, are usually the slaves of some defunct economist. -- JohnMaynardKeynes
Ditto defunct methodologists? -- GeorgePaci
See also DefinedProcess, SoftwareProcess, CowboyCoding.
Costin asks a bunch of questions above, hitting on a handful of topics, which I've organized below with my responses. Perhaps KrisJohnson will use the same framework, integrating his answers with mine, extending the outline where necessary. -- wm
Manager perspective vs. engineer perspective
Managers and engineers certainly have different perspectives when it comes to software development, but they need not hold opposing opinions on the value of process discipline. Engineers get frustrated when they find they're working at cross purposes to each other. Engineers get frustrated when there are different quality criteria among them such that one set of engineers ends up shouldering the chronic quality problems of another. Some visible process stuff can help here. Managers are usually charged with the job of making commitments on behalf of their programming teams. Visibility into the development process makes it more likely that those commitments will be reasonable. Knowing what the team is actually going to do makes it easier for a manager to stand up to the criticism of a schedule, for one thing. Another benefit to a visible and discussed process is that it gives the manager an opportunity to address risks before they become problems. This all goes to making good commitments. Process is not a tool to be used by one faction against another (used by management against engineers, for example). It's a tool to get to cooperation.
It is best when the engineers are in control of the process. If managers have to mandate changes to the process that the engineers don't like, something is wrong. -- KrisJohnson
Trust engineers more than process
Engineers who are not trusted often come up with this false dichotomy. "Which are you going to trust, me or some process?" Since process is not separable from the engineers doing it, the question has no real meaning. It approximates other possible meanings, though. One of them is "Which do you trust more, a talented engineer whose practices you know nothing about, or a bunch of flunkies following some written orders?" This is a useful question for analysis, even though it has degenerated into discussing two extremes neither of which is the most trustworthy thing available. What would be more trusted is a group of talented engineers willing to do their work visibly. Is there something about talent and visibility that makes them mutually exclusive?
Process prevents best practices
The notion of software process has been formalized expressly for the purpose of pushing best practices into the mainstream, allowing less experienced engineers to learn from more experienced engineers (and managers) without having to endure all the pain of trial and error starting at square zero. When "process" prevents good practices from finding use, you have a bastardization of "process", a kind of dirty bathwater that needs to be let out. In its wake, good practices should assemble themselves into patterns of behavior that we might call "effective process". That's our baby.
Wrong process versus no process
In a healthy software management environment, the notion of the "wrong process" shouldn't be cause for much alarm. This is the point that most engineers with "process allergy" don't get. All processes are "wrong" in the sense that there isn't one that can't be improved on. Your job as an engineer or a manager is not to adhere to the written word, but to gently and positively respond to areas in which the process "doesn't work". There is a vital balance point between stability and adaptability that can't be defined in the general, it can only be felt on the project. A good process will disappoint the rabid innovator, because it won't allow him to burn project resources exploring a never ending stream of improvements. A good process will disappoint the lemming who wants detailed instructions for every step, because it will leave a certain grain of problem up to individual creativity.
To state this in another way: a formal process is not a magic formula that is supposed to guarantee success. Instead, it is simply the practice of studying and doing things that have worked well for you or others in the past when approaching similar problems. If those things are not working for you now, then stop doing them. But don't ignore what you've learned. -- KrisJohnson
"No process" is a phrase without applicable meaning to the living. Any sequence of state changes is a process, and nothing gets done without it. If by "no process" you mean "an open mind to consider even radically different process than what we've used before", then at least we have a topic for discussion. The question of "how radical a departure" is not trivial. People doing things radically new to them don't usually do very well until those things become familiar, at which point they're not radically new anymore. Since our consciousness is built upon distinction, we are biased towards noticing differences over similarities. Another way of putting that is that differences evoke a stronger emotional response, pushing them higher up in awareness. What seems like radical difference to you (the guy who has to deliver) may seem like a variation on a common theme to an indifferent bystander. So, the question becomes "are we allowed to throw away all normal ideas of process in order to survive on this unique new project?", and the answer is "Yes, provided you've scoped that from many different perspectives, including some indifferent ones."
"Undesirable on RUP or XP team"
Costin, I'm tempted to say that you'd be undesirable on any team, regardless of their awareness of RUP or XP, but that would be AdHominem, and cruel humor to boot. Let's not go there.
For you to be undesirable, that means there would have to be someone who does not desire you on a given project, and that hinges a lot on the squeamishness of the team, something we can't predict. Meanwhile, your pride in being an irritant is apparent, and your underestimation of humanity to appreciate that seems likewise so. Given that I had to lead a team chartered to do RUP, I'd want you on that team, especially if the rest of the team was deficient in CriticalSpirit (by my standards, not yours).
What is the "buy in" process?
In light of previous discussion, the question has little or no meaning. I'm aware that in process circles there's much talk about getting CEO's and such to "buy in to the process". This suggests a wholesale transaction to me, a large-grained shift in values, the kind of thing that makes silver bullets attractive, a paradigm shift, if you will.
I don't buy in to that crap. As for how to judge a method good or bad, I suspect the question is a little sidewise, intending to expose some brittle beliefs. I have nothing special to contribute on that. I anticipate my likes and dislikes, and sometimes substantiate them with real experience. I liked CMM when I read it because it acknowledged real frustrations I had. I liked it still when I used it as a reference to plan some requirements process stuff with my team, and still when that work put us into the limelight of "the best managed development group in the company". Still, there's CMM stuff that I won't do. Have I bought in, or haven't I?
To address the question of "How do you select a process model to use?", the answer is that you study your problem, look at the potential solutions, and make an educated guess about which one will be the best. Or you have a process model imposed upon you by management or the customer. But the process model you start with is not as important as how you tailor it for your specific purposes, and refine it as you get feedback about what is working and what isn't. -- KrisJohnson
Jackson on "misfits" and "process"
Jackson's book on ProblemFrames addresses a range of problems that software has to solve, and casts the software into the role of a "machine" placed in proximity to the problem such that the problem and the machine interact in a solution. The frames apply to that class of problem.
If you begin with a problem that sound like "there are vague requirements, software engineers, a budget and some time in the schedule; the customer wants a working software system, but the engineers might not produce one within those constraints that would be acceptable to the customer" then you will probably recognize a Control Frame as best representing that class of problem. If you then think of the software process as the software analog in the control frame, then you realize that without notions of intermediate state and intermediate products and controlled progressions from one state to another, the customer stands a big chance of being disappointed. Making that stuff visible is the leverage without which control is not possible, and that is the level at which software process operates.
The question of whether a given design method produces good control software is in a different scope. For a thorough discussion of the significance of clear scopes, read more Jackson.
See also XpIsDogmatic
EditText of this page
(last edited August 27, 2013)
or FindPage with title or text search