Bugs Arent Just Stories

Bugs show that our process of producing software or our thinking was faulty. This deserves separate treatment and analysis.

Stories are a chance to improve our software.

Bugs are a chance to improve our ability to produce software.

One of my favorite windmills: that so many fail to recognize the differences between a testing group and a QA group. -- DanilSuits

So don't just sit there - tell us the differences!

One take on this is that monitoring what a testing group does is just one of the things a QA group would do. While a testing group would determine whether some code is working or not, a QA group looks at wider issues of release-readiness or conformance to requirements or overall goals.

Another take is that QA should monitor quality everywhere in the organization. Quality is the responsibility of everyone, it can't be added on during the testing phase. (slightly off-topic: Can anyone relate an experience with a cross-functional QA team that is responsible for, e.g. monitoring testing progress, development process, etc?)

What about problems that the customer wants fixed Right Now, rather than at the end of the next iteration? For example, everything worked fine in the test environment, it was delivered, installed, and accepted, and developers started on work for the next iteration, but then it turns out there is something different about the production environment that causes a serious failure. The developers believe a patch could be tested and installed within 24 hours. Waiting until the end of the next iteration would have substantial costs (because the system is not available to do its work), and there will be no useful feedback from the users until the system is running. Is there a way to incorporate such immediate fixes into the XP model?

I see no option but to go back to a known-stable version (note: this may not be the same as the one the customer is running - there may have been uninstalled iterations in the mean while) and make the change there. This means the current iteration will not be installable by the customer, unless you change the plan to re-fix it there (normally not a good idea?). The next iteration plan should include re-implementing the fix

Why isn't the patch the next iteration? Make the change, bump the version number, deliver it. The cycle is the same, let someone else worry about whether you call it a patch or release.

Because

a) you don't know if it's stable. Remember: the fault you're fixing is evidence your testing isn't covering everything - a fault was revealed only in the production environment. Stablity can only be assessed after the release, not before it.

b) You may be in the middle of some set of changes and not able to release it *now*.

c) it contains more changes than the customer is expecting, and there may be other things that need doing to install it that have nothing to do with the patch - e.g. db schema migrations, different screen layouts.

a) You don't know if it's stable. Why not? Why believe or accept the system should be "unstable" during development. Also, you cannot extrapolate a single fault into telling you anything about the remainder of the system.

Have you actually made this work? If so, then your experience is very different from mine. I find that stablity really can only be measured, not predicted. I find that live environments really do normally differ from development ones. And a single fault tells you your testing is not reflecting production performance. Rather than sticking your head in the sand and saying it was just an unfortunate one-off, I think you should be looking to see why the testing didn't catch it, and if it's a sign of a generic shortcoming in your testing approach.

b) You may be in the middle of some set of changes ... I have found that by carefully scoping and partitioning your work, you rarely need to be more than 1 day away from completing some work. A lot of benefits derive from doing incremental, but complete, changes.

"Rarely" is not good enough if the customer wants it right now. So you need a way of going back to a previous release for these situations (an ex-colleague once worked for an organization with a contractual obligation to supply fixes within 24 hours - which they did successfully, by following the practices outlined above).

I don't believe that UserStories can be broken down so that they can be atomically implemented in the necessary timescales to do this. And half a user story is just a pain for the customer to deal with.

c) It contains more changes than the customer is expecting ... This is a marketing issue. Go ahead and give the customer something extra in exchange for having had to live with the original problem. If you have been doing incremental development, you should be fully prepared to roll out the full set of changes, including database migrations, file conversions, whatever.

And training the user base? What if there are 1000 of them?

If you have screen layout changes that are not readily understandable to your current users, I think you have a deeper problem than delivering a patch.

No, you have an evolving system - maybe more functionality is being added, and you're only half way through the changes (not all of which are software - perhaps the datafeed's not online yet)

It is really kind of silly to say that an iteration is planned to take this long, so we cannot react and deliver something earlier. Between patches, trade shows, key customer demos, etc., there is always a good reason to have the latest version of software ready to go.

It's not at all silly. I think you underestimate the difficulty of rolling out changes to live systems. If the changes are significant, it takes significant planning, and your proposal basicallty requires the customer to be ready to deal with an arbitrary subset of whatever changes were planned for the next iteration.

Again, have you seen this work? At any arbitrary point, a fix can be dropped into the current development stream and immediately released? I don't believe it, no matter how extreme you've got.

CategoryBug CategoryStory