What About Testing

Testing is a subject very dear to my heart and, whilst I can see that XP offers a fascinating programming ethos there seems - IMHO - to be a large gap in the thinking of the role of testing. I agree with Kent Schnaith (another Kent?) when he says that "There seems to be some confusion about the roles of QA and of testing. These are two very different things.". True. Generally testing is ensuring the system is being written right, whereas QA is about ensuring that the right system is being written. I will attempt to restrict myself to testing issues, but unfortunately the scope of my use of the term 'testing' creeps into the QA arena.

To make unit testing an integral part of the development process is a massive leap of quality in the right direction. But it is not enough. To polarise it into unit (techie) and functional (user) testing is to misunderstand the very real contribution that can be made by testers and testing throughout the development process.

Jen Wu said: "Some background ... a sophisticated QA team will do most if not all of the following (among other things):

To these I could add:

In my experience with RAD style developments, I have encouraged developers to 'own' the unit/module testing process and all white-box testing (much of which comes with the modern development tools). I have also maintained a need for an independent test team as part of the project to assist the users (bless them) with their testing requirements, planning, specifying and writing the black box oriented tests from day one of the project. Such a team can offer real benefits to the developers (giving a more coherent and technical description of user/customer aspirations) and to the user/customer (explaining what the developer has just said). Where RAD projects have dispensed with most formal testing techniques the ultimate success of the resultant code becomes a matter of luck rather than good code engineering. You can only get away with this approach for so long.

David Brady put it well when he said "If you have a good QA team, it's easy to get lazy and write lots of hopey code, because if it doesn't work, QA should catch it. In a better world - one I believe *can* exist, but so far haven't seen, and therefore am trying to create - the developers write the Quality, and the (good) QA team does the Assurance." XP offers the prize of quality code being generated first time by developers rather than the traditional 'sling it over to the test guys' approach that has fundamentally crippled so many projects. Testers can then ensure that the product will be more acceptable to the users/customers by feeding back user-related concerns before the user/customer gets sight of the application.

I also agree with Malte Kroeger when he says that "... what I think is most important about a QA-Team is, that it stays in very close contact with the development team." In a way I see pair programming as an implicit form of 'testing' in itself, but independent testers seeking to produce quality black box tests will often ask the question of the system design that the designers and developers hadn't thought of. So they are independent but working very closely with the developers. As Malte continued, "QA-Team members often know much better what to test. What problems usually occur. How the tests should look like to find the most likely bugs". We bring a wealth of experience, use of tools, and an enthusiasm for testing that can only positively impact on any project.

Before everyone throws their hands up and shriek 'what about the cost?!', the case has been made that pair programming improves quality and drives down overall costs, professional QA/testing has a much longer track record in the industry of doing likewise. Imagine the impact and power of properly implementing both! 

 ColinDavidMiller
 Senior Test Specialist
 (One of those with the particular demonic twist of mind
  required to destroy all your happy little programs ...)

If TestFirstDesign produces an order of magnitude fewer defects (an hypothesis someone should test), then the value of many of the activities recommended above drop dramatically.

Because TestFirstDesign and the activities I recommended above are not mutually inclusive. Testing a unit of code does not necessarily test whether the user story is correct in the first place (and does not contradict other stories), does not necessarily verify that the HCI is usuable to the business, that the business process is correct (writing the system right versus writing the right system) or prove the application's performance under any kind of loads. Unit tests do not reflect user story functional coverage or offer any kind of independent review of the code written.

I would argue that the value of these activities are assured whether unit testing happens or not - hopefully their role should be to verify what a wonderful team of XPers you have ... but then again (cdm)

XP doesn't reduce the value of QA as much as change it. Without XP, QA can get overloaded with basic testing duties -- validating that a build works. XP allows the QA people to do more in-depth testing: load tests, hardware compatibility tests, etc. They can also look at quality issues such as ease-of-use. This makes sense because XP is an automation technology; and these typically free people from repetitive tasks. Bank clerks no longer have to worry about doing arithmetic on deposit slips and can focus on serving the customer. --IanRae

EditText of this page (last edited July 17, 2002) or FindPage with title or text search