Combining Xp With Quality Assurance

Introduction

I have worked as a systems developer for the past 14 years. Many of the projects I have been involved in have been subject to strict QA procedures required by ISO 9000, DS/EN 4500 (Good Laboratory Practise) or GAMP4 for the pharmaceutical industry. With the emergence of Agile Methodologies, new and more efficient techniques are developed to cope with requirements as a moving target, and planning and developing quality software fast and efficient.

Agile Methodologies such as XP, SCRUM, Crystal Orange, Feature Driven Design and others have a proven success track record and are gaining ground in the industry. However, it seems that there is a trench between Agile Development and Quality Assurance. It is hard to find references to companies that have attempted that combination. Graham Wright [9] presents the first and only case I have seen where a company (http://www.WorkShare.com) uses the XP methodology and is certified ISO 9001:2000.

As I am working in industries that a wrapped in QA standards and protocols (Health Care, Pharmaceutics and Defence), I find the combination appealing. I want to do Agile Development, but I must convince management and customers that quality and predictability will not be compromised. This article will attempt to bridge Agile and QA and propose a roadmap for introducing QA procedures to XP with minimal modifications to XP.

HenrikThomsen

Is It Possible to Combine XP and ISO 9000?

There is a myth that Agile Methodologies and Quality Assurance are incompatible because agile equals chaos. The myth is easily defeated in Hillel Glazer's article [4] by the statement: "The CMM is tailorable, and XP is disciplined." If anything, the ISO 9000:2000 standards are more open to interpretation that XP, hence implementing XP would yield more predictability than ISO 9000!

The notion that Quality Assurance is bureaucratic and cumbersome is not a myth. Speed is an important factor in Agile Development, and QA procedures will slow the project down. But everything has a price; if the customers require that their subcontractors have QA (which they are most likely to do if QA is imposed on themselves), they must accept the additional cost and time that QA requires.

What Is QA?

HillelGlazer [4] makes an interesting distinction between Quality Control (QC) and Quality Assurance (QA). QC is product-oriented, and XP has a lot of focus on product quality with numerous practices aimed at product quality [1], such as "Run Acceptance Tests Often", "All Code Must Have Unit Tests", "All Code Must Pass All Unit Tests Before It Can Be Released", "Code The Unit Test First", "Integrate Often" and "When A Bug Is Found Tests Are Created." QC should not be confused with QA, which is process-oriented, meaning that QA makes sure that people are doing the right things and are doing them right. Lisa Crispin [2] makes an excellent point (among many) concerning quality; one must make a distinction between external and internal quality. External quality must be negotiated with the customer; otherwise we would attempt to treat all customers alike, which would make us look bureaucratic and inflexible. Internal quality must not be compromised because short-cuts will only give a short-lived progress on the project and add additional costs later on. Short-cuts also hurt staff morale, an XP team have the right to do their best job [1]. If procedures are seen as inefficient or unproductive, they should be dealt with as a process-optimisation. This simply means that any part of the process may be changed at any point (XP practise: "Fix XP when it breaks"), but there must be procedures that insure that the new, altered, process is followed by everyone in the project. In other words: Avoid improvisations (short-cuts), encourage optimisations (changes implemented by everyone).

When Is It Relevant to Combine XP and ISO 9000?

Agile Development and QA has a common goal: Meet the customer's expectations. However, the customer may have opposite drivers for Agile Development and Quality Management:

• Use Agile Development if you want maximum software for minimum money in a short period of time
• Use Quality Management if you want software that is well documented, and a manufacturing process for building that software, which is well documented.

A study [3] showed that XP combined with ISO 9000 gives a more mature organisation that XP alone. This is consistent with XP's focus on product and QA's focus on process.

Are There Friction Points Between ISO 9000 And The Agile Manifesto?

Yes there are! The Agile Manifesto [5] states that while processes and tools have value, the signatories value individuals and interactions more. Introducing QA procedures will put emphasis on the process more than individuals and interactions. QA also tends to focus more on documentation than working software. It might be tempting to see XP combined with QA as an enhancement of XP, making it more reproducible. Don't. QA and Agile Development has different aims, don't introduce QA unless it is a specific customer requirement.

XP+: XP Modified To Include QA and Documentation

Using the AgileProcessTool we can investigate a typical waterfall implementation of ISO 9000 at an IT supplier [6], and how XP practises fits in with ISO 9000 requirements [7]. As we can see from the overview reports, XP covers a lot of ground in terms of ISO 9000 requirements. In order to meet the requirements of ISO 9000, we can do pure XP and some more. According to Hillel Glazer [4], XP could be assessed at CMM level 2 (and thereby easily with ISO 9000) if we amend the following to the process:

• Ensuring that the XP Rules and Practices are taught to new developers on the project.
• Ensuring that the XP Rules and Practices are followed by everyone.
• Escalating to decision makers when the XP Rules and Practices are not followed and not resolved within the project.
• Measuring the effectiveness of the XP Rules and Practices.
• Providing visibility to management via appropriate metrics from prior project QA experience.
• Knowing when the XP Rules and Practices need to be adjusted.
• Having an independent person doing the above.

Graham Wright [9] maps the ISO 9001 requirements (specifically section 7.2 of the ISO 9001:2000 standard, Design and development) to XP practises:

• ISO 9001: 7.3.2 Design and development inputs. Inputs relating to product requirements shall be determined and records maintained. These inputs shall include functional and performance requirements.
• XP: Requirements are specified in customer stories and acceptance tests.
• ISO 9001: 7.3.3 Design and development outputs. The outputs of design and development shall be provided in a form that enables verification against design and development input and shall be approved prior to release.
• XP: Test first design both specifies design and verifies the implementation of that design. Tests are preserved forever and their results recorded. Release is dependent upon all the tests in the system running successfully.
• ISO 9001: 7.3.4 Design and development review. At suitable stages, systematic reviews of design and development shall be performed in accordance with planned arrangements.
• XP: Pair programming is continuous code review. Short iterations enable the customer to continual review functionality as it is developed.
• ISO 9001: 7.3.5 Design and development verification. Verification shall be performed in accordance with planned arrangements to ensure that the design and development outputs have met the design and development input requirements. Records of the results any necessary actions shall be maintained.
• XP: Acceptance tests detail customer requirements. Acceptance tests allow the customer to know when the system works and tell the programmers what needs to be done [1]. The success or failure of these tests is recorded. The verification by the customer that the story meets their requirements is recorded ("customer seen").
• ISO 9001: 7.3.7 Control of design and development changes. Design and development changes shall be identified and records maintained.
• XP: XP expects requirements and therefore design and implementation to change throughout a project. These changes are recorded in the stories contained in each iteration.

A modified XP process, XP+, is proposed. The goal in XP+ is to make as few modifications to XP as possible. In principle, there are only two modifications:

• The XP Tracker role is modified to XP Tracker+. This role does what the XP Tracker does plus monitoring and regulating QA procedures.
• The XP Tester is modified to XP Tester+, which does what a XP Tester does plus document designs and other artefacts required by the QA procedures.

The rationale for these modifications is that QA work should be integrated into the normal work routines. We don't want a new "QA worker" role. Furthermore, the XP Tracker and XP Tester is assigned new work, which requires skills close to the ones needed by these roles as defined in XP.

Automating XP

A way to ease the transition to ISO 9000 for an XP team would be a tool that automates and streamlines the forms and documents that needs to be written in order to comply with ISO 9000. Marco Melis et al. [8] have made a thorough normative study of ISO 9000:2000 and have pin-pointed areas that are candidates for tool support in order to manage XP under ISO 9000. The areas that should be supported by a project management tool are:

• Release plan
• Iteration plan
• Definition of relative responsibilities and authorities
• User stories and other inputs relating to products requirements
• Association of code to its user story and relative tests
• Documentation of reviews: Iteration meeting reports, pair-programming refactoring and test reports
• Acceptance test reports
• Restricted access to product validation to the customer (which is in charge of the validation in XP)

Roadmap From XP To XP+ / ISO 9000

• Use The AgileProcessTool or some other tool to model what XP practices you want to use against ISO 9000 required activities
• Decide on a process that suits your organisation
• Verify that the process meets customer's requirements for sub-contractors
• Verify that the customers are prepared to pay for QA and documentation overhead
• Make options for customers for external quality, never compromise internal quality
• If QA is not a specific requirement for your business, avoid it! Don't be tempted to see XP+ as a more mature process than pure XP. Trust XP to work, or use another process altogether.
• Agile Development requires Agile QA. The XP tracker+ is responsible for fixing the process when it breaks, and make sure that everyone knows and uses the current process
• Look at documentation as a system feature that must be planned and prioritised just as user stories. Let the XP Tester+ do documentation in the same pace as code is produced, don't postpone it or leave it to someone outside the team. You will probably need as many (or more) testers as programmers in order to keep the pace.
• Find a XP project management tool that supports tracking XP activities and integrates well with other project tools such as configuration management tools

Further Investigations

It should be possible to do XP under ISO 9000, at least in theory. I'm looking forward to taking part in, or reading about, actual attempts other than the one presented by Graham Wright [9]. I foresee that there may be a few practical obstacles in building the bridge between agile and predictable. In the possible wrestling match between agilists and formalists I foresee that the agilists will fight to preserve XP in its original form. And the formalists will try to introduce an analysis phase and reviews, which are poison to XP people. If anyone has experiences with agile development combined with QA, don't hesitate to add comments to this page!

References

[1] Extreme Programming: A Gentle Introduction, Wells, J. Donovan, http://www.ExtremeProgramming.org.

[2] Is Quality Negotiable?, Lisa Crispin, 2001 XP Universe, http://www.xpuniverse.com/2001/pdfs/Special02.pdf

[3] Combining Extreme Programming With ISO 9000, Jerzy R Nawrocki et al., The First Eurasian Conference on Advances in Information and Communication Technology, October 2002, http://www.eurasia-ict.org/ConfMan/SUBMISSIONS/81-ertpospnaw.pdf

[4] Dispelling the Process Myth: Having a Process Does Not Mean Sacrificing Agility or Creativity, Hillel Glazer, The Journal of Defence Software Engineering, Nov. 2001, http://www.stsc.hill.af.mil/crosstalk/2001/11/glazer.html

[5] The Agile Manifesto, http://www.agilemanifesto.org/

[6] PROCESSUS Overview Report, http://www.conexp.dk/docs/Processus.pdf

[7] XP+ Overview Report, http://www.conexp.dk/docs/xpplus.pdf

[8] Requirements of an ISO Compliant XP Tool, Marco Melis et al., Lecture Notes in Computer Science, Springer-Verlag Heidelberg, http://www.springerlink.com

[9] Achieving ISO 9001 Certification for an XP Company, Graham Wright, Lecture Notes in Computer Science, Springer-Verlag Heidelberg, http://www.springerlink.com

---

See CombiningXpWithQualityAssuranceDiscussion, QualityAssurance, QualityControl, QualityAssuranceIsNotQualityControl, QualityAssuranceIsNotResponsibleForQuality, ExtremeProgrammingQualityAssurance, IsoNineThousand

---

CategoryQuality