Shodan Input Metric

Shodan 1.4 Input Metric

See XpEvaluationFramework for the latest stuff. We've had these metrics around long enough to complete some studies to show the effect of XP.

BillKrebs

How eXtreme are you? At XP Universe 2002 I heard the following:

They are all right!

Putting that all together, I came up with ShodanInputMetric. It is a 5 minute survey that helps teach people XP and also helps you tell if you're out of balance.

It's named in honor of BobMartin who was kind enough to demonstrate his Jujitsu skills at XP Universe 2002. Shodan means 1st degree black belt - or someone who is just ready to learn.

I've collected data for my small team's project over the past year. In Comparing two releases I found we improved our defect injection rate by a factor of two when we moved from 57% XP to 72%. More importantly, I had a lightweight way to measure that does not bog down a small agile team yet gives us data we need for CMMI, ISO, or formal studies. Since we don't trust either subjective gut feel or detailed numbers we measure both so they can cross check each other. Sometimes the detailed numbers can make it hard to see the forest through the trees. The survey does not corrupt my data, it's an integral part of it.

You can read more in our NC State University Technical reports at http://www.csc.ncsu.edu/research/tech/reports.php Specifically, you can read the output of the study at ftp://ftp.ncsu.edu/pub/unity/lockers/ftp/csc_anon/tech/2003/TR-2003-18.ps.Z. You can read details on how to do it at ftp://ftp.ncsu.edu/pub/unity/lockers/ftp/csc_anon/tech/2003/TR-2003-20.ps.Z. I'm working on a set of more readable power point slides to discuss the concept. The papers are a little hard to read.

The survey is based on my XP Universe 2002 paper "Turning the Knobs: A Coaching Pattern for XP through Agile Metrics" by William Krebs in "Extreme Programming and Agile Methods - XP/Agile Universe 2002". DonWells, LaurieWilliams (Eds.), published by Springer 2002 (lncs2418).We reviewed some of the ideas at the XP Universe 2003 Data Workshop were many people offered very helpful ideas (http://sern.ucalgary.ca/eeap/).

You may also want to backup the survey with numeric metrics. I suggest "A Metric Suite for Evaluating the Effectiveness of an Agile Methodology" by Williams, Succi, Stefanovic, Marchesi in "Extreme Programming Perspectives" Addison Wesley 2003. The CK Metrics referred to there seem especially useful for sniffing for bad code smells and seeing if you've achieved SimpleDesign. hackystat also looks interesting (see http://csdl.ics.hawaii.edu/Research/Hackystat/, PhilipJohnson)

The 1.1 number on the Shodan metric lets us tell if we've changed the questions incase we want to compare past data. Note that the weightings are derived from the practice dependency diagram in Extreme Programming Explained by Kent Beck from the diagram on page 70.

The following numbers help you decide between 1 and 10:

The numbers in parens () are weightings. To compute your overall XP score multiply your team's average for each question by the weighting, add it up, and divide by the total possible. For example, if my team's average is 7.5 for automated tests than I multiply 40 times 7.5. I sum these products for all scores, divide by the total of the weightings (640) and ta da! out pops a percentage showing how 'extreme' my team is. The weightings are in there based on Kent Beck's diagram on page 70 of Extreme Programming Explained (Addison Wesley 2000). The diagram shows the practices are interdependent. I counted the number of lines feeding into each practice and multiplied by 10 to get the weighting. There was too much meat in the test practice, so I split it out.

My team's averages have been 70%, 59%, and 63%.

Foundations

Customer Planning Teaming Craftsmanship Introspection

My Problem with this approach is that it measures conformance to a method, but not the benefits of following that method. Does anyone have a good sample of data relating these measures to value measures- - such as quality, productivity, total cost of ownership etc? --JasonGorman

CategoryMetrics

EditText of this page (last edited August 1, 2008) or FindPage with title or text search