What Happens When We Try

What happens when we try a method and experience failure, or at least do not have the success others claim for it?

The criticism of a method which has been tested and found to fail, says more about the test than it does of the method. Failure can occur from the extreme of not conducting any definitive test, to the other extreme of exhaustive definitive testing of all facets, which is not practical for the reasons of expense and time. Since testing usually carries some little component of bias, the results of tests often merely confirm the tester's bias. Proponents will say as above "they tried x and they failed" and opponents will say "we tried x and it failed" The unbiased might say "we tried x and were unable to duplicate the successes expected". The proof of XP will be in the completion of successful projects with workable, satisfying results.

It would seem that proponents might sponsor a page "XpProjectSuccesses?", while opponents would counter with "XpProjectFailures?". From these, the unbiased might be able to explain the reasons for success or failure of the project, from the standpoint of all the factors including not only the method, but the personnel, the extent to which the method is supported and encouraged by all facets of the client's organization, and by the analysis of all the factors bringing about the result.

To actually get a scientific study of the applicability of processes and methods, such as XP, we should at least try to use modern ScientificMethod. Due to methodologies being a social phenomenon, but also an economic one, the two most likely candidates to yield good experimental procedures would be Sociology and Economics.

In general, I think that if there is ever going to be a valid result we first need to have at least one reproducible experiment, written up as a paper and submitted to a peer-reviewed journal. Whether such a journal exists is unknown to me. I'd be happy with submitting it to Wiki. For this kind of science, PeerReview is the essential mechanism for establishing good theories.