During my last Blog Predicting Testing Outcomes – Part 1 I talked about predicting Defect Detection Rates and this appeared to be quite controversial. In Part 2, I thought I’d touch on a related topic that was raised by several people – the coming into existence of the Defect (or Bug) itself. When is a Defect a Defect and when is it a perceived Defect?
What is at issue here is -at what point does a Defect come to life? At what point is it reasonable for a Defect to exist? During analysis, design, coding, testing or only once the software has been declared ready for consumption (i.e. the code is Live or in Production)?
Software development projects are typically full of creatively iterative processes and therefore while thoughts and ideas are being shared can we categorise something as a Defect? While someone is writing a specification and they misinterpret a process, do we have a Defect? Can a Defect exist before a document has been signed off as Version 1.0? Can a Defect exist before code has been loaded into a controlled staging or testing environment. Note: I see the majority of Development environments as sandpits and therefore uncontrolled; however, many organisations have staging environments where code is readied for deployment.
Why is any of this important? Because if we are to track (or predict) defects/bugs we have to agree at what point an event becomes a Defect. Over the past 10 years or so I have insisted that, the projects where I have been responsible for the Testing phase(s), specify exactly when an event can be classified as a Defect. The way I do this is to initially classify all unexpected outcomes as just that – unexpected outcomes. These situations are not classified as Defects until we have triaged them and agreed that they are indeed Defects. This means that an unexpected outcome can also be classified as a misunderstanding, an oversight, a user error, etc. Some may see this as semantics, but as I said in Part 1, semantics are very important in the world of QA/QC/Testing.
So, when I predict testing outcomes I am very careful about what I determine to be Defects.
By the way, a Defect can still be declassified upon further analysis or investigation, but it can never be removed from the record of the Project. It happened and therefore it must be recorded for future analysis or investigation when something unexpected happens.
Dateline: Tuesday January 14, 2014