Predicting Testing Outcomes (Part 2)

During my last Blog Predicting Testing Outcomes – Part 1 I talked about predicting Defect Detection Rates and this appeared to be quite controversial. In Part 2, I thought I’d touch on a related topic that was raised by several people – the coming into existence of the Defect (or Bug) itself. When is a Defect a Defect and when is it a perceived Defect?

What is at issue here is -at what point does a Defect come to life? At what point is it reasonable for a Defect to exist? During analysis, design, coding, testing or only once the software has been declared ready for consumption (i.e. the code is Live or in Production)?

Software development projects are typically full of creatively iterative processes and therefore while thoughts and ideas are being shared can we categorise something as a Defect? While someone is writing a specification and they misinterpret a process, do we have a Defect? Can a Defect exist before a document has been signed off as Version 1.0? Can a Defect exist before code has been loaded into a controlled staging or testing environment. Note: I see the majority of Development environments as sandpits and therefore uncontrolled; however, many organisations have staging environments where code is readied for deployment.

Why is any of this important? Because if we are to track (or predict) defects/bugs we have to agree at what point an event becomes a Defect. Over the past 10 years or so I have insisted that, the projects where I have been responsible for the Testing phase(s), specify exactly when an event can be classified as a Defect. The way I do this is to initially classify all unexpected outcomes as just that – unexpected outcomes. These situations are not classified as Defects until we have triaged them and agreed that they are indeed Defects. This means that an unexpected outcome can also be classified as a misunderstanding, an oversight, a user error, etc. Some may see this as semantics, but as I said in Part 1, semantics are very important in the world of QA/QC/Testing.

So, when I predict testing outcomes I am very careful about what I determine to be Defects.

By the way, a Defect can still be declassified upon further analysis or investigation, but it can never be removed from the record of the Project. It happened and therefore it must be recorded for future analysis or investigation when something unexpected happens.

Any thoughts?

Dateline: Tuesday January 14, 2014


3 thoughts on “Predicting Testing Outcomes (Part 2)

  1. Pingback: Testing Bits – 1/12/14 – 1/18/14 | Testing Curator Blog

  2. I like to call them issues and only during triage do we refine further such as Change Request, Defect or Not an Issue (always with an explanation). I find that developers can take it personal if you don’t distinguish between the different types. I also find it helpful to track where the issue was injected i.e. Requirements, Development, etc. I can then use this during PIR to help highlight areas that could be improved upon across whole of the life cycle.

We're here to help

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s