A few years ago I asked an audience at a Software Testing Conference – “If I disagree with my wife and she’s not in the room, am I still wrong?”
The overwhelming (and triumphant) response from the ladies was an almost operatic “Yes”, while the men were split between those who winced and those who just looked at each other – resigned and beaten. This is my spin on the famous George Berkeley quote regarding the falling tree in the forest.
I use this story regularly to explain the concept of existence to those who are new to Testing. How can we be sure that a software bug exists, (especially, if we don’t find it)? Even more important – how can we be sure that a bug does not exist? Existence is an interesting topic (for me anyway), because philosophers and mathematicians have been debating this for centuries – and are still debating it. Descartes declared that because he could think he must exist – arriving at this conclusion from a philosophical standpoint, while Einstein stated something similar from a mathematical perspective. So, if these guys thought it was important that’s good enough for me. Their thinking (together with that of other great philosophers and mathematicians) has driven my approach to software development and testing for as long as I can remember.
But, how do you approach software? Do you actively search for bugs, or do you cruise around your keyboard hoping not to find anything untoward, while occasionally stumbling on a gem? Does absence make the heart grow fonder or does it just make you feel uneasy when a new release of software fails to reveal even a trivial nuance?
What do you feel/think when you unearth your latest bug? Does it make you happy when software fails? Or do you throw up your arms in a gesture of complete and utter despair and consider casting evil spells on the development team and all their future offspring?
Do you ever provide anything other than a conditional pass? After all, mathematics tells us that there is no such thing as faultless software. Is the statement – “There are no (more) bugs in this software” ever true, or should we always be stating – “I can’t find any (more) bugs in this software”
I know that the majority of Project Managers would be very unhappy if we used the latter, whereas they are more than happy to sign-off on the former….. So, should we lie to our employers? Of course not!!
So, now we’re back where we started – if a software bug exists but nobody finds it, does it really exist – of course it does. Does it matter that we didn’t find it? Philosophically – YES, mathematically – NO…. Some will say that “a bug is a bug is a bug” while others will state that a bug can lay dormant for the entire life of the software (within which it exists) and never cause an issue. Similarly though, it can exist for a nano-second and cause a nuclear explosion.
This is where most people would say the mathematicians have the edge. Mathematically we can calculate the probability and impact of an unfound bug. Philosophically we could spend days/weeks/months/years debating the cazillion possibilities. However, do we always turn to the mathematicians or do we just as often take the philosophical standpoint?
In my experience we try to do both – depending on the context. If we’re testing safety critical systems we call in the mathematicians, while if we’re testing a mobile App that tells us our favourite coffee, we may accept something a little more philosophical (just messing with the minds of coffee snobs here). Either way, epistemology will (hopefully) sneak it’s way in somewhere and we get to something approaching a “good enough” solution.
Philosophically, I do as little testing as possible to achieve my goal. Mathematically, I do one more test than the absolute minimum to ensure my customer sends me a Christmas card.
What do you do? If an answer doesn’t formulate within the next 5 seconds, call me.
Dateline: November 28, 2012; Bagshot, England