Existence, Existentialism & Epistemology

A few years ago I asked an audience at a Software Testing Conference – “If I disagree with my wife and she’s not in the room, am I still wrong?”

The overwhelming (and triumphant) response from the ladies was an almost operatic “Yes”, while the men were split between those who winced and those who just looked at each other – resigned and beaten. This is my spin on the famous George Berkeley quote regarding the falling tree in the forest.

I use this story regularly to explain the concept of existence to those who are new to Testing. How can we be sure that a software bug exists, (especially, if we don’t find it)? Even more important – how can we be sure that a bug does not exist? Existence is an interesting topic (for me anyway), because philosophers and mathematicians have been debating this for centuries – and are still debating it. Descartes declared that because he could think he must exist – arriving at this conclusion from a philosophical standpoint, while Einstein stated something similar from a mathematical perspective. So, if these guys thought it was important that’s good enough for me. Their thinking (together with that of other great philosophers and mathematicians) has driven my approach to software development and testing for as long as I can remember.

But, how do you approach software? Do you actively search for bugs, or do you cruise around your keyboard hoping not to find anything untoward, while occasionally stumbling on a gem? Does absence make the heart grow fonder or does it just make you feel uneasy when a new release of software fails to reveal even a trivial nuance?

What do you feel/think when you unearth your latest bug? Does it make you happy when software fails? Or do you throw up your arms in a gesture of complete and utter despair and consider casting evil spells on the development team and all their future offspring?

Do you ever provide anything other than a conditional pass? After all, mathematics tells us that there is no such thing as faultless software. Is the statement – “There are no (more) bugs in this software” ever true, or should we always be stating – “I can’t find any (more) bugs in this software”

I know that the majority of Project Managers would be very unhappy if we used the latter, whereas they are more than happy to sign-off on the former….. So, should we lie to our employers? Of course not!!

So, now we’re back where we started – if a software bug exists but nobody finds it, does it really exist – of course it does. Does it matter that we didn’t find it? Philosophically – YES, mathematically – NO…. Some will say that “a bug is a bug is a bug” while others will state that a bug can lay dormant for the entire life of the software (within which it exists) and never cause an issue. Similarly though, it can exist for a nano-second and cause a nuclear explosion.

This is where most people would say the mathematicians have the edge. Mathematically we can calculate the probability and impact of an unfound bug. Philosophically we could spend days/weeks/months/years debating the cazillion possibilities. However, do we always turn to the mathematicians or do we just as often take the philosophical standpoint?

In my experience we try to do both – depending on the context. If we’re testing safety critical systems we call in the mathematicians, while if we’re testing a mobile App that tells us our favourite coffee, we may accept something a little more philosophical (just messing with the minds of coffee snobs here). Either way, epistemology will (hopefully) sneak it’s way in somewhere and we get to something approaching a “good enough” solution.

Philosophically, I do as little testing as possible to achieve my goal. Mathematically, I do one more test than the absolute minimum to ensure my customer sends me a Christmas card.

What do you do? If an answer doesn’t formulate within the next 5 seconds, call me.

Dateline: November 28, 2012; Bagshot, England

Advertisements

4 thoughts on “Existence, Existentialism & Epistemology

  1. Criticality and impact of failure sets the starting level for the quality bar mostly…Then you’d better know what it takes to absoutely test the hell out of something if required. Uncommon, in my experience as the business drivers are rarely there.

    On what basis does it hold that “Mathematically we can calculate the probability and impact of an unfound bug”.

    Jared

  2. On what basis do you believe “Mathematically we can calculate the probability and impact of an unfound bug” to be true (or at least, useful, given that you’ve said we can calculate but have made no claims about the reliability of those calculations)?

    • Hi Jared, thanks for the interest and question.

      Over the years I’ve been looking at and working on a myriad of mathematical models for estimating and predicting bug rates and I’ve now got a model that so far predicts with over 97% accuracy the numbers of bugs based upon inputs, outputs, processes, calculations and interfaces. For me, it’s about identifying all the moving parts (good design reduces these where possible) and calculating the permutations based upon the possible pathways. If a pathway isn’t possible/allowable then it can’t fail. We assign basic impacts to bugs depending on the type of action with external interfaces scoring highest. There are obvious adjustments depending upon the context and user base as well as various industry models.

      I can provide you with the White Paper that we submitted to several Universities for validation and comment.

      • Ha, just asked the same thing 4 years later… I think you have my email through this post, if you could send the white paper or publish on LinkedIn?

We're here to help

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s