My Final Word on Software Tester Certification

When I posted my Tester Certification Debate Just Got VERY Personal article on June 17 I had no idea what a profound and long-term effect the episode I described in that Blog post would have on me. Today (after considerable reflection), I no longer value what the ANZTB stands for in the Asia Pacific region. Today, I no longer respect the current Chairperson of the ANZTB. Today, I no longer care that the income the ANZTB has received from the (now over 10,000) certificates they have issued is not spent more wisely. Today, I no longer care that the Managing Director of PlanIT (a major provider of ISTQB training in Asia Pacific) is the new President of the ISTQB and that this (in my humble opinion) creates a serious conflict of interest.

I still believe in the advancement of our craft. I still believe in the value that the majority of us bring to the IT industry. I still believe in the clear separation of duties between those who define, design, build and launch software. I just don’t believe that we have the right people determining HOW we should go about developing future generations of software testers.

According to the ISTQB website – “The Foundation Level qualification is aimed at professionals who need to demonstrate practical knowledge of the fundamental concepts of software testing. This includes people in roles such as test designers, test analysts, test engineers, test consultants, test managers, user acceptance testers and IT Professionals. The Foundation Level qualification is also appropriate for anyone who needs a basic understanding of software testing, such as project managers, quality managers, software development managers, business analysts, IT directors and management consultants.

I don’t know about you, but it seems to me that the ISTQB wants everyone to become a certified tester and this (for me) devalues it as a currency – and a currency it certainly is – if you read the myriad of job advertisements that currently specify ISTQB certification as a pre-requisite. The way I see it, they can’t have it both ways – either ISTQB certification has real value or it comes free with cornflakes (which is where I currently see it residing). The ISTQB recommends (on it’s website) that you have at least 6 months experience in software testing before you attend a training course, but it also wants Project Managers, Software Development Managers etc. to attend – I haven’t met too many PMs or Dev Managers who have 6 seconds of software testing experience, let alone 6 months….

There are many consulting businesses around the world who state as a marketing proposition that all of their software testers are ISTQB certified. BIG DEAL. So you have people in your organisation who have passed an exam after spending less than 5 days in a classroom. Have they ever tested any software? Have they ever reverse engineered testing requirements because there are no business requirements to validate UAT against? Have they ever found a software bug that could have prevented a travel website crashing or a payment engine putting money into the wrong account? Have they ever attended a bug triage meeting and explained the root cause of a problem in the general ledger? Have they ever identified a reconciliation discrepancy in a data migration? You won’t find any of this in an ISTQB Foundation exam. What you will find is a multiple choice question asking you the definition of boundary value analysis or an orthogonal array. Most of the testers I have worked with wouldn’t know an orthogonal array if it bit them on the bum, but they do know how to find bugs in software – lots of bugs in lots of software, every day of the week, every week of the year…. and they don’t have an ISTQB certificate in their back pocket to prove they know what they’re doing!!

I was involved in ISEB/ISTQB syllabi reviews in the 90’s and I hoped that they would find more practical ways to assess competency and achievement, alas nothing has changed except that the number of certificates issued has grown twenty fold while the value put on these certificates has grown out of all proportion to their true worth.

As I have said several times over the past few months, I am greatly saddened by this current situation. If ISTQB certification is ever to have any relevance in our industry many things and many personnel need to change. Far too many people are being misled into thinking that ISTQB Foundation certification indicates that someone has an aptitude and/or capability for testing software. Of course there are many great software testers who are ISTQB certified, but they would still be great Testers without this certification.

And my final, final words on the subject… As I have declared before, I hold a current ISTQB Foundation certificate. Evidently, under the current rules, I will still be an ISTQB certified tester when I am 99 years old. I wonder what value future generations of Testers will put on their certification when they realise that my (decades old) testing knowledge will have equal value to theirs in the eyes of the majority of recruiters and HR departments….

Dateline: Tuesday July 30, 2013


20 thoughts on “My Final Word on Software Tester Certification

  1. One thing I found interesting in the list of roles that ISTQB believes would benefit from software testing knowledge: Software developers are missing from the list.

    Developers are testers too.

  2. Pingback: Five Blogs – 1 August 2013 | 5blogs

  3. I agree. The ISTQB “hide behind the veil of accreditation” scam is just another example of the commoditisation and devaluing of our industry. I for one won’t stand for it and we at Access Testing are actively fighting against it. I had a non testing staff member sit the exam having read some notes for 3 hours. When he didn’t know the answer he guessed “D”! He passed

    Unfortunately when the majority of the industry buys into a flawed certification, it becomes an effective certification regardless of outcome

    • Great to see your thoughts Tony. If only we could get a voice at the table we’d be able to try and convince them that multiple choice and non-experiential questions with practical lab-style challenges would begin to meet the minimum requirement we should expect.

  4. Hey Colin,

    Mate I like your stuff but isn’t this all getting a bit too emotive? Yes, you’re miffed because some people didn’t like a presentation you did. It happens. Actually, your first story had nothing to do with any certification debate so how did any of it get ‘very personal’? Didn’t make sense to me.

    Your latest story seems way off the mark. I’ve done the basic istqb certification and have loads of testing experience. I’ve also read books, done training courses and sometimes got along to conferences (but there aren’t many in WA). I reckon you’re getting too hung up on the word ‘certification’. It’s just some training with a test at the end to make sure people have remembered a thing or two. No big deal. All learning is good. Don’t become bitter and twisted and dump on certification because some presentation didn’t work out. I’m glad when my staff want to do certification, especially the advanced ones, because it shows they want to extend their knowledge.

    Anyway, love your online mag. Keep up the good work.

    • Hi Mark, I really appreciate the feedback. My presentation went down really well with the exception of one or two folk ( out of over 200) who must have been looking for something to gripe about. My issue was with respect to how the feedback was provided by the current Chairperson of the ISTQB and the heavy handed way he then prevented me fulfilling an obligation I had already made. I should also reiterate that I was invited to speak in Canberra, I didn’t submit a proposal as is the case with most conferences.

      Secondly, I am a believer in the concept of certification I just have a problem with how it is being interpreted and marketed by the ISTQB and the ANZTB. As I have said on numerous occasions, I hold an ISTQB Foundation certificate and have studied for the Advanced Test Managers qualification so I have in the past been a supporter of the brand. However, I now believe that there are many improvements that could be made to improve a generally outdated and poorly tested syllabus.

      Once again, thanks for taking the time to enter the debate.

    • Hey Mark,

      I think Colin’s brush with ISTQB on the prezzo was just the wake-up moment. As I read it there is a lot more reason for his stance.

      It is an emotive issue as it is our profession we’re talking about. In my eyes ISTQB certification is detrimental to our profession in a big way. It takes away from what we should actually be doing. It’s a nasty way of assuaging us in certainty, where there is none.

      I agree learning is good but I do see that we should try and focus on good learning and not just anything that crosses our path. The $$$ we throw at ANZTB & training companies for ISTQB could be used much more effectively. I’d say you can just buy a book and take a day to read it to learn what ISTQB is all about. The cost is minimal and the effect would be the same.

      Where it all comes undone is, where employers think that ISTQB certification has a value. I.e. they make it a precondition to employment not realising it’s not much more than if you had a barista certification (sorry if that comparison is not PC to barista’s!). So good testers, that just won’t follow a delusional certification crowd, will not end up in your organisation, where you are probably desperately needed.

      And then there is the whole topic of making the wolf the shepherd as in ANZTB’s case. You have to wonder.


      • Hi Oliver,

        I agree 100% with your comments.

        One of the biggest challenges we have (as those who really care about our craft and the wider Testing community) is to show very clearly how much misinformation is out there with respect to (so called) ROI with respect to ISTQB certification. Surely any bone fide 5-day training course (with practical exercises) will provide a similar return on investment. In my experience the biggest drawback with ANY training is that the learnings must be applied within the first 30 days of receiving them otherwise the value deteriorates rapidly. Therefore, the materials provided by the specific training companies and any community created by it becomes the most valuable commodity NOT the exam or it’s outcome.

  5. To prove a point to a group of 10 IT graduates I was hiring into a testing consultancy, I had my HR manager (on 2 days notice) sit the ISTQB Foundation exam. She got the top mark of the group but did it make her hireable as a tester, no, she would hire herself. I have never really valued the Foundation level certification and have never viewed it as a must have.

  6. It’s great to see that someone has challenged the ‘qualification’ of ISTQB. I too cannot understand why employers ‘insist’ on having someone hold this qualification for employment etc. Yes, it is good to see that they have spent the time in attending the course, and understanding basic concepts on testing, but its not a ‘qualification’ as such. As a manager, I look at whether to send someone on the course, and make the monetary investment in providing for a staff member to attend such courses – and ask the question ‘what’s in it for me?’ or in other words, ‘what’s in it for the company?’ – who is ultimately paying…

    As a manager, this money spent on sending someone on these courses is far better spent on developing ‘core’ skills, that will deliver additional benefit (or look at it another way – ‘return on investment’).

    That’s my two cents….

  7. Could not agree with you more Colin (though you are far more eloquent than I).

    I work for one of those consulting businesses who demand we become certified (and yes I did) and only last week had to hang my head in shame when my practice lead advised that funding was not likely to be available for conference attendance for my team, but would not be an issue for certification training (which by the way costs more) as the organization sees an ROI for it.


  8. Hi Colin,

    Plus one from me.

    I did due dilligence on the ISTQB foundation level syllabus when the ANZTB was setting itself up a few years ago. I have never seen anything of value in the “training” on offer, and like you am more than a little pissed off by the use of this “qualification” as a pre screening tool for recruiters and test managers who don’t know any better.

    The more voices speaking out about this the better.



    • Thanks Andrew for your comments. I’m not saying everything is bad when it comes to this interpretation of software testing education, but I do believe it has lost it’s way since the early (ISEB) days and the current “custodians” in AsiaPac (ANZTB) are not helping. The fact that so many people are taking notice of my Blogs on this subject means there must be plenty of interest in improving matters.

  9. Ok Colin, tell us what you really think 😉

    My current employer “demands” I become foundation certified. Do I think it will make me any better at my job? Does it annoy me that our sales staff are just selling me as certified, not for the fact that I have knowledge and skills far beyond what could ever be contained in a “one size fits all”, multiple choice exam?

    Maybe I am just being elitist, but the slaves to ISTQB always go down a notch or two in my respect levels. It is a bit like demanding every worker attended Primary School.

    • Hi Doug,

      The marketing folk at ISTQB have done a great job in selling their certification story, unfortunately the folks working in the background are not on par with them. This is probably because the “technical experts” are not employees but volunteers or consultants who are doing the work in addition to their day jobs – how else would it take almost two years to update a syllabus!! In Canberra (at the latest Annual Conference) we heard from Mike Smith who is a UK-based (Testing) business owner and entrepreneur who has been guiding a major syllabus update over the past couple of years. It was an obvious marketing ploy to fly someone across from the UK to spend 45 minutes talking about a syllabus update – I could have saved them a heap of dough and done it for no cost or Mike could have done a Video Link at a fraction of the cost – as I said in the main Blog post – they don’t seem to be very careful with our Certification dollars…

      On the subject of HR departments and recruiters stipulating the necessity for ISTQB certification, I think a lot of these organisations are looking for an easy way pre-select staff. In my experience ISTQB Foundation certification does not mean you get a better class of Tester, it just means you get someone who has spent a few days in a classroom (although even this isn’t a pre-requisite for taking the exam) and spent a couple of hours answering a few multiple choice questions. My aim is to expose this silliness for what it is.

We're here to help

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s