A few years ago I thought we’d seen the last of the (outlandish and absurd) claims around the capabilities of software testing tools, but it appears that we are now going through a new phase driven predominantly by the “every tester needs to learn how to code” mantra.
I am revisiting the capability (or not) of software testing tools after reading two intelligent blog posts by @TestPappy (Patrick Prill) and @therockertester (Lee Hawkins) over the weekend. Patrick wrote a post entitled “Test Automation – Am I the only One?” while Lee wrote “More on the commoditisation of testing“. Both of these posts tackle the various expectations and capabilities of software testing tools, but from completely different perspectives. Patrick has been on a 3 year journey to implement a test automation framework, while Lee highlights claims made by the Head of QA at UBS and the impact test automation will have on headcount and budgets.
My story is one of expectation and risk management. To provide some additional context, I will provide some details of my background and experience to show that I have the credentials to write on this topic. I began writing software over 40 years ago and still dabble to this day. I began testing software the day after I wrote my first lines of code. I designed my first test automation framework in 1999. I have managed over 25 programs/projects where software testing tools were an integral part of the process. I have given keynotes on the challenges of implementing software testing tools.
So, here’s my tuppence worth…
Ever since the dawn of time men (and I mean men NOT women) have been over-optimistic about their capabilities. Ever since the first days of the industrial revolution men have believed that no matter what problems they create (let’s take global warming as a current example) they will be able to fix them with more technology. Ever since the first computer software project was initiated men have under-estimated the time and cost and over-estimated what will be delivered. I have NEVER worked on a single project that delivered everything it set out to, let alone on budget and on time!! And, I’ve worked on almost 100 projects over the years.
So what has suddenly changed so that (again, predominantly) men can claim that buying “X” tool or learning to code in “Y” language will solve all of our software implementation problems. Well, I’ve got news for you – every new piece of technology brings with it it’s own risks and challenges. If test automation tools were going to have a major impact on our capability to deliver software projects on time and on budget don’t you think we’ve had long enough now (it’s been over 40 years since I wrote my first test harness) to make this happen?
And while I’ve got this head of steam going… Why does every tester suddenly think are capable of writing code? We (software testing professionals) have been upset for years by claims of “anyone can be a tester” and now we’re at it with “anyone/everyone can/should write code“. As I said, I was writing code over 40 years ago and I didn’t just sit down one day and say to myself “I think I’ll become a computer programmer”. I went through a very vigorous selection process and spent years learning how to be a competent and effective software developer. I know we didn’t have the same toolset as we have today, but we still had to write coherent and maintainable code – much of which is still in use today, I might add!!
And, back to the UBS component highlighted by Lee Hawkins…. If I had a dollar for every time I’d heard a senior manager say that “we have a test automation strategy that will save us thousands/millions on our software testing budget”, I’d be even richer than I am now!!
Software implementation projects frequently fail. Software test automation projects fail even more frequently. Why? In my humble opinion (as I said before) – because men always over-estimate their capability and under-estimate how long something will take. Add to this the inability of most people to be able to succinctly define the goals of any project and you will begin to understand why I still despair when I read/hear about the promises for the next test automation tool.
Software testing is a skilful endeavour, not something that can be easily commoditised (although I believe that one day we will be able to achieve some basic commoditisation). Replacing software testers with software tools is a fools errand. Assisting software testers by providing software tools is a far more sustainable outcome.
In summary – writing AND testing software is not a trivial undertaking, so why do we trivialise strategies to perform them? If you want to reduce your software tester costs, employ the best possible testers you can find, not the cheapest. If you want to automate software development processes focus on the tasks that require least intellectual capability. If you want to run a world-class software testing capability employ the people with the most battle scars, not the fresh (out of uni) drones that will work twenty hours a day, never questioning what you tell them to do. And, don’t get me started on sending your software overseas to the cheapest labour markets to archive better bang for buck….
Dateline: Melbourne, Tuesday April 26, 2106