Complex AI systems with non-deterministic outcomes pose challenges for testers and programmers. Such systems will increasingly become normal in high-impact, high-risk applications, argues Fiona Charles; testers should increase their capacity for thinking and learning and develop a number of personal strengths such as courage and good judgement.
Fiona Charles, a Software Test Consultant and Program Test Manager, will give the keynote presentation « Multiplying the Odds » at the European Testing Conference 2017 :
Software daily increases its hold on the world, and we continually devise different ways to create and implement it, as well as embracing sophisticated tools for assisting every stage of the process. There is no longer a robust dominant paradigm of software testing. Our expectations of how to test, when to test, who should test — even whether we need to test at all — keep changing with new methods and technology.
InfoQ interviewed Fiona Charles about the main challenges that testing has to deal with and how these challenges impact the test profession, the value that testers are expected to deliver, essential skills for testers and how to acquire them, and what the future will bring for testers.
InfoQ: Which are the main challenges that testing nowadays has to deal with?
Fiona Charles : I don’t think the basic challenges are any different from those we’ve always had in software testing. We never have enough time or capacity to test everything, so how do we decide what to test, and how thoroughly to test so that we deliver optimal value to our stakeholders? Then having made those strategic decisions, how can we do that testing most effectively and efficiently?
The coverage challenge is universal. Outside the mainstream—as yet—there is also a significant challenge in the proliferation of complex AI systems with non-deterministic outcomes. These systems will increasingly become normal in high-impact, high-risk applications and I think more software practitioners will need to step up to both programming and testing them. As just one example, we’re seeing AI systems used now to make decisions about sentencing and parole in the US justice system. We’ve all learned from the media that « learning » systems are vulnerable to biases in the data selected for them to learn from. It should also be obvious that they are very challenging to test. And they will bring ethical challenges for programmers and testers alike.
InfoQ: How do these challenges impact the test profession?
Charles : The practice of professional testing is morphing, partly to meet the testing challenges, but mainly prompted by advances in technology and software development methodologies. I think it has always morphed and for similar reasons.
Certainly the advent of agile has changed where testers fit on a team and what they do on software projects. The old, so-called « waterfall » methods often misused testers’ time and skills as they busily sat about designing tests for weeks or months, only to find themselves mired in silly bugs because nobody or almost nobody had done any unit testing.
