The real cost of not testing your software (hint: it’s not just money)

People and business owners rely so much on customized software nowadays so it’s imperative that the software works and that bugs are found and fixed before it goes live.

A report by the Consortium for Information and Software Quality (CISQ) stated that in 2020 alone, approximately $2.08 trillion was lost by U.S. companies due to poor software quality.

While there is no such thing as a perfect IT project, some of these losses could have been avoided if there was a robust software testing process in place during development and deployment.

Some companies skip the testing altogether because it’s an added cost to the overall price of a software project. While testing will increase the price of the initial project, it will lower the lifetime costs of the project if done correctly.

Here’s how we define software testing

When we say software testing, we are referring to testing done by actual software testers who are trained to spot, report and ensure that errors are corrected.

In an ideal setup, developers shouldn’t be testing at all. The problem with developers testing is that most of them do not like to do it and when they are asked to test, they tend to use the same test scenarios over again, resulting in bugs not being found.

Dedicated software testers, on the other hand, have the experience and expertise paired with a rigorous testing process and tools to ensure that errors are found and reported.

Not all software companies have a dedicated team of testers, so they either let their customer test the output or hire an external team of software testers to work with the development team throughout the project.

The real costs of not doing software testing

1. Errors

No matter how competent developers and designers are, errors and bugs are inevitable in software projects. Without a formal testing process, the end users (customers) will do the testing—which does not always produce ideal results.

Imagine a webshop with an error in the checkout. Most customers will not report the error, they will simply move on to the next similar webshop and the company will not get the sale. Scale that up to 10, 20 or 30 times.

2. Time and effort

Fixing errors in the later phases of a development project is more challenging than fixing them as the project is progressing.

This is because errors discovered at the tail end of a project may require code written at the beginning to be modified. This can then have a trickle-down effect where errors are introduced elsewhere in the application.

The 2002 report by the National Institute of Standard Technology (NIST) states that an average bug found in early coding/unit testing takes 4.9 hours to fix, whereas an average bug found in post-product release takes 15.3 hours to fix.

3. Money

The time and effort spent in fixing the errors cost money. Instead of spending money on features and upgrades that could improve the functionality or quality of the software, the money is spent on fixing errors that would have cost a lot less to fix had they been identified earlier in the development process.

Furthermore, if you’re a project manager in a software development company, there are cases where you cannot ask the client to pay for the time you’ve spent on fixing the errors that were discovered too late. The time spent therefore becomes non-billable.

What testing can bring in every project phase

Often, when companies incorporate testing, they fall at the later stages. Testing should not be a one-time thing.

There’s a common saying amongst the testers—which is to test early and test often. After all, software evolves as you add features and make upgrades. As such, good software development means including testing in all phases.

A note on ‘Automated Testing’

For complex software projects or projects where developers make constant updates (and therefore have to make many deployments), it is not only expensive but also may not be possible to do manual testing after each deployment due to the number of tests that have to be performed.

This is where automated testing is useful because it makes it possible to ‘remote control’ a website, webshop, or web application just like if it was a human—and that way find and report errors to the developers.

This feature can be used in conjunction with automated deployment so that ‘all’ a developer has to do is to click a button and then a computer will install and test the updates without any human intervention.