Kate has been a part of our Tester Community for almost two years now, joining us from South Africa. She has a lot of experience in functional testing and logging bugs and has a special superpower of spotting the ‘what’s missing’ bugs and usability issues. Here’s her useful guide on how to assess bug risk and how to correctly determine the severity of an issue.

How to assess a bug’s risk

Finding bugs is an art. It’s the art of minimizing risk.

In some applications, the impact of critical bugs can result in loss of life, for example in healthcare, where an incorrect procedure on the wrong patient might mean physical death. In other applications, like banking, the existence of a critical bug might mean serious financial losses. Luckily, the bugs in most applications don’t result in such drastic losses. The riskiest impact of bugs in most software is the loss of reputation.

When you encounter a bug in software, it either stops you from achieving your goal, leaving you no option but to abandon the software and find an alternative. Or if there are non-critical bugs, the software is perceived as unprofessional, leaving the user doubting the capability of the organization behind the software to do what it is intended to do and to be trusted.

With so many options available for any one function you possibly would want to do, there is little incentive for a user to continue to struggle through errors in an attempt to complete a goal. It’s much easier to find an alternative without the bugs.
Finding bugs is thus no longer only crucial for risky software like healthcare, banking, or insurance products, but essential for any product to stand out and grow in a saturated market filled with options.

But how do you decide which bugs are critical and which are not? It’s not as easy as providing a checklist, as a spelling error in one application might be critical and in others considered trivial.

One of the most famous examples of how a spelling error resulted in an $18.5 million cost can be found in a spelling mistake in the Nasa software: on 22 July 1962, Nasa launched the spacecraft Mariner 1. A mere five minutes after lift-off, however, the mission was aborted and the spacecraft destroyed. Surely one of the most expensive failures in history, caused by a single missing hyphen in the code.
In most cases, however, a spelling error will probably have no impact on the users and most people might not even notice it. And if someone does, the biggest impact is that you might become the laughing stock of the public for a short period of time.

The key is thus to understand the impact of the error: what might be the consequence of leaving the bug in the software? To help you decide how risky the software is, ask yourself these questions before you log a bug next time:

1. Would there be a security impact? If the bug was encountered in production, would it be possible to stop someone from accessing assets they should be able to access? Or could an unauthorized person access it more easily?
2. Might there be a financial impact in either the form of a loss or a gain as a result of the bug? A simple calculation error might, for example, result in thousands of losses or gains as a result.
3. What about the reputational impact? What would people say and how would they behave if the error was left in the code?
   Would there be any impact on the ability to provide a service as a result of the bug? Does the existence of the bug stop someone from getting or providing the service customers expect?
4. Finally, is there an impact on any resources as a result of the bug? Would it cost more to do the same task? Would it take more time to achieve the same goal? Or would you need more people to do the same job as a result of the error?

When there is a substantial impact in any of these areas, it will very likely be considered a critical bug. Although the ideal is to have perfect software, the cost of fixing a trivial bug with a small impact far outweighs the cost of leaving it in the software.
Fixing, for example, a trivial bug of correcting a non-critical spelling error will result in the cost of the tester who reported the bug, the cost of the test manager reviewing the bug, the cost of the development team who needs to implement it in the code, and again the cost of another test cycle once the bug has been fixed. Compare this to the benefit and it is simply not worth the effort and cost.

So next time before you log a trivial bug, ask yourself these questions and evaluate whether the risk of leaving the bug in the software might outweigh the benefit and cost of fixing it. Not only will you get paid more for logging better bugs, but your tester rank will also go up, improving your chances of being selected for the next test cycle.