The Two Approaches to Test Coverage • Slash

Test coverage, let’s begin by outlining what we mean by this. Test coverage is a method for measuring how many codes have been covered by your test cases. Simple, right? We think so too! Though why is something so simple, this important? It’s important to emphasize that although unit testing can increase the amount of time it takes to implement a feature, the overall time requirements balance out over the product’s release cycle due to increased quality and maintainability.

Using code-coverage tools to help check the total percentage that you cover all over your code base is key.

Coverage Metrics

There are four key coverage metrics that you must know to understand test coverage. Those four metrics are as follows:

1. Statements – The number of statements that have been executed at least once during the unit tests. In some programming languages, there can be multiple statements in a single line of code.

2. Branch – The number of control statements which conditions have been fulfilled at least once during the unit tests.

Branch coverage provides more precise results than code coverage because it helps cope with code coverage’s shortcomings. Instead of using the raw number of code lines, this metric focuses on control statements, such as if and switch statements. It shows how many of such control statements are traversed by at least one test in the suite.

Each condition can be counted as one; however, there are some cases where you don’t have the explicit “else” so there are some tricks you can use to count the number of branches.

Control statements
- if… else (even is implicit else)
- switch counts as branch
- ternary operators
- else if count as 2 branches
Default parameters

Please note, not all loops count as a branch and branch is the most important metric.

3. Functions – The functions that have been called at least once during the unit tests. One function counts as one, if the file has two functions and you test the file, you’ll get 100%.

4. Lines – The number of lines of code that have been executed at least once during the unit tests.

The uncovered line metric simply clues you into which lines of code are not covered in the test. Now, the statement metric represents instructions that have been executed at least once during the unit tests. Lines represent code lines that have been executed at least once during the unit tests. Often you will see the statement and line metrics have the same percentage. This is a reflection of coding itself; that being, typically you only write one statement per line. Though naturally there are exceptions like when using two statements per line for a declaration.

The final question on yours and everyone’s minds is… How many tests should we cover? Should we do 100% test coverage or…? Good questions and to answer these, let’s consider two schools of thought to approach test coverage.

Table of Contents

Benefits

Let’s wrap-up with a quick review of the benefits of test coverage. For one, it helps improve the quality of your software, and also ensures you do not ship bug-filled software to your end-user. Secondly, code refactoring is helpful for the developers because in unit testing when considerable change is applied you have to ensure all the previous functionality is working. Thirdly it benefits adding new features as when this is done you have to ensure that all the existing features don’t break.

Fourthly, documentation helps as it includes documentation for your function so other team members can see how the function will work by viewing the unit testing. Fifthly, early bug discovery allows you to have a proactive stance and stay out in front of those troublesome obstacles to development. This in turn leads to the sixth benefit of simplifying the debugging process. Here instead of testing and brand applications over and over, you just need to have one unit test that can do the debugging for you.

Finally this culminates into developer happiness as once you have more unit tests, developers will gain confidence when launching the product. A confident developer is a happy developer who is capable of producing stunningly clean code for a superb product!

Approach #1

The goal is not to score the highest percentage for test coverage, rather we want to test that which is isolated and doesn’t require a mock function. If the tests require a lot of mock functions, the test is not so effective. However, this can be ideal for small dev teams short on manpower. So in this case, unit testing is deployed. Now the total test coverage may be around 30%, but you’re only testing the very unit functions that don’t require some mockup. Consider that if you try to reach a higher percentage of test coverage, you will need to maintain and refactor more code.

Therefore, unit testing is part of your code base, increasing the need to do more integration testing. How the implementation works is not the key point as your focus should be on the final result. Remember that you’re only testing functions which do not require any mocking up.

Approach #2

On the flip side to approach #1, we have the idea that the more coverage, the better chance of finding bugs. This approach is well suited for a larger dev team with the manpower and resources to run all these tests. A simple goal is to measure the percentage of the code covered by the tests. However, this is not a bulletproof approach that guarantees the discovery of any and all bugs; hence the existence of approach #1.

You could easily have close to 100% code coverage with bad tests that don’t mean anything. Low coverage is a bad sign; high coverage is a possible sign that things are better, but never perfect. This approach hamstrings the team’s ability to implement integrate testing due to spending too much time on test coverage.

In the end, regardless of the test approach you select, testing not only needs to be done, but is inherently key to the success of the code you’ve written. Both tracks lead you to testing and aim to ensure clean code at the terminus!