The Software Testing Pyramid • Slash

Table of Contents

3 Key Layers

Nothing is perfect, or so the saying goes. Everything can always be improved upon and try as we may, nothing is devoid of flaws or defects in regards to manufactured products. Software is no different, it’s a product subject to the same imperfections other products experience. We of course, are referring to the much dreaded software bugs. Those little programming hiccups and imperfections that demand attention before wreaking havoc.

Just like any product, software is subject to a QC/QA process to verify quality, but also confirm there are no bugs present in the code. In steps software testing. Testing is vital to delivering bug free code and reducing or hopefully eliminating the negative ramifications of uncaught bugs. Software bugs can cost money and even lives. Two examples come to mind:

In 1985, a software bug caused Canada’s Therac-25 radiation therapy machine to malfunction, delivering lethal doses of radiation to patients resulting in the deaths of 3 people.
In April of 1999, a software bug caused the failure of a $1.2 billion military satellite launch, the costliest accident in history.

So clearly , testing is important. Whether you’re testing software for some corporate enterprise solution, public mass transport system, online banking or other, the tests should not be taken lightly.

A testing pyramid is a framework that can assist both developers and QA engineers in building high-quality software. It minimizes the amount of time required for developers to determine if an update they made affects the code. It also aids in developing a more robust test suite.

Khorikov,V. (2020) Unit Testing Principles, Practices and Patterns. Page 87

As you can see from what the image above suggests, the layer width corresponds directly with the amount of testing. Additionally, the height is a measure of how close the tests are to emulating end user behavior. The layers represent three different tests:

- Unit tests – find logical errors at the most fundamental level. They are fast and require very few resources to run.
- Integration tests – verify that services and databases work well together with the code and the written classes. They can only find problems at the interfaces where two or more components meet.
- End-to-End (E2E) tests – depend on the complete application being able to start. These are the most comprehensive type of tests and therefore, require the most computing resources and time to run.

Let’s look at each layer individually to better understand the purpose they serve.

Unit Testing

The testing begins with unit tests which are very low level and close to the source of an application. The procedure consists of testing individual methods and functions of the classes, components, or modules used by the software. The key goal here is to isolate and focus on testing the code where all external dependencies are replaced with pre-programmed behavior, ensuring that the test’s outcome is only determined by the correctness of the unit being tested. Unit tests are generally quite cheap to automate and can be executed quickly by a continuous integration server. The drawback, however, is that unit tests cannot cover the interaction between dependencies; therefore, integration tests become beneficial at that point.

For a simple example, consider you have a function called “getPopularMovies.” This function will call an API to get a list of movies and the top three records to return as popular movies. In this case, we need to mock the API call and only test the popular movie logic.

So what is good unit testing? That is without a doubt the Arrange-Assert-Act (AAA) test pattern process. This three step process allows you to first “arrange” or establish the conditions needed for the second “act” step to manage while you “assert” or confirm the expected results were reached in the final step.

File: movies.js

File: movies.test.js

Integration Testing

Moving up the pyramid we come to integration testing. Integration tests verify that different modules or services used by the application work well together. Obviously there are many moving parts behind the scenes when the software is working so it’s key to ensure all those parts are playing nice and getting along. Integration tests are quite useful for small systems especially as they deliver quick results. That being said, there are some limitations to them as well. It can be more difficult to locate faults, riskier modules don’t receive concentrated attention and there is always a possibility of missing a module.

Now for example, let’s consider you are testing the interaction with the API and confirming that the third party API works together as expected. In this case, we test the “getPopularMovies” call to the real API. If we give the code the same parameters and it works correctly, then it has passed. This groups together two or more modules of an application to ensure they function collectively. This type of testing also reveals the interface, communication and data flow defects between modules.

These types of tests are more expensive to run as they require multiple parts of the application to be up and running; therefore, this testing cannot begin early. Additionally, mock function testing is not used as use of the external library itself is necessary.

File: movies.integration.test.js

End-to-End Testing

Finally we reach the peak of the pyramid, the ultimate test of code excellence. End-to-end testing replicates a user behavior with the software in a complete application environment. E2E tests verify that various user flows work as expected; this can be as simple as loading a web page, logging in, or much more complex scenarios like verifying email notifications, online payments and so on. The goal is to mimic how the software works in real life through executing typical user scenarios and subsequently noting any and all errors that occur.

End-to-end tests are very useful, but they’re expensive to perform and can be hard to maintain when they’re automated. It is recommended to have a few key E2E tests and rely more on lower level types of testing (unit and integration tests) to be able to quickly identify breaking changes. There are some major JS frameworks for E2E testing such as Cypress, TestCafe, NightwatchJS, Protractor, Webdriver JS and so on which can be utilized.

Khorikov,V. (2020) Unit Testing Principles, Practices and Patterns. Page 6

To summarize, the value of testing should be obvious and as a result, demand an appropriate amount of attention paid to it. It is more expensive to fix a bug at the end of the project than during; the later you find the issue, the more costly it will be to fix. The software industry is not the automotive industry, we can be releasing our products to then only be forced to recall them to correct issues we should have caught before it went out the door. When you find an issue post production, it costs time & money for a developer to find & report the bug; plus, if considerable enough time has passed before something goes wrong, you may lose business value so the earlier you detect and correct, the better!