Testing 101: From our Slash testing playbook • Slash

Clients are often confused as to how to get started, whether it’s a new project or a takeover of an existing one. The main challenge is that there is a gap between the clarity they think they have and the clarity we need, to provide project costing and timelines.

To overcome that confusion and form a clear, mutually comprehensible action plan, we at Slash have developed our own approach. In a series of articles, we will present the predefined phases of our processes. Today’s article is devoted to Testing.

***
In general, the testing is done by two teams – the Delivery Team and the Dedicated QA team. In some cases, an independent third party can get involved in the testing as well.

Each of the teams completes a set of tasks within its scope of competence, and some of their functions may overlap. The QA team can be embedded in the Delivery team as well.

Table of Contents

Delivery Team

The Delivery Team implements 5 main testing practices:

Agile Testing: in agile development, testing starts early and happens regularly, so that features can be tested as they are developed, not after. Product Owner (PO) provides the detailed Acceptance Criteria to the Delivery team and the QA Team, prior to each Sprint.

Manual Testing: The code is tested manually by the developers in accordance with the pre-defined Acceptance Criteria and an end-to-end test of the user journeys and the User Interface.

Automation Testing: These are a set of automation techniques that allow developers to compare the expected and the actual outcomes of a unit of code. The benefit is that problems can be identified early in the development cycle, including bugs in the programmer’s implementation and flaws or missing parts of the specification for the unit. Practices include test-driven development (TDD), end-to-end testing platforms, device farms and emulators, Unit Testing (which is expressed in “% coverage of a codebase”).

Definition of Done (DoD): This is an agreed-upon set of items that must be completed before a project or user story can be considered complete. Typically this includes agile testing, manual testing, automation testing and deployment checks.

PO final Validation: the Product Owner conducts a final check or empowers the QA lead to do the final validation before the story is deployed and considered done.

Dedicated QA team

The QA team’s role includes 5 key processes.

1. The team reviews the Acceptance Criteria with the PO and the QA writes the test plan incrementally.

2. QA Manual Test strategy and Test plan: this includes Scenarios, Steps, Expected Results, Signoff, Traceability matrix, etc.

3. Automation Testing: this process could either be structured as a separate user story or as part of the Definition of Done for each User Story. If a QA specialist with automation skills is assigned to a product, the QA team can handle automation testing. Otherwise, this would fall under the responsibility of the software development team. Examples of automation testing include:

E2E Testing Platform (e.g Selenium, Cypress, APUIM….), Android Emulator (e.g. GenyMotion), iOS Emulator (e.g Test.io), AWS Device Farm.
Unit testing with Eslint and Mocha.

4. End-to-end manual testing: the team tests the application’s workflow from beginning to end.

In the case of mobile app testing: while mobile emulators can help to test a mobile app’s basic behavior, it is limited as a testing platform.
In order to effectively test a mobile app across real world scenarios, we believe it is essential to use real mobile devices. We use a physical mobile test wall with different devices, whenever the device is available.

5. Open Bugs: in case bugs are discovered, the team works with the Dev/PO team and the client to identify the criticality of a bug. Pre-agreeing a triage system with the client that defined the criticality of a bug (based on whether the user experience is seriously affected) and the expected timeline to resolve the bug (the “SLA” or Service Level Agreement) is important.

Mixed:

Depending on skillset of the Delivery Team and the QA team as well as project requirements, the following two functions can be done by either:

API Testing: the Delivery team tests the APIs directly to check their functionality, reliability, performance, and security. Typically done using tools like Postman.
Load Testing: the team models the expected usage of the software by simulating multiple users accessing the program concurrently. This is done using tools like JMeter, K6, etc.

Examples of independent party testing:

Penetration tests (“Pentest”) is when an independent external party conducts a detailed security assessment of your application and simulates the malicious activities of real-world attackers to identify security holes in the systems or applications. This assessment can be complemented by humans or by 3rd party Pen Test systems such as Detectify, Nessus, etc.
System audit: a third party conducts a systematic review of the application and underlying infrastructure to identify the RAID (Risks, Issues, Assumptions, Dependencies) in the solution architecture and source code, extract the future product roadmap and assess the potential strategic IT considerations based on the existing architecture runway.
Smart contract audit: in the case of blockchain projects, a smart contract audit is an extensive methodical examination, verification and certification of a smart contract’s code to ensure the smart contract delivers on its intended value. Organizations that provide such certification include the Blockchain Consilium.