SmartQA Community

15 categories of tooling for digital test automation

Featured image of article 15 categories of tooling for digital test automation

T Ashok @ash_thiru


In this article I have tried picture(ise) the landscape of the plethora of tools for testing software which has moved away from just testing to build-test-deploy in a continuous manner. Keeping the interesting visual I have listed the FIFTEEN broad categories of tools that make up the modern digital testing landscape.

Given the plethora of front ends for digital applications of today,  from PCs to tablets and mobile phones wth varied form factors, OS and browsers, with varying connection speeds, sometimes uncertain, the integration with many external systems via services with demands on non-functional attributes and the frequent nature of releases has made the challenge of automation and keeping in sync harder. 

In this article, I have attempted to picture(ise) the landscape of test tooling with the entity-under-test(EUT) at centre (note that an EUT may be a small component, a subsystem or the complete system) with multiple ways to access it via API, Message/Service or UI on the left, evaluation by various test types at the top, the EUT enclosed in a deployment environment that may be an container/system with various ‘platform choices’.  Keeping this as the base I have attempted to enumerate the various activities related to evaluation as inject/stimulate, observe/measure, validate/oracle, log/record and generate mocks as necessary to test for functionality or other non-functional attributes in the larger context of test design, automate, build and deploy on a variety of platforms as necessary. 

Test Tools Landscape – 15 Categories of Tools

Using the above picture we have the FIFTEEN categories of tools with some example tools as a table below.

#CategoryDetailsExample tools
1inject/stimulateEnabling accessing the EUT via API, Service/Message or via UIxUnit, SoapUI, Postman, Selenium
2observe/measureEnabling run time aspects of EUTCoverage, Resource leaks
3validate/oracleEnabling assessment of pass/failFile comparators, Asserts
4log/recordEnabling logging data, test informationlog utilities, test execution recorders
5mocksProvide stubs for yet-to-be developed codeMockito, Wiremock
6non-functional test toolsEnable assessment of non-functional attributes JMeter, SonarCube
7platformsEnable testing on different mobile devices with different browserspCloudy
8virtualisation/deploymentEnable visualisation and deployment of codeJenkins, Tricentis TOSCA
9mocks/simulatorsEnable simulating or mocking large systemsPayment gateway simulators
10test designEnable design of test case via specification based testing, BDDCucumber, SpecFlow
11test data generation Enable large test data generationMockaroo, Worksoft
12buildEnable building of codeAnt, Maven
13test management Enable the management of testsJira, TestRail, PractiTest
14unit testEnable unit testingxUnit
15system testEnable testing of full system via UISelenium, TestComplete