T Ashok @ash_thiru
Summary
In this article I have tried picture(ise) the landscape of the plethora of tools for testing software which has moved away from just testing to build-test-deploy in a continuous manner. Keeping the interesting visual I have listed the FIFTEEN broad categories of tools that make up the modern digital testing landscape.
Given the plethora of front ends for digital applications of today, from PCs to tablets and mobile phones wth varied form factors, OS and browsers, with varying connection speeds, sometimes uncertain, the integration with many external systems via services with demands on non-functional attributes and the frequent nature of releases has made the challenge of automation and keeping in sync harder.
In this article, I have attempted to picture(ise) the landscape of test tooling with the entity-under-test(EUT) at centre (note that an EUT may be a small component, a subsystem or the complete system) with multiple ways to access it via API, Message/Service or UI on the left, evaluation by various test types at the top, the EUT enclosed in a deployment environment that may be an container/system with various ‘platform choices’. Keeping this as the base I have attempted to enumerate the various activities related to evaluation as inject/stimulate, observe/measure, validate/oracle, log/record and generate mocks as necessary to test for functionality or other non-functional attributes in the larger context of test design, automate, build and deploy on a variety of platforms as necessary.
Using the above picture we have the FIFTEEN categories of tools with some example tools as a table below.
| # | Category | Details | Example tools |
| 1 | inject/stimulate | Enabling accessing the EUT via API, Service/Message or via UI | xUnit, SoapUI, Postman, Selenium |
| 2 | observe/measure | Enabling run time aspects of EUT | Coverage, Resource leaks |
| 3 | validate/oracle | Enabling assessment of pass/fail | File comparators, Asserts |
| 4 | log/record | Enabling logging data, test information | log utilities, test execution recorders |
| 5 | mocks | Provide stubs for yet-to-be developed code | Mockito, Wiremock |
| 6 | non-functional test tools | Enable assessment of non-functional attributes | JMeter, SonarCube |
| 7 | platforms | Enable testing on different mobile devices with different browsers | pCloudy |
| 8 | virtualisation/deployment | Enable visualisation and deployment of code | Jenkins, Tricentis TOSCA |
| 9 | mocks/simulators | Enable simulating or mocking large systems | Payment gateway simulators |
| 10 | test design | Enable design of test case via specification based testing, BDD | Cucumber, SpecFlow |
| 11 | test data generation | Enable large test data generation | Mockaroo, Worksoft |
| 12 | build | Enable building of code | Ant, Maven |
| 13 | test management | Enable the management of tests | Jira, TestRail, PractiTest |
| 14 | unit test | Enable unit testing | xUnit |
| 15 | system test | Enable testing of full system via UI | Selenium, TestComplete |