SmartQA Community

15 categories of tooling for digital test automation

T Ashok @ash_thiru


In this article I have tried picture(ise) the landscape of the plethora of tools for testing software which has moved away from just testing to build-test-deploy in a continuous manner. Keeping the interesting visual I have listed the FIFTEEN broad categories of tools that make up the modern digital testing landscape.

Given the plethora of front ends for digital applications of today,  from PCs to tablets and mobile phones wth varied form factors, OS and browsers, with varying connection speeds, sometimes uncertain, the integration with many external systems via services with demands on non-functional attributes and the frequent nature of releases has made the challenge of automation and keeping in sync harder. 

In this article, I have attempted to picture(ise) the landscape of test tooling with the entity-under-test(EUT) at centre (note that an EUT may be a small component, a subsystem or the complete system) with multiple ways to access it via API, Message/Service or UI on the left, evaluation by various test types at the top, the EUT enclosed in a deployment environment that may be an container/system with various ‘platform choices’.  Keeping this as the base I have attempted to enumerate the various activities related to evaluation as inject/stimulate, observe/measure, validate/oracle, log/record and generate mocks as necessary to test for functionality or other non-functional attributes in the larger context of test design, automate, build and deploy on a variety of platforms as necessary. 

Test Tools Landscape – 15 Categories of Tools

Using the above picture we have the FIFTEEN categories of tools with some example tools as a table below.

#CategoryDetailsExample tools
1inject/stimulateEnabling accessing the EUT via API, Service/Message or via UIxUnit, SoapUI, Postman, Selenium
2observe/measureEnabling run time aspects of EUTCoverage, Resource leaks
3validate/oracleEnabling assessment of pass/failFile comparators, Asserts
4log/recordEnabling logging data, test informationlog utilities, test execution recorders
5mocksProvide stubs for yet-to-be developed codeMockito, Wiremock
6non-functional test toolsEnable assessment of non-functional attributes JMeter, SonarCube
7platformsEnable testing on different mobile devices with different browserspCloudy
8virtualisation/deploymentEnable visualisation and deployment of codeJenkins, Tricentis TOSCA
9mocks/simulatorsEnable simulating or mocking large systemsPayment gateway simulators
10test designEnable design of test case via specification based testing, BDDCucumber, SpecFlow
11test data generation Enable large test data generationMockaroo, Worksoft
12buildEnable building of codeAnt, Maven
13test management Enable the management of testsJira, TestRail, PractiTest
14unit testEnable unit testingxUnit
15system testEnable testing of full system via UISelenium, TestComplete

#35 – “Great Expectations”

SmartQA Digest

In today’s world of heightened expectations, what does an organisation and end users expect of IT application team? Zulfikar Deen says end users expect application to work 100%, nothing less, and has a contrarian opinion on DevOps. He also says that management expects to showcase business benefits and hence it is necessary to bake-in metrics of adoption, usage & usability into the application… and much more in the SmartBites video Expectations of end users & management from IT team.
In the ‘beEnriched’ section I outline 25 things I expect of a great tester like 
(1) Be disciplined, but stay creative (2) Ask questions, find answers (3) Observe well, see things that are hidden and many more.
“It is easy to make perfect decisions with perfect information. Medicine asks you make perfect decisions with imperfect information” says Siddhartha Mukherjee in the book “The Laws of Medicine: Field Notes from an Uncertain Science”.  Check out what this wonderful book is about HERE!







Visual thinking and Sketchnotes- Anuj magazine

Ashok : As an avid visual thinker who uses Sketchnotes to communicate, please tell us the importance of visual thinking and how it can help us understand/think better and influence people?

Anuj : I think, one of the ways Visual thinking has helped me is to find a way to better stay in the moment and what I mean by that is when we are in the moment, we will be able to appreciate life even more than what it is. So, staying in the moment in one of the big benefits. While I don’t claim to be a big one, every artist seeks inspiration from life happening around them and the quest of seeking that inspiration itself is one that lets you live that moment better than in a condition without that, so that is one. 

The second way visual thinking I believe has helped me is – One of my online friends is Tanmay Vora, a very good Sketchnote artist. One of the blogs that he wrote and that stayed with me, talks about one of the principles he follows  – to consume less and create more. In essence, what he means by that is, people with all the revolution which has happened around smart phones are always consuming content. We can blame apps for it – in a way they have been designed to create that stickiness, but we are always in the consumption mode. What happens if we start eating lot of food? It shows up on our body. Ironically, consuming lot of content does not show up as visibly in our minds. We can feel our minds getting bloated up, getting overwhelmed with lot of stuff, but you got to catch those signals. So, in order to balance it out, one of the principles of consuming less and creating more comes into picture. How it helps the visual thinker in me is that if I read stuff, I try to restrict myself to reading good stuff, and whatever I read I have a kind of pact with myself that I will create something out of that, be it a blog, a sketch or some other consumable form. That really creates balance because you are not holding up information for too long and getting it to stale in your mind without it being put to the right kind of use.

Third way visual thinking has helped me is – I will tell you an instance where I had organised one of the sessions on quantum computing. As complex a subject as that is in today’s times, it was equally important for people to figure out how to explain it simple. So, one of the things that I had set myself to do in that session was to Sketchnote the session live. Eventually, it turned out to be a good summary and in doing so, I realised that Sketchnoting is helping me actively listen to the speaker. What I mean by active listening is that again I am not consuming the content for the sake of consuming the content. I am creating something out of it and also actively removing the noise out of the whole experience of listening. You can’t write each and every word in a Sketchnote, but you can write the key points and summarise it. I did present it to the speaker after the event. So, bringing in the ‘intention’ in the listening is one of the key traits that I learnt. 

Overall, the main areas where visual thinking has helped me, is to be more aware, be more present in situations and listen intently and balancing that continuum of creation and consumption which is important.

Ashok: So what you are saying is it does help you certainly be more mindful, absorb better and obviously assimilate it and keep up with the most important things so to speak and do it very continually along with the person who is doing his job.

Anuj:  I would like to add how will it help QA professionals.More often QA professionals find themselves in a situation where the bug reports, unfortunately still are considered as the key output. In absence of any innovation happening in creating new bug reports, they are again thought of as one of the predictable outputs from the profession. What if you create a Sketchnote out of a bug report? I think that might help people look into your bug with more interest and getting more motivated to fix them.

click to video