by T Ashok

In a consulting engagement with a product company, I noticed that the tester came up with a complicated decision table to represent the behaviour model. On closer examination I noticed that he had arrived at behavioural conditions by examining the complex UI visually, rather than digging deeper to understand the behaviour. This is not the first time I have seen this, I have observed the challenge with behaviour modelling many times in numerous companies.

So “why does a complex UI faze a tester?” It is possibly the inability to see beyond the UI, the inability to question beyond the obvious. The result –  too many test cases, and the overwhelming feeling that scientific modelling & design cannot be practiced due to time constraints. Well this is the fun part of testing ‘the intellectual one’, where one decomposes an entity through a process of rational analysis and utter curiosity by thinking better to test smarter.

So how does one smartly model the system with a rich UI? How does one go beyond the veneer of the what is visible and see the various behaviours offered by this rich UI and smartly design scenarios that comprehensively evaluate the system?

What is the problem that a rich UI poses to a tester? 
What I have observed is that a rich UI results in ‘spaghetti test cases” . This is illustrated in this video below (1m:48s).

Rich UI problem analysis
Let us now analyse a rich UI of a real software to understand the challenge posed. (2m:48s minute video)

Decompose clearly to enable good design
Now let us dig deeper into the problem by analysing the various elements to test in this rich UI so that we can decompose clearly to enable good design. (3m:30s)

So what is good design?
It is about high cohesion and low coupling, as we say in software design. Let us see how we can apply this in the context of testing for good test design. (4m:19s)

Application of design principles
So what we do we now? How do we apply the good design principles here? 

1. Decompose ‘what-to-test’ into various of entries of different granularity : Fields, Features, Flow and Screen. 
2. Use a simple checklist (which is a really standard set of scenarios) to test the smaller granularity entries – Fields and Screen 
3. Model the behaviour of the higher granularity entity – Feature and Requirement maybe, using say a ‘Decision table’ and generate various test scenarios

This is outlined in the video below (2m:21s)

Summarising

1. We have ‘decoupled’ the complex UI into  ‘cohesive’ entities – Field, Feature, Flow and Screen.
2. We then applied a simple checklist to test Fields and Screen  and then applied behavior driven approach(say decision table) to design scenarios for each feature.
3. To test a flow, we combined the various unique outputs of each feature to generate the flow test scenarios.

---

In today’s world of Agile development, where minimal documentation and rapid evolution of features rule, understanding what may be appropriate for end users, what may be intended behaviour and what is actually present, requires not only a logical mind but a very creative mind. A smart approach to understanding the various parts, connecting to the bigger picture, and identifying missing parts in a rapid manner is the order of the day.

It is interesting that in the current technology/tool rich world, we have realised that human mind is the most powerful after all, and engaging it fully can solve the most complicated problems rapidly.  One of the key ingredients of an engaged thinker is “Thinking visually”, to clearly see the problem, solution or gaps. 

Design Thinking relies on sketching/drawing skills to imagine better ideas, figure out things, explain and give instructions. Daniel Ling(1) in his book “Completing design thinking guide for successful professionals” outlines this as one of the five mindsets – “Believe you can draw”. 

Sunni Brown(2) in her book “The Doodle revolution” states “doodling is deep thinking in disguise – a simple, accessible and dynamite tool for innovating and solving the stickiest of problems“ by enabling a shift from habitual thinking pattern to cognitive breakthroughs. 

David Sibbet(3) a world leader in graphic facilitation and visual thinking for groups in his brilliant book “Visual Meetings” outlines three tools for effective meetings to transform group productivity : (a) ‘Draw’ to communicate visually (b) ‘Sticky notes’ to record little chunks of information and create storyboard (c) ‘Idea mapping’ which are visual metaphors embedded in graphic templates and worksheets to think visually. 

Dan Roam(4) in “Show and Tell” states that the three steps to create an extraordinary presentation are (a) Tell the truth (b) Tell it with a story and (c) Tell the story with pictures. The book ‘written’ beautifully in pictures entirely is about ‘how to understand audience, build a clear storyline, create effective visuals and channel your fear into fun’. 

Jake Knapp(5) in “Sprint – How to solve big problems and test new ideas in just five days” outlines a five-day process to problem solving relies on SKETCHING on Day 2. He says that “we are asking you to sketch because we are convinced it’s the faster and easiest way to transform abstract ideas into concrete solutions. Sketching allows every person to develop those concrete ideas while working alone”. 

Not to forget my very good old friend “mind mapping”, from Tony Buzan which I have extensively used to understand, document, come with up with ideas and the plan.

It is interesting to note that visual thinking has taken centre stage now with mind mapping, sketch noting and doodling as a means to unleashing the power of the mind. 

As software QA folks, going beyond the given documentation (detailed or otherwise) is very necessary. In the current times with documentation being terse, a logical approach to exploring and covering a lot of ground rapidly and taking notes that forms the basis for a great visual summary is very necessary.  Mind maps have been very useful here and we now have other tools like SketchNote and Doodling.

These visual tools are useful not just for capturing information to aid understanding but can applied to create plans, and design scenarios to validate better. These allow us to move from the structured templated documentation to a breezy and creative way that not only allows to expand our thinking but ensures that we are terse and focused and in a flow. 

As I explore the product and ideas start to flow, visual tools are what I resort to using both software tools and good old paper to ensure that I can capture this rapidly and ensure I stay in the flow.

I use mind mapping extensively to take notes as I explore a product to understand using the tool iThoughts on my iPad along with sketches and doodles in my notebook accentuated with colour pens and PostIt.

Templates that have typically served as the backbone for test documentation are like “horse blinders”, for they provide a sharp focus in a narrow field restricting purposefully the peripheral vision enabling strict compliance. On the contrary “Fish Eye Vision” allows for a 360 degree vision to be able to see all around. Visual thinking enables this “fish eye vision”.

“Software testing is a funny business where one has to be clairvoyant to see the unknown, to perceive what is missing and also assess comprehensively what is present ‘guaranteeing’ that nothing is amiss.”

As much as tools and technology helps us to perform tests far better,  “Smart understanding and documentation” based on visual thinking tools can be very useful in today’s rapid product development cycle.

Testing is scientific exploration and the first step to do this smartly this is
“Exploit visual thinking – Smart exploration of software”.

---

References
(1) Daniel Ling “Completing design thinking guide for successful professionals”, CreateSpace Independent Publishing Platform, 2015.
(2) Sunni Brown, “The Doodle Revolution: Unlock the Power to Think Differently“, Portfolio, 2014.
(3) David Sibbet, “Visual Meetings: How Graphics, Sticky Notes and Idea Mapping Can Transform Group Productivity”, Wiley India Private Limited, 2012.
(4) Dan Roam, “Show and Tell – How everybody can make extraordinary presentations” Penguin, 2014.

(5) Jake Knapp, “Sprint – How to solve big problems and test new ideas in just five days”, Bantam Press, 2016.

---

In the current world of rapid development, software is constantly updated with new features, incremental additions and bug fixes. While features (new & incremental) are the focus for revenue generation and market expansion, bug fixes are necessary to ensure that customers stay.

“As product grows, so does regression,
increasing costs & threatening to slow down releases”

So what can be done to solve this challenge? Automate, right? But can we automate all the scenarios right from the early dev tests all the way to end-to-end flow tests? A significant of the latter is pretty difficult, right? And these do consume time and effort to build. Not that all the earlier tests will be completely automated either. Also automation is a catch up game with the software updated continually. 

So what else can we do smartly ? Well in addition to whatever we can do via automation to relegate the effort to a machine we can look at two ways to lessen regression (1) Do less (2) Don’t do !

What would be a good way to analyse the impact of change? i.e What other entities may be affected and what aspects of the entities may be impacted? The technique “Fault Propagation Analysis” of HBT (Hypothesis Based Testing) can be very handy. 

The first aid uses this technique and is illustrated in this picture:

Given that one of the most preferred approach to efficient regression is automation, how do you ensure the automated scenarios can be kept in sync with the constant updates in the software? Well the logical answer is short and less complex scripts.

So how can we achieve this? By ‘level-ising’ scenarios that implies scenarios are decomposed into multiple scenarios each focused on uncovering a specific type of issue of a specific entity. What we want to do is prevent  ‘spaghetti’  scenarios that attempts to uncover multiple types of defects over various entity.

For example a scenario that attempts to validate inputs, validate the UI aspects of an input in terms of say colour or format, validate  logical behavior correctness, and compatibility on different devices is a sure short recipe for disaster. Automating this scenario would result in a complex script making this fragile and potentially maintenance heavy. 

The second aid titled “Automation fitness analyser” illustrates how to analyse the fitness of scenario(s) for automation so that we can re-test efficiently. Remember this is about ‘Do-less’.

Moving on to the third part of tackling change smartly, the objective is “what scenarios can we avoid executing” especially those that are not yet automated.

This is illustrated in the third aid “Yield Analyser” illustrated below. The gist is to analyse test cases that have continually passed over the last few cycles and use this information to judge what parts of the system may have ‘hardened’ and therefore not execute these scenarios.

In summary it is about “How can we do less to do more”?  

Do less regression or Don’t regress and Do less automation maintenance is what “Smart Regression Testing ” is all about. The three aids outlined enable you to smartly accomplish this.

If all regression test scenarios have been automated, then it is real smart use of technology!

---

In today’s world of extreme agility, intelligent systems, SAAS, automated test and deployments, what is the role of humans in QA? How has the way we test software changed?

We live in the age of speed and smarts. Will machines take over testing? Will testing be subsumed into dev? With systems becoming very intelligent, it is only natural that testing become far more smarter. The theme of Smart QA explores the various dimensions of smartness and how we need to reinvent what we do.

To meet today’s demand for speed of delivery, automation seems to be the way forward. With a wide range of testing tools spanning across the entire lifecycle covering all design, execution, management, data creation, structural assessment spanning all types of tests from early stage development test to system test supporting a plethora of technologies, automation is deeply entrenched in how we test software. Is testing therefore about creating automated suites and setting up a pipeline to execute these periodically and automatically?

Yes, automated suites are the order of the day as we move into a continuous delivery mode. The continual enhancement of software demands a mechanism that enables a frequent system health check ensuring nothing is broken as we continually enhance the software. This is where tools and automation come in very handy. 

Let us step back as to what we are attempting to accomplish when we test. Well, we are looking for potential issues that would upset the end users experience and risk the business outcomes. So would the automated suites that run periodically uncover potential newer issues as they seep in? Hmmm, they cannot as they are programmed to execute the same paths/flows with the primary focus of ensuring good health. So what do we do? Add/update the automated suites continually and also test humanly. Note newer scenarios are created and some converted into suites, however not all can be figured out and automated as creativity cannot be boxed. It is necessary to be smarter to outwit the elusive bugs. As software is continually modified becoming richer, automated suites need to be synched with them too. A well-formed test strategy and a well-architected test assets are necessary to exploit the power of automation continually. 

So where smartness figure in this? Is it not just plain tooling tech to come up with scripts/suites? Let us take a deeper look. What we are doing really is converting our scenarios into scripts and also using appropriate tools & frameworks to enable static assessment and incorporate this into the tool chain. So never forget that the test scenarios to  are really the key to effective testing. To be able to this well, we need to be  able to understand the system context well, decompose the system into various sized entities to test so was to come up a great strategy, to create well-organised test assets(scenarios/cases/data) covering all the different-sized entities and ensuring these ‘tightly’ cover the software ensuring high coverage spanning all types of tests right from early dev stage to whole system stage. All this up front work demands brilliant smartness, before we exploit tech and tools to make the execution efficient. And this is why I think “Tooling and automation are great, but Smart QA is brilliant”.

Smart QA is about doing less, doing quick and achieving far more. In every facet of the lifecycle be it from the earliest stage of understanding, to exploring the context, charting out the scenarios, scripting suites, intelligent regression, doing static assessment to continuous course correction, it is necessary to do the least, do the fastest and get great outcomes.  Let us take a look at the “smarts” in the various parts of the lifecycle in detail.

In today’s world, information as to what is expected is not spelt in great detail. It is terse and simplistic, that is at best clarified by ‘acceptance criteria’. It is no more elaborately articulated as was done in the past. It is expected that the tester explore the system far more carefully and creatively to discern what may have been intended, what would be most meaningful for end users and then ask questions to clarify, to understand, or come up with scenarios to evaluate. Smart exploration is what is needed to rapidly understand the context, and how visual thinking can aid is discussed in “Exploit visual thinking – Smart exploration of software”.

The UI of today’s software is very rich and contains very many features packaged densely on a small screen. I have noticed that how one decomposes a rich UI to into various entities to test, decides how well and how quick one can evaluate. Smart modelling is key to this, this is dealt with in detail in the article “How to test rich UI – Smart Modelling”.

Software is continuously modified necessitating constant retesting/regression. Automated testing is very handy but not everything can be automated. It may not be really necessary to test all every time as systems harden with time. How can we do regress less is where Smart Regression comes in and this is examined in detail in “Three Aids for Smart Regression Testing.

Bugs seep in right from earliest stage of code development/modification, which we believe should be uncovered by automated unit tests. This does seem right, but what a minute – Can we become sensitive to issues that may seep at the early dev stage and prevent these, or detect this as cheaply as possible using ‘smart checklists’ as is done in other mature industries rather than resort to unit test?  What a smart checklist is and how this can help is the crux of the article “Smart checklists make you think”.

Smart QA is really about applying intellect to deliver ‘clean software’ by doing the least amount of work and exploiting tools to do the grunt work or be the enablers. 

“Tooling and automation are great, but Smart QA is brilliant”