SmartQA Community

8 Heuristics for identifying corner cases for testing

Summary

Corners are interesting as they are subtle, invisible really. They could be complex with many things that intersect and therefore display an unique behaviour. They may not necessarily be symmetrical at ends, nor be similar to behaviour in the middle.


As a developer focused on solving a problem for typical or generic cases one may not see the interesting extremities. For example we do everything right for a system in normal state, but miss out what happens when it is brought up the first time. This article outlines eight heuristics I discovered when testing a product that we were building, a SaaS platform. The heuristics outlined are based these aspects : Time, Lifecycle, Transformation, Position,Space, Size, Linkages and Limit.

#1 Heuristic based on TIME

The first time when something is used. Thinking from the perspective of time. Doing something on an absolutely fresh system. Creating the first project, registering the first user.Doing the final action, removing an user. First time when a transaction is done on a empty system. Purging a system of content, signifying removal, the end.

#2 Heuristic based on LIFECYCLE

Repetition of system states in terms of cycling through. Starting off, then doing activities and coming to an end. Then restarting and continuing. A workflow that is half done, suspended and then continued to finish. Finish by abandoning it  or ending with to a logical closure.

#3 Heuristic based on TRANSFORMATION

Changing something like say formats, views an act of transformation. In the case of UI, this could be relate to responsiveness like reaching the extremities of views? In case of content transformation, reaching the extremities of too large or too small or null content being transformed.

#4 Heuristic based on POSITION

Looking for interesting behavior in case of the elements that are right at start or end. What happens when elements in the middle move to either ends?

#5 Heuristic based on SPACE

The nation of space as when contents close are far, shrunk or expanded, especially when at extremities of too close or distant, too small or large. An example of  responsive UI, when screen is shrunk or expanded.

#6 Heuristic based on SIZE

The notion of volume, size when some is really small or extraordinary huge. Say uploading super large files, or really nothing or rally small ones. In case of display, showing tiny/large content/diagram, maybe via zoom?

#7 Heuristic based on LINKAGES

Using notion of  linkages like  1-1, 1-N, N-N or thinking in terms of increasing chain length like 1-1-1, N-1-N.   Could linkage integrity be violated when N=0 or when chain is long? The notion of propagation especially in longer chains with differential N.

#8 Heuristic based on LIMIT

The most commonly understood that of min/max, the typical extremities of value given an definitive range.


Simplifying rich UI testing

T Ashok @ Ash_Thiru on Twitter

Summary

Why does a complex UI faze a tester? Is it possibly the inability to see beyond the UI, the inability to question beyond the obvious? Too many test cases, the overwhelming feeling that scientific modelling & design cannot be practised due to time constraints are some of the issues. This article outlines “How to decompose, question & analyse” to design smartly using a complex UI entity as an example.


During a recent consulting engagement, I noticed that a tester came up with a complicated decision table to represent the behavior model. On closer examination I noticed that he had arrived at behavioural conditions by examining the complex UI visually, rather than digging deeper to understand the behaviour. This is not the first time I have seen this, I have observed the challenge with behaviour modelling many times in numerous companies.

So “why does a complex UI faze a tester?” It is possibly the inability to see beyond the UI, the inability to question beyond the obvious. The result – too many test cases, and the overwhelming feeling that scientific modelling & design cannot be practised due to time constraints. Well this is the fun part of testing ‘the intellectual one’, where one decomposes an entity through a process of rational analysis and utter curiosity by thinking better to test smarter.

A rich UI typically has large number of data fields, some dependent on others or system settings, possibly in multiple tabbed dialogs, and doing the same thing in different screens i.e. a feature available in multiple places. This seems to pose difficulty in (1) “where-to-start” to model the behaviour (2) mixing test cases related to fields, UI and behaviour, business flow & environment dependencies.

Let us look a problem to see how we simplify testing of a rich UI using an airline application, done in four screens, each of one which seems complex.
(From http://awery.net/pages/software.html#page-screenshots)

1.Setting up a flight schedule for a flight

2. Setting the route information -“start location details”

3. Setting up the route information – “permissions”

3. Setting up the route information – “cargo details”

4. Setting up the route information – “setting up crew”

What do we see in the rich UI?

Let us analyse…
1. Firstly we can see data fields – independent and dependant one.

2. Secondly we see feature/function (some of them a mini feature) and a business flow that is really a combination of many features.

So, this single rich UI screen consists of a set of features, and enables a complex flow to be done. How do we test this?

Let us look at good design principles “COHESION & COUPLING “. Good design of software is high cohesion and loose coupling. Grouping similar actions & data (cohesion) keeping minimal dependencies between them (coupling).

How is this useful in TEST DESIGN for rich UI?

Let us dig in…

So, what have we done to tackle challenges of complex UI?

Well, we have:
1. DECOUPLED complex UI into COHESIVE entities : FIELD, FEATURE, FLOW & UI
2. Applied behavior driven approach for feature test design (decision table)
3. Combined outputs (variations) optimally to test the FLOW (variations table)
4. Used checklist as needed to leverage prior design


In a rich UI, one tends to see HOW-to-do, whereas if it was UI-less, one seems to focus on “WHAT-is-to-be-done.
The latter is important for a tester to figure out ‘WHAT-ifs’. 

T Ashok

Be in a Flow. Test Brilliantly.

T Ashok (@ash_thiru on Twitter)

Summary

Good testing is a great combination of intellect, techniques, process and heuristics.  What does it take to do brilliant testing ? It requires one to be immersed in the act, be focussed yet unbounded, be keenly observant but non-judgemental, with outcome that are beautiful, the activity so joyful that time stops. A state of flow. What does it take to get there?


As engineers we use good techniques, tools, process and intellect to do things well. So how can we do far better, that is brilliantly? Having possibly exhausted all things “external”, in terms of tools, techniques, process and intellect, it is time we considered the huge “internal” potential that we all possess.

It is about going beyond the intellectual mind, deeper into the sub-conscious to harness the infinite potential.  A good start for this would be get into a state of ‘flow’.

So, what is FLOW?
Flow is the state when you are immersed in what you are doing, where you are totally mindful. It is when all energies are harnessed fluidly to do brilliant work without tiring or trying hard, becoming an observer, fine tuning actions with extreme agility, when time seems frozen. It is when you accomplish a lot, doing work effortlessly and experiencing absolute joy. 

So how can we get into the state of flow?
When multiple sensory excitations converge harmoniously, there is a good chance of entering that state of flow. What does that mean? It is engaging the various senses well – colours, picture, visual test, mind maps for the eyes, using pen/pencil, paper for the touch, and maybe background music or talking to oneself for ‘engaged hearing’. Note the keyword is ‘harmonious’.

When we stimulate our creative side with interesting visuals, tactile and possibly sound, it activates us to get into an ‘engaged state, a state of mindfulness. 

Exploit visual thinking by using visual text , sketch maps, mind maps  to : (1) enable  deeper understand the system under test (2) to sketch test strategy (3) jot down test ideas (4) note down observations, questions, suggestions (5) list scenarios (6) record issues.

Exploit the tactile sense by using pen/pencil on paper or finger/stylus on tablet instead of keyboard.  The objective is to be write/draw freely rather than be constrained by the keyboard. 

If you want to engage your auditory sense, quietly talking to yourself, melodious quiet whistling or if you are music person, then a suitable background song played could enable you to get into the flow. 

To ensure that we can perform in the state of peak performance being in a FLOW, it is imperative that we do testing in short sessions. A session may be anywhere between 45-90 minutes. The key is to stay focussed, setup an objective at the start of session and then engage as stated earlier to get into a state of FLOW.

How does this help?
When we are fully engaged,  in a state of FLOW,  it is no more about just left brain centred logical/deductive thinking, or the creative right. It is an expansive multi dimensional thinking brought about by the harmonious stimuli enabling one to become a keen observer and absorb deeply and rapidly. This is when we exploit the infinite power of what we possess, delving into the sub-conscious which is much larger that conscious mind. Interesting questions pop up,  ideas germinate rapidly, actions are done quickly, smallest observations captured resulting in brilliant effective testing.

In closing
Test automation allows us to do more, and machine intelligence to do better. It is now necessary for us to delve deeper so that we complement the machine rather than compete, enabling us to be super efficient and effective by being smarter. Get into the state of flow to harness the power of sub-conscious to do brilliant testing.

CLICK HERE to view the presentation slidedeck.


7 Habits to SmartQA

T Ashok @ash_thiru on Twitter

Summary
Smartness is a brilliant combination of knowledge and skill. Skill is acquired by intense practice. Intense practice happens when you have good habits. In this article I look at SEVEN habits that are key to SmartQA.


Habit #1
Visualise well, see the big picture of the system.
Who uses, where, when, using what interface. See the big picture of how it is structured/architected, visualise the flow of control and data and the technologies used(aka tech stack). Appreciate the environments it should run on the dependencies of external systems and changes/modifications done.

Habit #2
Question well. Question to know. Question to know what you do not know. 
Ask about users and their expectations ask about the how it is constructed, ask about conditions a behaviour should satisfy.
Ask, ask… The limit to questioning is unbounded. Note you may not get all the answers, the key is to question.

Habit #3
Prove with facts. 
Show proof of adequacy of tests, hypothesise potential issues and prove that they may be detected.
Understand changes done and prove why an entity(ies) need to be regressed, after all you do not want to more that needed.

Habit #4
Iterate continuously. 
SmartQA is about experimenting, exploration.
It is about iterating with minor changes to understand better, to observe better.
Chew well to digest.

Habit #5
Unlearn. 
Revise what you know.
 Understanding is not about knowing more, it is also discarding what you know. 
Chew to digest and spit out the rest.

Habit #6
Empathise.
Put yourself in others’ shoes and feel them.
Get inside them to understand what they expect, what they will dislike, their constraints, what value they expect of the system. SmartQA is not testing the system technically, it is about delivering value.

Habit #7
Simplify the problem solving.
End users use systems to solve their problems. Great software solve them well.
Good testing is about ensuring systems do just enough i.e. simply.


TEN suggestions for SmartQA

T Ashok @ash_thiru  on Twitter

Summary
SmartQA is a beautiful combination of thinking styles, a mindset of brilliance and minimalism,  tech prowess, heightened clarity and  great design and meaningful pauses, outlined as ten suggestions in this article.


#1 Embrace multiple thinking styles
Inculcate the deductive ability of a mathematician, creativity of an artist, mind of an engineer, value perception of a businessman, technical savviness, empathy, doggedness and nimbleness.


#2 Have a mindset of brilliant engineering
Step into end user’s shoes, architect/design robustly, inject code to aid testability, strive to test minimally, do test related tasks lightweight.


#3 Analyse well, exploit tools for doing
Much like the skill of a doctor to diagnose with exploiting the machines in the process. “Doing SmartQA”s a brilliant combination of “human powered and machine assisted” . The WHAT to-do is human while HOW-to-do is powered by machine/tools.


#4 Do minimally
Strive to prevent issues, embed testability, review code carefully, use smart checklists, write minimally, regress intelligently.


#5 See better, cover more, test less
Continuously see and assess product from multiple views – USERS, ATTRIBUTES, ENVIRONMENT, CODE, ENTITIES


#6 Pause to speed up
Periodically pause and analyse to be sure that you are staying on the right track, reflect on outcomes to ensure you are doing it right and efficiently.


#7 View system from multiple angles
View the system from internal, external, regulatory/compliance, operations, maintenance and release like code structure, architecture, technology, terms of behaviour, end users, environment, usage, standards.


#8 Be sensitive and aware
Be sensitive and aware to issues you encounter and potential causes of issues, after all issues creep in due to untold expectation, accidental omission, quiet assumptions, incorrect implementation,inappropriate modifications,interesting side effects ,deliberate abuse , and innovative usage.
Sharpen your senses  to smell better!


#9  Design for robustness 
Don’t just test, design system robustly. Codein firewallsto be disaffected by inputs, configuration/settings, resources or dependent code.


#10 Design for testability
Hook in code to be able to inject inputs to stimulate, check status, create traces/logs  to debug/check-for-correctness, even embed ‘self-test code’.

You may want to read a detailed version of these in these two articles FIVE thoughts on ‘Doing SmartQA’and FIVE *MORE* thoughts on ‘Doing SmartQA’.


8 things I do to solve problems creatively

T Ashok @ash_thiru on Twitter

Summary
Over the years there are a few things that I do consistently to solve problems technical or business. The process is magical, to see the larva of an idea become a beautiful butterfly, the solution. This article outlines this as eight things I do as a collection of posters with crisp text.


Explore problem by puttering around, to understand it better,
to try out mini  experiments of potential solutions to germinate ideas.
The path to solving a problem is never straight line.
Focus, meander, move continuously,
observe to form a good mental picture.
After all, great clarity is key to brilliant problem solving.

As you explore, picturise the various facets of problem and
various solution bits as doodles, mind maps, or diagrams.
Be non-linear, use colours to enhance stimulation of thoughts.

Experiment with the problem,
see if you can spot similarities to prior problems,
connect to get a better handle to understand
and vigorously pursue possibilities.

As you explore,experiment,
jot down interesting observations,
ideas, solution possibilities.
Be terse, so that you are
not distracted from what you doing.
After all, quick notes help in assimilating better
and jotting down quick ideas as they come by.

Describing the problem, explaining ideas,
solutions to a willing listener is something I do every time.
It clarifies my thoughts,
enables me to identify solutions that have been dogging me.
And this is just me explaining,
not including suggestions and more ideas
that I get from the listener.

Sometimes when mind is filled with information,
struggling to process, clueless about coming with ideas, solutions,
I stop whatever I am doing, empty my mind 
and in the utter calm, ideas/solutions fly in gently.

At times when it has been frustrating
to see nothing emerging to solve the problem,
I have found it wise to discard the pursuit
for solution and do something else.
Deflect my mind and engage in something else .
And after some time, suddenly ideas flash.

Once an idea to a solution presents itself, I work feverishly on it.
Implementing, refining, continually polishing by working feverishly.
Totally immersed, all senses absolutely tuned to the act of implementation.
Magical it is to see a larva of the idea become the beautiful butterfly of a solution.


39 tips to being productive – Do SmartQA

In an era where we are obsessed with productivity, it is not about doing more, of being busy that is deemed as high productivity. In fact it is the converse, of being smart, of doing less and accomplishing more.

Software tools help in increasing productivity by allowing us to do faster, cheaper and better. But the most powerful tool “the human intellect” can help us do lesser ad coupled with tools of speed, productivity scales geometrically. In these times of AI, it is necessary to exploit HI (Human Intelligence) to do stuff beyond intelligent machines to deliver a higher value.

Here are 39 tips to being productive, in the article “39 tips to being productive – Do SmartQA“.

Shift Right to Shift Left “Being creative”

T Ashok, @ash_thiru on Twitter

Summary

With machines become more intelligent enabled by AI, it is time we focus on creativity to think differently and exploit machines for its intelligence.
A focus in shifting to using right brain more will certainly help in the goal to “Shift Left” – that of prevention or doing less.
Good testing demands a great logical ability to dissect, decompose, analyse, synthesize, correlate, question, hypothesise, experiment, ascertain and make appropriate choices. It is surely a left brain activity. And tools take the outputs of these activities in becoming scripts to be used in an unattended mechanistic manner to reap significant benefits.


Having exploited the logical abilities and the technology to do well and faster, what next?

Testing is not just about finding issues, it would be valuable if this can help prevent issues or find it earlier before issues fan out. This is what we call as Shift Left, the ability to move left-ward in the software dev cycle to find earlier, ideally prevent. Today automation is seen as an enabler here too with the process, forcing one to think and come with scenarios/scripts before code commences or very immediately after code so that whatever is coded is validated. Given that tools are available in plenty, does not imply they have to be used every time, as it takes effort to script and maintain it.

Minimalism and simplicity are popular today, these can help in doing less by heightening sensitivity to aid in prevention and leveraging existing assets. Being creative enables sensitivity, the ability to not be pre-conditioned, to be random, to see beyond the visible and to come up with interesting possibilities. We know that covering everything from all angles seems impossible, certainly costly in terms of time, effort and money. Yes, we do resolve to use the logical ability to analyse risky areas and appropriately channel our energies.

The fertile right brain can cut through the chasm rapidly and enable significant benefits beyond the logical right brain. What does it take to cultivate a fertile right brain? What are the ingredients to fertility?

  1. Constraints – Setting up constraints enables creativity. When we are faced with pressure, we find means to do less, do quick and do well. It is like the survival instinct kicking in. Testing in short sessions helps in here.
  2. Write less – Writing long sentences as part of note-taking or worse still using software to document outcomes distracts you from exploiting the full power of creativity as we become logical in forming sentences and ensuring its correctness. Writing tersely in short phrases. Colours – Stimulate the visual senses too. Use colour sheets, pens, pencils, highlighters. Go beyond the black and white.
  3. Be non-linear – Sketch, mind-map, draw to express. Let not the act of writing be left-right & top-bottom deaden your senses. Sharpen creativity. After all minimalism is a constraint forcing us to go beyond to express sharp and crisp.
  4. Plan, yet stray – Before commencing a session, draw up a quick plan as to what you want to accomplish. As you go about the work, stray when you find something interesting, explore and improve understanding. Be sensitive to the clock though.
  5. Flow –  Enjoying the work enables one to stay immersed in the work, and that is where time stops, a state of being in the ‘flow’. This is when the creative side is most fertile. Sharp and observant, super efficient, light and nimble a state when the work and the worker become one.
  6. Favour paper – During a session, to take notes, to jot down observations/questions, pen down to-do-actions, adopt a lightweight method. Using paper to do this is the easiest, also using Post-It notes, white board is super light, enabling one to stay immersed in the work and not be bothered about the act of writing/drawing. Using a heavy weight word-processor or a spreadsheet IMHO is a no-no, as it is seriously high friction. If you prefer software, use a light weight note taking software.

Testing is not a mundane job of evaluation bereft of fun. It is not a clinical act of evaluation that is done by machines. Our job is not just to write scripts and feed these machines.

It is true that in these times, the focus is on speed, on the act of evaluation. Yes, a machine executes tests really fast and is useful in continuous assessment. In a continuous development scenario, it gives us a sense of confidence that we are marching forward and nothing is broken. But remember that the ability to trust that it is not broken is a function of test scenarios/cases.This is normally the result of intelligent thinking.

With machines become more intelligent enabled by AI, it is time we focus on creativity to think differently and exploit machines for its intelligence.
A focus in shifting to using right brain more, will certainly help in the goal to “Shift Left” – that of prevention or doing less.


Seven habits to healthy code

T Ashok @ash_thiru on Twitter

Summary


Healthy code is not about just working correctly. It is about future-proofing, maintainability, adaptability, reusability and so on. As in real life where face shines when you are in the pink of health, beautiful code also shines!

“Healthy code is not the outcome of review or testing, it is from doing simple things diligently.”

What comes to your mind when you hear the word ‘HEALTHY’?

It really is a combination of healthy mind and healthy body. So when I say ““HEALTHY HABITS”, what do you think of?

Now, let us relate to code. What is healthy code?

So what are the habits to stay healthy? Let us correlate these habits to software.

Let us reorder these seven habits …

“Healthy code is not the outcome of review or testing, it is from doing simple things diligently.”


Deep dive to rapidly understand an entity

Summary

Deep dive is about going deeper, by reading available spec, by exploring, and using information from past experiences to understand an entity. This is done rapidly with a sharp focus and a 360 vision in eight steps.

The original article is at https://stagsoftware.com/dosmartqa. CLICK HERE to read the complete article.