SmartQA Community

8 Heuristics for identifying corner cases for testing


Corners are interesting as they are subtle, invisible really. They could be complex with many things that intersect and therefore display an unique behaviour. They may not necessarily be symmetrical at ends, nor be similar to behaviour in the middle.

As a developer focused on solving a problem for typical or generic cases one may not see the interesting extremities. For example we do everything right for a system in normal state, but miss out what happens when it is brought up the first time. This article outlines eight heuristics I discovered when testing a product that we were building, a SaaS platform. The heuristics outlined are based these aspects : Time, Lifecycle, Transformation, Position,Space, Size, Linkages and Limit.

#1 Heuristic based on TIME

The first time when something is used. Thinking from the perspective of time. Doing something on an absolutely fresh system. Creating the first project, registering the first user.Doing the final action, removing an user. First time when a transaction is done on a empty system. Purging a system of content, signifying removal, the end.

#2 Heuristic based on LIFECYCLE

Repetition of system states in terms of cycling through. Starting off, then doing activities and coming to an end. Then restarting and continuing. A workflow that is half done, suspended and then continued to finish. Finish by abandoning it  or ending with to a logical closure.

#3 Heuristic based on TRANSFORMATION

Changing something like say formats, views an act of transformation. In the case of UI, this could be relate to responsiveness like reaching the extremities of views? In case of content transformation, reaching the extremities of too large or too small or null content being transformed.

#4 Heuristic based on POSITION

Looking for interesting behavior in case of the elements that are right at start or end. What happens when elements in the middle move to either ends?

#5 Heuristic based on SPACE

The nation of space as when contents close are far, shrunk or expanded, especially when at extremities of too close or distant, too small or large. An example of  responsive UI, when screen is shrunk or expanded.

#6 Heuristic based on SIZE

The notion of volume, size when some is really small or extraordinary huge. Say uploading super large files, or really nothing or rally small ones. In case of display, showing tiny/large content/diagram, maybe via zoom?

#7 Heuristic based on LINKAGES

Using notion of  linkages like  1-1, 1-N, N-N or thinking in terms of increasing chain length like 1-1-1, N-1-N.   Could linkage integrity be violated when N=0 or when chain is long? The notion of propagation especially in longer chains with differential N.

#8 Heuristic based on LIMIT

The most commonly understood that of min/max, the typical extremities of value given an definitive range.

Transforming test teams

(In this SmartBits, Sudhir Patnaik outlines “ Transforming test teams “.  The video is at the end of this blog)

As a leader running platform engineering organization, industry changes made us reimagine how we test software platforms. We decided that in the context of extreme ownership mindset by Scrum team it is best that we transform quality engineering organization and merge them with Scrum teams.

This required us to look at the entire quality engineering organization, find out people well-versed in functional, performance testing and  security testing,  and automation. In a step-by-step process, we did a shift left of quality engineering organization, re-skilling many engineers towards learning the development technologies. We moved one function at a time into the respective Scrum team, and gave them about a year to transform themselves into developers. We provided every training that was needed.At the same time, we also trained developers to embrace functional testing concepts. If one is the owner of a service, one owns its unit testing, code review and its functional testing too.

From Quality Engineering perspective this meant that half of quality engineering organization ended up merging with the Scrum team and transforming themselves into developers in a period of one year. The remaining half of the team still stay as a central organization, but largely responsible for end-to-end and non-functional testing.

They are not executors of end-to-end and non-functional testing scripts, they have transformed themselves to become more creators of tools, frameworks and scripts that are needed to get these done. Then who executes those – “Well it is the Scrum team”!

Adopting DFT thinking

(In this SmartBits, Girish Elchuri outlines “Adopting DFT thinking“.  The video is at the end of this blog)

Understand big picture Part 1
When developing a product, one must have an understanding of the big picture. It’s not only about the product, it’s about the entire process of developing it. Putting oneself in the shoes of every role is a must be it a tech writer writing product documentation, quality engineer testing the product or a salesperson. One must understand the function of each of these roles for the success of the product . We must facilitate all these roles including quality while developing a product. We must treat ourselves as a product manager, a documentation person, a quality engineer or a marketing person in order to understand the roles better.

Put yourself in every role
While trying to understand each role, we need to put in more efforts to make sure that the requirements of each and every person are met. Understanding the big picture from a product perspective, process perspective, and having an attitude of making this successful and facilitating this for every role in the product development and launch is certainly required. This is what is needed to be done and this is how we can contribute to make sure we achieve quality.

Understand big picture Part 2
Quality is not achieved by just meeting functional requirements, customer requirements, quality requirements or sales requirements. For example a perfectly working umbrella when opened the wrong way, passes the quality, but does not result in any sales as it is no more useful to the customers just like the popular phrase “operation successful but patient dead”. This should never be the case. At the end of the day, that product should succeed in the market Hence we need to have an understanding of the big picture and this is what needs to be understood by every person in the organization.

Inject probes to test better
You talked about the ability to put in hooks, put in test switches etc. Do you think that is also a mentality that I have to develop, to facilitate the probing the system?

Absolutely. I put in probes to do scalability testing. If there are a few isolated test cases that I am running manually, I can always go and look at the log and manually do these things but when I am doing scalability testing, that’s where I find the scale at which things will happen. It’s not possible to manually collect this data and write them, hence the probes to collect such information.

Attitude key to probe design
So in my opinion, it’s the attitude that matters. When you have an attitude that I want to help this guy, be it a test engineer, documentation person or salesperson, then what I do is to facilitate those things to happen in the product. What will a sales guy want to do? Sales guy would want a demo system with live data. You should facilitate that in the product. You should facilitate creating scenarios so that it can be shown to customers. You cannot expect the sales guy to do it after the product is developed without assistance.  That is why I say that attitude is key to fulfil the needs & expectation of every single person associated with the product.

Deep dive to rapidly understand an entity


Deep dive is about going deeper, by reading available spec, by exploring, and using information from past experiences to understand an entity. This is done rapidly with a sharp focus and a 360 vision in eight steps.

The original article is at CLICK HERE to read the complete article.

20 approaches to Smart Test Design

T Ashok @ash_thiru on Twitter


This article outlines twenty approaches to smart test design based on seven views of user, logic/analysis, construction, test, experience, operational and evolution.

Let us talk a bit of test design now. We focus a lot of execution, and therefore the ability to cover more. The focus has veered to how frequently we are able to execute the tests and therefore on automation. Let’s step back and ask to what the objective was. Well it was to primarily deliver clean code. So we need to have a deeper sensitivity to the quality of tests. This is where design becomes important.

So what is a smart way to design to come with good scenarios, ideally few that can uncover issues that matter most.

Smart Test Design is about looking at the system from multiple views to:

I want | expect | would-like behaviours to satisfy needs that are implemented well and comprehensively covered to help me do well on my environments with no side effects.

Then decide what you want to prevent, statically detect or test, be it via human or a machine. Focus on intent and then the activity.  

Let me list down TWENTY approaches to smart test design looking at the system from SEVEN views.

1. User view based

1.1 Use the requirement specification to design.
1.2 See actual users of how they work and then use this to design.
1.3 With users doing experience sessions with system and use this information to design.

Smart test design based on 'user view'

2. Analytical view based

2.1 Use software test techniques (black or white) on spec/structure to design.
2.2 Construct behavior models and use this to design. 
2.3 If the system needs to comply with a standard, use the standards information to design. 

Smart test design based on 'logic/analysis'

3. Construction view based

3.1 Use the code properties like lines/conditions/path to design.
3.2 Exploit your deep understand of technology to understand potential mechanisms/flaws and use this design.
3.2 Understand how the system has been architected, composed & integrated to design. 

Smart test design based on 'construction view'

4. Test view based

4.1 Hypothesise potential faults probable given your understand of usage, structure, architecture, environment, conditions to design.
4.2 Use potential error return codes, exceptions, deliberate bad inputs, violations of system states to design.
4.3 Identify potential end failures and failure modes and use this to design.
4.4 Explore the system to understand its behaviour in various contexts and use this to design .
4.5 Probe the system with a series of questions (say what-if) and use this to design.

Smart test design based on 'test view'

5. Experience-view

5.1 Use the past history of issues encountered with various customers and design.
5.2 Apply the learning of various situations or deeper knowledge to come u with fault patterns and use this to design. 

Smart test design based on 'prior experiences/support'

6. Operational-view

6.1 Use the understanding of actual business flows, usage profiles of features on various environments to design.
6.2 Identify various deployment configurations and use this information to design.

Smart test design based on 'system operations''

7. Evolution-based

7.1 Use the information of what code has been changed, to understand how these may propagate and may cause issues to design.
7.2 Use the information of changes in the environment to understand their impact on the system and design 

Smart test design based on 'system evolution'

Smart Design is about:

I want | expect | would-like behaviours to satisfy needs that are implemented well and comprehensively covered to help me do well on my environments with no side effects.