by T Ashok

Summary
The act of testing is a scientific exploration of a system done in three phases – RECONNAISSANCE to understand and plan, SEARCH to look for issues, REST&RECOVER to analyse and course correct. To enable the various activities in each phase to be done quickly and effectively, is where the SEVEN Thinking Tools outlined in this article help. How to apply these tools in a session-based approach is also briefly outlined.

When I hear people talking about testing as Manual or Automated  with the latter being the need of the hour, I am flabbergasted. All the word ‘manual’ conjures in my brain is that of me doing a menial job of painful scrubbing!

It is time we used Intellectual & Tool-supported. “Think well. Exploit tools to do.”Enough of rant!

In current times, speed is everything, right?  What can we do to test quickly ? Use tools. Automate. Right? Wait a minute – This is about execution, right? What about prior activities?

To answer, let us ask the basic question what is testing after all? Testing is exploration. Let me correct it. Testing is scientific exploration.  And exploration is a human activity that is aided by tools & technology. How can we do scientific exploration rapidly? By using tools that help us think better and do faster.

Let us say you want to explore the nearby mountain range by foot. Would you just pick up your backpack and go on? I bet not, unless it is a really short trip. Otherwise I think you will study the geography/terrain, read others’ experiences, do a reconnaissance, create various maps of terrain, of pit stops, of food joints etc before you chalk out the full route. Once the route is setup, you will pack your bags and go. As you explore, you will discover that “the map is not the terrain” and be taunted, surprised, challenged and you will learn, adjust, improvise, revise the maps, routes as needed. Tired, you will rest, analyse, replan & recover to continue on your journey. This is not ad-hoc nor driven by sheer bravado. This requires logical thinking(scientific), planning and ability to observe, adjust continuously and also some bravado and good luck!

This is what we can apply too in testing our software/systems. This article distils this and provides you with SEVEN THINKING TOOLS to enable you do these easily and scientifically.

Applying the above analogy, we look act of testing as being done in THREE phases: RECONNAISANCE, EXPLORATION, REST&RECOVER.

RECONNAISANCE : Do survey and create maps
Survey : Get to understand the system under test by reading documents, playing with the software/system, discussing with people, to clearly understand who the end users are, what the entities (e.g. features, requirements..) to test are, what the various attributes the end users expect are, and the environment in which it will be deployed. In a nutshell we want to know the Who, What-to-test, What-to-test-for and Where. This is done by the Tool #1 “Landscaper”.

Create Maps: Now that you know the key information, connect them to four useful maps: Persona map, Scope map, Interaction map and Environment map.

(Tool #2) Persona map : A list that clearly connects  the “Who” to What”. This helps us understand who uses what and therefore helps us prioritise testing and certainly enables us to get user centric view to validation.

(Tool #3 )Scope map: A list that connects the ‘What’ to ‘What-for’. This helps us to understand what the expectations of the various entities are i.e. That for even feature F1, we have an expectation of performance. What does this help us do? This helps us identify the various types of tests to be done.

(Tool #4) Interaction map: No entity is an island i.e. each entity may affect one or more entities. i.e a feature F1 may affect another feature F2 and therefore a modification of F1 may require retesting of F2. How does his map help us? Well, this helps plan our regression strategy intelligently.  

(Tool #5) Environment map : This lists out the various environments on which the final system may be run so that the functionality and attributes may be evaluated on various deployment environments.

Now that we have done done the reconnaissance, we should have good idea of the system under test and therefore be ready to explore.

SEARCH : Now that we have the maps, the next step would be chalk out the routes and then we are ready to commence our search for issues. This is done using the “Scenario creator” tool. Once this is done we commence our search for issues. When doing this we will encounter things we don’t know, things that we did not anticipate, issues and therefore will need to revise and course correct revise the landscape, maps and routes. This is accomplished via the Dashboard tool in the Rest&Recover phase.

(Tool #6) Scenario creator: This tool helps to design the various test scenarios that would serve as the starting point. Note that these will be continuously revised as we explore and gain a deeper understand of the system and its context and usage. What is important would be segregate the scenarios into Levels so that the test scenarios are focused and clear in their objective. Robust Test Design approach of HBT helps you design scenarios that may be done using a mix of formal techniques, past experience, domain knowledge, context but clearly segregated into various HBT Quality Levels.

REST& RECOVER: In this phase, the objective is to analyse the exploration phase results and improve what we can do, track progress of doing and judge of the quality of the system-under-test. This is done by the tool ‘Dashboard’

(Tool #7) Dashboard: This tool helps you to do there things : (a) judge adequacy by look at the map and route information and improve the same (b) track progress of work done by checking the what has been done vs.planned as far as routes are concerned (c) judge quality by looking at the execution outputs of the scenarios level wise.

So how do we apply this tools?
We saw that these 7 tools could be used in the THREE phases of RECONNAISSANCE, SEARCH, REST&RECOVER by a session based approach “Immersive Session Testing”. Each session is suggested to be short and focused say 60-90 minutes with a session objective to one or a mix of the phases.

Note that a session could be an exclusive RECONNAISSANCE or SEARCH or REST&RECOVER on a combination of these. Why is the session time suggested to be 60-90 minutes? Well this is to ensure razor sharp focus on each on the activity done. Also a short focussed sessions allow one to get into a state of flow enabling higher productivity and enjoy the activity!

Click here to play the webinar video.

Click here to see the slides in SlideShare

About SmartQA The theme of SmartQA is to explore various dimensions of smartness to leapfrog into the new age of software development, to accomplish more with less by exploiting our intellect along with technology.  Towards this, we will strive to showcase interesting thoughts, expert industry views through high quality content as articles, posters, videos, surveys outlined as a SmartQA Digest weekly emailer. SmartBites is soundbites from smart people”. Ideas, thoughts and views to inspire you to think differently.

Signup to receive SmartQA digest that has something interesting weekly to becoming smarter in QA and delivering great products.

[grwebform url=”https://app.getresponse.com/view_webform_v2.js?u=hzsfs&webforms_id=BxdSM” css=”on/off” center=”on/off” center_margin=”200″/]

by T Ashok

Summary
The confluence of extreme speed of business, rapid new technology adoption, cloudification & platform-ising of apps, finicky end users has created serious inflexion points, threatening the  QA practice. How do we test smartly to survive & continue to grow? 

Smart testing is about doing less and accomplishing more, and  rapidly with razor sharp focus on business outcomes and end user experience.  It is about doing what it takes to ensure great end user experience all the way, ‘extreme ownership’ as my friend called it!  It is not just in the act of assessment/evaluation, it is across the entire of lifecycle of dev/test of smartly using the process, technology and tools.Technology, business, process models are evolving rapidly and disrupting the way we do things and testing too is disrupted.To survive and accomplish more in these interesting times, ‘Test smartly. Accomplish more’.

In a discussion with Sudhir Patnaik,  I asked him as what the key disruptors  to our discipline of testing are. His single phrase answer was “Extreme Ownership”. That it is, not just owning the act of validation, but the ownership of entire customer experience.And this he said, changes the team structure , the way we perform test, and the  skills required.

Watch the full video of my discussion with Sudhir Patnaik on the theme “The changing facet of our discipline/industry”.

The ownership mindset requires a two faced tester, one adept with technology and development on one side and the other to test well. Code when required, dig deeper to understand better, shift left tests, automate as much as possible and test intelligently. 

The confluence of extreme speed that business demands today, the rapid adoption of new technologies, the cloud-ification and platform-ising of applications and the  finicky end users have created serious inflexion points. It is threatening the test/QA practice, what do we do now?  The demands that we have on how we need to complement dev, expand our base of skills, deal with far more imprecise information and embrace technology to get work done implies a far greater reliance on our intellect. To survive this QA disruption, ‘Smart testing’ is very necessary. So, what we do ‘smartly’ across the lifecycle of dev/test?

Smart understanding
Being able to appreciate the intended customer experience implies a smarter approach to understanding the larger context, figuring out what is expected rather that ‘what is there’ to be validated. It is about figuring out how the sum total of various entities of a system contribute  to the wholesome  end user experience, beyond the typical piece meal validation of an entity. ‘Smart Understanding’ is an essential ingredient here. Understanding that is multidimensional, of business, end users, environment, context of usage, architecture and deployment, the technology stack and its nuances. 

Smart strategy/plan
How much do we test? How much do we test earlier (Shift Left)? How much can be done  statically using software tools on SmartChecklists? How much testing can be avoided? How and what do we need to do to maximally automate? How to set up a continuous health check? How much can we test under the hood? What are the ‘-ilities’ that we need to test? What is the impact of changes done? Coming up with answers to these at start and continuously refining them enables us to setup a continuous test flow that is rapid, optimal and effective.It is the sensitivity and refinement of what, how much, how early, how to test minimally, employing maximal automation. It is making appropriate choices to deliver a great customer experience continually, that demands ’Smart strategy and planning’, so that we may be continuous, responsive and effective. 

Smart Design and Execution
How do you ensure that you have “potent” test scenarios/cases? That is, test scenarios/cases that are powerful enough to explore all nooks and crannies of the software to uncover issues. It is not just optimising execution via automation but using intellect and technology to ensure the net is cast wide enough and this can be done quick enough. Never underestimate that all scenarios can be designed a-priori;  it is about being sensitive to the context and execution, and refine continuously. ‘Smart design’ is about being logical and use design techniques to model better, whilst at the same time be highly observant , curious and creative to come with really interesting and meaningful scenarios.

As much as good design matters, it only translates into great reality when its executed, statically or dynamically. Well as much as it sounds great to use this earlier via shift-left, it would be smarter to be sensitive and prevent. Static execution does make great sense and these  can be done using software tools or thinking tools expressed as a ‘Smart Checklist’.In the case of dynamic execution to evaluate software, we have various choices : (a) under the hood automation via API (b) Front based automation of short feature thread or long business flows (c) execution by an intelligent human. Making appropriate choices using available information at the earliest and refining would be ‘Smart execution’.

Smart Documentation
Why do we document? For posterity, for future support, right? Nah, that would be old school thinking! In today’s world, documentation is to enhance one’s understanding, with the side effect of making these understanding,  choices and actions available to others. Note that first reason is ‘improving one’s understanding’, to think better. So ‘Smart documentation’ is about being to-the-point (terse), staying uncluttered i.e crisp, visual as applicable with meaningful notes, and appropriate elaboration that is strictly on need basis. Given that testing today is done in short sessions, the act of documentation should never disrupt the flow of thinking, in fact it should catalyse smart thinking.

Smart Automation
Smartness is not only in doing the work oneself, it is smartly exploiting tools and technology to get this done. Automation is not limited to execution of tests to assess correctness, it is also about doing all other work in the larger context of assessment.

This could encompass test design, environment setup, data generation, problem analysis, runtime measurement, health checks, project management and deployment. In the context of assessment, ‘Smart automation’ can be (a) enabling self-test integrated into the code (b) health check suites that monitor health when change is done (c)under the hood automation using API  that is more resilient (d) front end based user oriented suites (e) building frameworks that can assist the dev folks earlier to test easier (f) scripts that be easily generated and modified rapidly i.e. ‘code-less automation’ especially for UI.

Smart Analysis
When we test, we may find issues or we may not. The key is, what we do with this information.How does this help in getting a clear picture where we are and how good the system is ?  How does this help make meaningful business decisions rapidly? How does this allow us to rapidly refine what we are doing?  It is about sifting thorough the rich testing dataset and just showing the minimal crisp information to enable rapid decision making. This is what ‘Smart analysis’ is about. Quickly dig in deep and extract nuggets.

Smart testing is the way 
Smart testing is about doing less and accomplishing more, and rapidly with razor sharp focus on business outcomes and end user experience.  It is about doing what it takes to ensure great end user experience all the way, ‘extreme ownership’ as my friend called it!  It is not just in the act of assessment/evaluation, it is across the entire of lifecycle of dev/test of smartly using the process, technology and tools.Technology, business, process models are evolving rapidly and disrupting the way we do things and testing too is disrupted.To survive and accomplish more in these interesting times, ‘Test smartly. Accomplish more’.

The theme of SmartQA is to explore various dimensions of smartness to leapfrog into the new age of software development, to accomplish more with less by exploiting our intellect along with technology.  Towards this, we will strive to showcase interesting thoughts, expert industry views through high quality content as articles, posters, videos, surveys outlined as a SmartQA Digest weekly emailer. SmartBites is soundbites from smart people”. Ideas, thoughts and views to inspire you to think differently.