SmartQA Community

Lean thinking and agility

In this SmartBits video “Lean Thinking & Agility” Tathagat Varma outlines how Agile inspired by lean thinking is accomplishing agility in SW development. 

The transcript of the complete video is outlined below.
I would agree that a lot of foundations of Agile have come from the lean world, starting from the very famous Harvard Business article in 1986-” the new product development game where we learnt a lot about how the companies were applying a very different way of doing things and basically  that started the whole thinking . But let me also just make a very sutle distinction- if we take a manufacturing metaphor, by definition it means. I have a design in mind, I have a car that is there, the design has already been done . And now what am I doing? What is the next step in that? It is a production process. I am going to make 10,000 cars or a million cars. 

If we go back into the history of lean, how Toyota started the Toyota production system. Toyota realized  that Japan is a small country, unlike in the US where people had a buying habit of going to the dealership, there are 500 cars available , they could  just pick up whatever they want and drive out with that. They realized that in Japan it is not going to be possible because people do not have that kind of money. Nobody has that kind of a real estate where they can put that and as a young economy post-second World War, they didn’t have enough resources to essentially make so many cars and put them up in the dealership.

Toyota then came up with this whole idea that they will show a catalog to the people and when they show the catalog, people pay the money, cash down and then they will start making it.The reason was the constraints that were there which Toyota and Japan together inherited in the post second world war economy. What they were trying to do was they were trying to make it very inventory efficient, very real estate efficient, very time efficient so that   their own production process could take care of it. And when you are doing the same production process run for ten thousand times or a million times every microsecond saved basically in the long run saves a lot. At a very high level the lean philosophy is all about reducing the waste from a process which you are repeating “n” number of times. 

Now let us come back to the software. It has two components, one is the whole design component, the second is the whole production component. What do I mean by that? When I talk about design, I am taking a problem space and I am saying how do I really solve it? What is the right way of doing it? How will the user respond to that? There is a lot of experimentation back and forth. 

Software development is a discovery process where the outcome of that is really better insights about how humans are going to solve a given problem.  Once I have found the right way and I am actually creating a software program to do that, then let us say I am deploying that over net today, we are deploying a lot of our applications on the web or on the Internet. You are basically scaling it across thousands of servers and you have to have a very systematic way in which in a very error-prone manner you are pushing the code into production and making sure that it behaves the same way whether it is latency, whether it is error or whether it is usability 

Obviously a lot of principles that we learn from reducing the waste in variability in the performance when it comes to the production, it’s a very natural fit. I do not want my latency to be even half a second people may not agree in today’s world. People are getting used to literally, 50 millisecond or maybe microseconds.  Obviously you need to remove the variability in the process. 

if I am doing a push into production and every time the build breaks something is going wrong there. I should be able to remove all those vagaries of my process. Lean has a very strong fit into that part of the whole thing ,where it will allow me to remove the waste in the system and the waste could be in terms of failed builds. It could be in terms of bugs that I am finding. If I can create a better sandbox where I can test it before the production, then I am reducing the waste there. I am also reducing the waste in terms of features that nobody is going to use.

We have seen in the software industry time and again that probably one third of the features get used the most and 2/3 of the features don’t get used or never  get used. There are  enough studies that have shown that. One of the lean principles is also in terms of reducing the number of features that are not being used, because that is an overproduction in some sense and it is not just writing code. It is writing code, then it is doing the testing, it is putting the resources to available.  If I can bring a lot of lean principles, I can reduce it and really focus on what is the right value for the customer.

That is to me the whole production part of it where the lean thinking fits extremely well, now when it comes to the design part of the whole thing, where I am starting with a very fuzzy idea there and I am saying, how am I going to even solve it let alone the problem of how I am going to deploy it in the field. I do not even know in what shape or form it is going to look like . The lift and shift of the lean thinking does not make complete sense there because I am not repeating that operation n number of times. It’s very easy for me at the end of the process to say, we should have removed this waste but when I am starting the process from left to right, I don’t know if this is going to lead me to a waste or not. 

In a Six Sigma kind of a parlance if I have to walk from right to left, I can always knock off the unnecessary processes, but if I am in a discovery process, if I am in a creative process, I cannot say that  let us not do it because there is no guarantee, we are  solving it for the first time. The lean principles per-se are not a direct lift and shift. However, a lean mindset is an important part of the whole thing. What does it mean to me?

What it means to me is that when I am solving the problem as we saw during the dot com days, we have heard of horror stories of how people would build a startup and for one year, two years, they are in stealth mode and they would then say, this is the next big shiny thing and then people realize that this is a big mismatch between what the people are waiting for.

The whole thinking of Eric Ries where he has married the ideas of lean thinking into a start-up kind of context, that is where we are bringing the lean mindset into that where we are saying, we do not know what is going to work for us. So why don’t we really create a closed loop management system where we take a tiny bit of a problem and what is that problem? It is a very risky hypothesis. If we go down that path, maybe there is a 90% chance that it will fail but we do not know which 90% will fail.  Now what is the best you can do with that kind of scenario, you try to take baby steps. You reduce the problem into a very small thing, you test it as soon as you can with a hypothesis and then once you have a very confirmatory tests available, then you basically tweak on that, either you pivot there or you build on top of it. 

That thinking, the whole lean mindset is very important in the software design and construction which is essentially a design and discovery problem, but when it comes to the production a lot of these can help in terms of removing the wastage from the system, standardize it like coding guidelines. We have been using the coding guidelines for last 40 years or 50 years probably, which is a very standard thing in a 5S kind of a context if I see lean because we are also trying to make a standard way of doing it.  We should really be able to separate that out and say that these are lean thinking or lean mindset and these are lean practices which really help us in that.

7 Pictures to ‘Doing SmartQA’

T Ashok @ash_thiru on Twitter


In article are seven interesting thoughts each outlined as a picture on what it takes to do SmartQA. These are on Smart thinking, Smart Understanding, Smart Design, Smart Plan, SmartQA Test Organisation and Smart Planning.

SmartQA Thinking

Yesterday a good friend of mine told me about his recent conversation with a senior engineering manager in a product dev org. The Sr Engr Manager, a great believer in code coverage told him that he just focuses on covering close to 100% code as the only measure to ensure quality, and as a means to implement shift-left. Absolutely right, isn’t it?  After all, ensuring all code written is at least executed once & validated is logical and necessary.

What is/are the fallacy in this? (1) Well you have assumed that all code needed has been written (2) Well, non-functional aspects of code cannot be completely validated (3)Well, it assuming that this is what users really wanted, even if code is working flawlessly (4) Well, the number of paths to cover at the highest level of user oriented validation is just to many to cover, next to impossible! Code coverage is a necessary condition but simply not sufficient.

Doing SmartQA requires multidimensional thinking, of looking of the system from various angles both internal in terms of code, architecture and technology and external in terms of behaviour, end users, environment & usage and then making appropriate choices of what to validate later or earlier and what to prevent or detect statically.

Smart Understanding

Prevention occurs due to good understanding. Detection occurs due to good understanding. Understanding of what is needed, what is stated and what is implemented.

Doing SmartQA is about great mental clarity of visualising what is intended, what is present, what-may-be-missing that could-be added to enhance the experience. The intent to seek this clarity is what one drives to question well, build better, prevent and detect issues.

The act of testing is really discovering what-should-be-there but-not there, what-is-there but not correct, what-should-be-there but should-not-be-there. Finally it is about understanding the impact of something that been changed, be it in the system or outside the system.

Smart Understanding is about scouring the ‘landscape’  to understand overall context and the static structure of how it is built and then ‘deep-diving’ to understand  the intended dynamic behavior. Landscape and Deep dive are great mental tools to explore the system rapidly to do SmartQA. The associated picture illustrates these two thinking tools well.

Smart Design I

Doing SmartQA is not just evaluation. It is about enabling code that is being built to be robust. To resist errors creeping in, to code-in firewalls. To ensure that I have all I need in good condition before I consume it. This implies that data (inputs) process are clean, the environment I operate in is clean and the resources I need are indeed available, and the dependencies that I have on others are indeed working well.

All I do is to protect myself. How can I handle when irritants are hurled at me? Well I have three choices :

(a) reject them and not do what I am supposed to do (b) flag them (log) and not do what I am supposed to do (c) intelligently scale down and do lesser.

The key focus is be robust, to be disaffected by inputs, configuration/settings, resources or dependent code. The act of designing for robustness makes one sensitive to potential issues that may arise and ensuring we are edged into a corner.

SmartQA Design II

Let us talk a bit of test design now. We focus a lot of execution, and therefore the ability to cover more. The focus has veered to how frequently we are able to execute the tests and therefore on automation. Let’s step back and ask to what the objective was, it was to primarily deliver clean code. So a deeper sensitivity to the quality of tests. This is where design becomes important.

So what is a smart way to design to come with good scenarios, ideally few that can uncover issues that matter most. Smart Design is about looking at the system from multiple views to ensure:

“I want | expect | would-like behaviours to satisfy needs that are implemented well and comprehensively covered to help me do well on my environments with no side effects”.

Then decide what you want to prevent, statically detect or test, be it via human or a machine. Focus on intent and then the activity.

SmartQA Test Organisation

Once upon a time software engineers developed code and also tested them. Then dedicated QA teams became up the norm of day and testing was ‘owned’ by these teams. In current times with rapid dev driven by Agile, it is kinda merging back into dev, with dedicated QA becoming thinner.  A recent article in SD times talked out “who owns QA” – Is it dev org or a dedicated org? So what is the right fit?

What ‘dedicated QA’ really means is – focus on QA/testing is indeed there. In a software org we need specialists, be it analysts, architects, developers, support and testing too. Dedicated QA does not mean reporting only into dedicated QA leadership position, it just means that we have QA specialists who have deep knowledge to systems validation.

So what may be a right fit for building a Smart QA organisation? Think of QA as a mixture of Dev QA, System QA and Solution QA with a different objective of validation.

In addition to validation, a dedicated QA (System/Solution QA) is suited well to provide enablement & governance.  This implies test infrastructure setup, tooling frameworks, setting up process, metrics, publishing aids and doing reviews and improving the system.

Smart Validation

The approach to validation of software has typically been ‘manual’ or ‘automated’ Nah, that is really not the right phrase, it is really ‘human-powered’ and ‘machine-assisted’.  So when we decide to validate in a non-automated manner, what could be smart ways of ‘doing’ ?

Well, there are FOUR approaches :
A: Completely scripted, where we understand completely, then design and then evaluate. The typical way we do, when specifications are reasonably clear and complete, mostly employing the logical left brain to the hilt.
B: Completed unscripted, the typical approach to test when faced with severe time crunch or by a casual person wherein one jumps into evaluation largely guided by one’s creativity, luck and experience.
C: Evolving script, an exploratory approach where understanding, design and execution happen concurrently using a good mix of left and right brain.
D: Evolving script done in short sessions, where we setup up a clear objective of what we want to accomplish/do and then perform ‘only understanding’, or ‘understand & design’ or ‘understand, design & evaluate’ and only largely ‘evaluate’ using a good mix of left and right brain.

What approach you chose depends upon the context. You may veer towards (A) when specs seem clear, you may end up using C or D when system is evolving are when specs are at a higher level. whilst B is useful to complement A,C,D as quick checks or exploit the power of random.

Oh, “Immersive session testing” builds upon this and attempts to enable one to ‘immerse’ in the act of evaluation, to get into a state of ‘flow’. It is a session-based approach that has a suite of ‘thinking tools’ to understand, design and evaluate with the prime objective to ‘doing less and accomplishing more’.

Smart Plan

Smart plan is simple & objective based – what entity to test, test for what issue(by conducting what test), test in what environment, test by whom (in case of a team), test from whose point of view ad finally how to test (human or automated). A concise plan that is a collection of these, enables sharp focus enabling rapid evaluation, be it human powered or building nimble automated suite(s).

A higher level of what kind of issues to be uncovered at different levels of entities on specific environment(s) would be a strategy, whilst specific issues on specific entities would form a plan. Oh this form of thinking facilitates two kinds of scenarios to be designed: (1) an ‘objectiveScenario’ from an ‘output focus’, that are simple one-liner(s) that outline what issues to uncover for what-entity on which environment (2)an executableScenario from ‘input focus’, that is a set of inputs to stimulate with, for an entity to uncover specific issues for a given environment.