I have been a great fan of Dr Goldratt having read all this books, my favourite being his first book “The Goal”. This book “Necessary but not Sufficient” is written as a “business novel” and shows the fictional application of the Theory of Constraints to Enterprise resource planning (ERP) and operations software and organizations using that software. 

Here is an interesting comment by Alistair MacDonald (from Goodreads) “The stance of the book on the value of software is that “software is necessary but not sufficient”, Ie: software is a necessary evil. I think this is an accurate view of software: it’s valueless without the ability to reprogram humans to use it correctly. The book applies this concept to change in general; Ie: providing a systems approach to fixing a human problem is only half of the solution, you also have to change the mindset of the users so they are able to buy in to the paradigm shift that the system enforces. There is a hidden world of beauty among all of this, which is that the original meaning of software was “people to run the hardware” (prior to hardware having the ability to operate on procedural instructions from memory). So, “we need the software”, but “we can’t expect results without changing the users”.

An excerpt from Jack Vinson’s blog on this : “With regard to the story in the book, I enjoyed it for what it was.  It follows the usual path.  The vendor and implementer see they have a problem meeting their forecasts.  They come upon the idea of selling bottom line value, rather than the usual justifications that their industry offers.  And they discover just how hard it is to turn “visibility” into a number that means anything to the bottom line.  Eventually, they hit upon a way to think about their software in a new way – a way that is inspired by Theory of Constraints.”

T Ashok @ash_thiru on Twitter

Summary

Coverage, an indicator of test effectiveness is really multidimensional and has not been dealt with rigour most often(excepting for code coverage). This article outlines a “Smart coverage framework” that looks at coverage from multiple angles summarising it as a beautiful picture. 

The original article is at https://stagsoftware.com/dosmartqa. CLICK HERE to read the complete article.

Transcript of discussions with

(In this SmartBits, Srinivasan Desikan outlines “The evolution of dev”. The video is at the end of this blog)

If you look at industrial evolution, you can classify this into four dimensions. The first evolution that happened is on the infrastructure side where individual people used to have their own PC’s and they wanted to share those individual PC’s. That’s when the Test lab was created. if you really see that dimension was really initiated by test engineers, they did the consolidation of all the machines and that was called a test lab, then over a period of time, they added some compliance requirements to it became data centre when they added to the data centre, they wanted to have more operating systems than what they have, so they created virtualisation. 

The testing folks said, let us have a quick control of all these resources, whether it is virtual or whether it is real and that became data centre. Then looking at the reliability of the test lab data centre companies decided let us host our services on those data centres so that our customers can use it using the very own processes which were created by the test engineers. That became the industrialized data centre and then cloud evolved from it. 

The cloud is nothing but something which is elastic, you put a small data centre somewhere and then you keep elongating it based on the needs based on utilization. Elasticity does not mean only expanding but also whenever there is no load bring it down that is what is cloud and after Cloud now people are actually talking about, having virtual images within virtual images which are called Dockers, containers and Kubernetes that’s one side of the dimension. That is the dimension one, which was initiated by the test Engineers. 

The second dimension which obviously was not created by the testing teams, but it has evolved over a period of time. Those days we used to have desktops then it became laptops, smaller in size then it became palmtops even smaller in size. Then it became mobile and today we are actually talking about IOT devices which go and sit in each and every appliance at home that is a second dimension of the evolution. 

If you look at the third evolution of dimension- the process delivery model,  waterfall model where strict coding is completed and only then testing was done and after that people wanted to parallelize testing and development team then became V model and V model became spiral model then spiral model became agile model. Then the agile model is becoming super lean model, which is called lean development, processes and  related things that is on the process delivery model that is the third dimension to it.

The fourth dimension to the evolutions what I have seen in the last 30 years is the customer payment model. Initially when all the desktops and other things were available in the office, they were all purchased by paying full cash. Which is a hundred percent Capex and 0% Opex. The electricity was paid by the same company. The maintenance was paid by the same company. There were no operational expenses. Everything is a capital expenditure and slowly they moved into optimal capex and Opex, they wanted to outsource some of the maintenance part of those machines therein they brought in some element of operational expenses other than a capital expenditure. Then  went to another model where there is less capex but more opex they want ,started renting out the machines from outside and after that consumption-based volumetric model evolved where , you buy a product from us and depending on the number of users you have or depending upon the number of amount of utilization those users can, we will charge you that became the consumption-based volumetric model .

The last one which is becoming predominant, now is pay-per-use ,you do not invest anything that is zero cape and everything is opex.  It is going in Uber or Ola and now the insurance is paid, the car is bought by the company and you just go use it and for the amount of time or amount of a distance you have used, you pay them.  Whenever you want, you can do it you can rent out a Mercedes today, use it as it is your own vehicle but you still need not own it.

These are the four different dimensions in the industry has evolved in my opinion.The intersection of all these four is the sweet spot all the test engineer should work on. The dimension one and the fourth the infrastructure as well as the process delivery model. Both of them are invented by the key engineers. Moving to the waterfall model to V model is actually done by key engineer. Moving from the individually kept machines to a test lab was actually initiated by that testing teams Majority of the evolution what you see in the industry. The starting point is testing and the sweet spot for the test engineers is to look at all these four dimensions and how they are going. What I told you is only the 30-year story it will continue for another 30 years or another 60 years, but these dimensions will stay but the evolutions will continue to happen. The sweet spot here is which tells you how fast we should evolve, so speed is very important.

The second thing, no investment. Everything is becoming virtual, companies will become virtual, the offices will become virtual, people can work from home and people can take jobs workloads which you call it as work load from the internet and you may not have any manager. You may not have any infrastructure your capital expenditure, maybe zero sitting at home and you will be still delivering the products and most of it what you do sitting at home would be apt testing. That is  where I think the industry is evolving.

In this SmartBits video “Lean Thinking & Agility” Tathagat Varma outlines how Agile inspired by lean thinking is accomplishing agility in SW development. 

The transcript of the complete video is outlined below.
I would agree that a lot of foundations of Agile have come from the lean world, starting from the very famous Harvard Business article in 1986-” the new product development game where we learnt a lot about how the companies were applying a very different way of doing things and basically  that started the whole thinking . But let me also just make a very sutle distinction- if we take a manufacturing metaphor, by definition it means. I have a design in mind, I have a car that is there, the design has already been done . And now what am I doing? What is the next step in that? It is a production process. I am going to make 10,000 cars or a million cars. 

If we go back into the history of lean, how Toyota started the Toyota production system. Toyota realized  that Japan is a small country, unlike in the US where people had a buying habit of going to the dealership, there are 500 cars available , they could  just pick up whatever they want and drive out with that. They realized that in Japan it is not going to be possible because people do not have that kind of money. Nobody has that kind of a real estate where they can put that and as a young economy post-second World War, they didn’t have enough resources to essentially make so many cars and put them up in the dealership.

Toyota then came up with this whole idea that they will show a catalog to the people and when they show the catalog, people pay the money, cash down and then they will start making it.The reason was the constraints that were there which Toyota and Japan together inherited in the post second world war economy. What they were trying to do was they were trying to make it very inventory efficient, very real estate efficient, very time efficient so that   their own production process could take care of it. And when you are doing the same production process run for ten thousand times or a million times every microsecond saved basically in the long run saves a lot. At a very high level the lean philosophy is all about reducing the waste from a process which you are repeating “n” number of times. 

Now let us come back to the software. It has two components, one is the whole design component, the second is the whole production component. What do I mean by that? When I talk about design, I am taking a problem space and I am saying how do I really solve it? What is the right way of doing it? How will the user respond to that? There is a lot of experimentation back and forth. 

Software development is a discovery process where the outcome of that is really better insights about how humans are going to solve a given problem.  Once I have found the right way and I am actually creating a software program to do that, then let us say I am deploying that over net today, we are deploying a lot of our applications on the web or on the Internet. You are basically scaling it across thousands of servers and you have to have a very systematic way in which in a very error-prone manner you are pushing the code into production and making sure that it behaves the same way whether it is latency, whether it is error or whether it is usability 

Obviously a lot of principles that we learn from reducing the waste in variability in the performance when it comes to the production, it’s a very natural fit. I do not want my latency to be even half a second people may not agree in today’s world. People are getting used to literally, 50 millisecond or maybe microseconds.  Obviously you need to remove the variability in the process. 

if I am doing a push into production and every time the build breaks something is going wrong there. I should be able to remove all those vagaries of my process. Lean has a very strong fit into that part of the whole thing ,where it will allow me to remove the waste in the system and the waste could be in terms of failed builds. It could be in terms of bugs that I am finding. If I can create a better sandbox where I can test it before the production, then I am reducing the waste there. I am also reducing the waste in terms of features that nobody is going to use.

We have seen in the software industry time and again that probably one third of the features get used the most and 2/3 of the features don’t get used or never  get used. There are  enough studies that have shown that. One of the lean principles is also in terms of reducing the number of features that are not being used, because that is an overproduction in some sense and it is not just writing code. It is writing code, then it is doing the testing, it is putting the resources to available.  If I can bring a lot of lean principles, I can reduce it and really focus on what is the right value for the customer.

That is to me the whole production part of it where the lean thinking fits extremely well, now when it comes to the design part of the whole thing, where I am starting with a very fuzzy idea there and I am saying, how am I going to even solve it let alone the problem of how I am going to deploy it in the field. I do not even know in what shape or form it is going to look like . The lift and shift of the lean thinking does not make complete sense there because I am not repeating that operation n number of times. It’s very easy for me at the end of the process to say, we should have removed this waste but when I am starting the process from left to right, I don’t know if this is going to lead me to a waste or not. 

In a Six Sigma kind of a parlance if I have to walk from right to left, I can always knock off the unnecessary processes, but if I am in a discovery process, if I am in a creative process, I cannot say that  let us not do it because there is no guarantee, we are  solving it for the first time. The lean principles per-se are not a direct lift and shift. However, a lean mindset is an important part of the whole thing. What does it mean to me?

What it means to me is that when I am solving the problem as we saw during the dot com days, we have heard of horror stories of how people would build a startup and for one year, two years, they are in stealth mode and they would then say, this is the next big shiny thing and then people realize that this is a big mismatch between what the people are waiting for.

The whole thinking of Eric Ries where he has married the ideas of lean thinking into a start-up kind of context, that is where we are bringing the lean mindset into that where we are saying, we do not know what is going to work for us. So why don’t we really create a closed loop management system where we take a tiny bit of a problem and what is that problem? It is a very risky hypothesis. If we go down that path, maybe there is a 90% chance that it will fail but we do not know which 90% will fail.  Now what is the best you can do with that kind of scenario, you try to take baby steps. You reduce the problem into a very small thing, you test it as soon as you can with a hypothesis and then once you have a very confirmatory tests available, then you basically tweak on that, either you pivot there or you build on top of it. 

That thinking, the whole lean mindset is very important in the software design and construction which is essentially a design and discovery problem, but when it comes to the production a lot of these can help in terms of removing the wastage from the system, standardize it like coding guidelines. We have been using the coding guidelines for last 40 years or 50 years probably, which is a very standard thing in a 5S kind of a context if I see lean because we are also trying to make a standard way of doing it.  We should really be able to separate that out and say that these are lean thinking or lean mindset and these are lean practices which really help us in that.

T Ashok @ash_thiru on Twitter

Summary

In article are seven interesting thoughts each outlined as a picture on what it takes to do SmartQA. These are on Smart thinking, Smart Understanding, Smart Design, Smart Plan, SmartQA Test Organisation and Smart Planning.

SmartQA Thinking

Yesterday a good friend of mine told me about his recent conversation with a senior engineering manager in a product dev org. The Sr Engr Manager, a great believer in code coverage told him that he just focuses on covering close to 100% code as the only measure to ensure quality, and as a means to implement shift-left. Absolutely right, isn’t it?  After all, ensuring all code written is at least executed once & validated is logical and necessary.

What is/are the fallacy in this? (1) Well you have assumed that all code needed has been written (2) Well, non-functional aspects of code cannot be completely validated (3)Well, it assuming that this is what users really wanted, even if code is working flawlessly (4) Well, the number of paths to cover at the highest level of user oriented validation is just to many to cover, next to impossible! Code coverage is a necessary condition but simply not sufficient.

Doing SmartQA requires multidimensional thinking, of looking of the system from various angles both internal in terms of code, architecture and technology and external in terms of behaviour, end users, environment & usage and then making appropriate choices of what to validate later or earlier and what to prevent or detect statically.

Smart Understanding

Prevention occurs due to good understanding. Detection occurs due to good understanding. Understanding of what is needed, what is stated and what is implemented.

Doing SmartQA is about great mental clarity of visualising what is intended, what is present, what-may-be-missing that could-be added to enhance the experience. The intent to seek this clarity is what one drives to question well, build better, prevent and detect issues.

The act of testing is really discovering what-should-be-there but-not there, what-is-there but not correct, what-should-be-there but should-not-be-there. Finally it is about understanding the impact of something that been changed, be it in the system or outside the system.

Smart Understanding is about scouring the ‘landscape’  to understand overall context and the static structure of how it is built and then ‘deep-diving’ to understand  the intended dynamic behavior. Landscape and Deep dive are great mental tools to explore the system rapidly to do SmartQA. The associated picture illustrates these two thinking tools well.

Smart Design I

Doing SmartQA is not just evaluation. It is about enabling code that is being built to be robust. To resist errors creeping in, to code-in firewalls. To ensure that I have all I need in good condition before I consume it. This implies that data (inputs) process are clean, the environment I operate in is clean and the resources I need are indeed available, and the dependencies that I have on others are indeed working well.

All I do is to protect myself. How can I handle when irritants are hurled at me? Well I have three choices :

(a) reject them and not do what I am supposed to do (b) flag them (log) and not do what I am supposed to do (c) intelligently scale down and do lesser.

The key focus is be robust, to be disaffected by inputs, configuration/settings, resources or dependent code. The act of designing for robustness makes one sensitive to potential issues that may arise and ensuring we are edged into a corner.

SmartQA Design II

Let us talk a bit of test design now. We focus a lot of execution, and therefore the ability to cover more. The focus has veered to how frequently we are able to execute the tests and therefore on automation. Let’s step back and ask to what the objective was, it was to primarily deliver clean code. So a deeper sensitivity to the quality of tests. This is where design becomes important.

So what is a smart way to design to come with good scenarios, ideally few that can uncover issues that matter most. Smart Design is about looking at the system from multiple views to ensure:

“I want | expect | would-like behaviours to satisfy needs that are implemented well and comprehensively covered to help me do well on my environments with no side effects”.

Then decide what you want to prevent, statically detect or test, be it via human or a machine. Focus on intent and then the activity.

SmartQA Test Organisation

Once upon a time software engineers developed code and also tested them. Then dedicated QA teams became up the norm of day and testing was ‘owned’ by these teams. In current times with rapid dev driven by Agile, it is kinda merging back into dev, with dedicated QA becoming thinner.  A recent article in SD times talked out “who owns QA” – Is it dev org or a dedicated org? So what is the right fit?

What ‘dedicated QA’ really means is – focus on QA/testing is indeed there. In a software org we need specialists, be it analysts, architects, developers, support and testing too. Dedicated QA does not mean reporting only into dedicated QA leadership position, it just means that we have QA specialists who have deep knowledge to systems validation.

So what may be a right fit for building a Smart QA organisation? Think of QA as a mixture of Dev QA, System QA and Solution QA with a different objective of validation.

In addition to validation, a dedicated QA (System/Solution QA) is suited well to provide enablement & governance.  This implies test infrastructure setup, tooling frameworks, setting up process, metrics, publishing aids and doing reviews and improving the system.

Smart Validation

The approach to validation of software has typically been ‘manual’ or ‘automated’ Nah, that is really not the right phrase, it is really ‘human-powered’ and ‘machine-assisted’.  So when we decide to validate in a non-automated manner, what could be smart ways of ‘doing’ ?

Well, there are FOUR approaches :
A: Completely scripted, where we understand completely, then design and then evaluate. The typical way we do, when specifications are reasonably clear and complete, mostly employing the logical left brain to the hilt.
B: Completed unscripted, the typical approach to test when faced with severe time crunch or by a casual person wherein one jumps into evaluation largely guided by one’s creativity, luck and experience.
C: Evolving script, an exploratory approach where understanding, design and execution happen concurrently using a good mix of left and right brain.
D: Evolving script done in short sessions, where we setup up a clear objective of what we want to accomplish/do and then perform ‘only understanding’, or ‘understand & design’ or ‘understand, design & evaluate’ and only largely ‘evaluate’ using a good mix of left and right brain.

What approach you chose depends upon the context. You may veer towards (A) when specs seem clear, you may end up using C or D when system is evolving are when specs are at a higher level. whilst B is useful to complement A,C,D as quick checks or exploit the power of random.

Oh, “Immersive session testing” builds upon this and attempts to enable one to ‘immerse’ in the act of evaluation, to get into a state of ‘flow’. It is a session-based approach that has a suite of ‘thinking tools’ to understand, design and evaluate with the prime objective to ‘doing less and accomplishing more’.

Smart Plan

Smart plan is simple & objective based – what entity to test, test for what issue(by conducting what test), test in what environment, test by whom (in case of a team), test from whose point of view ad finally how to test (human or automated). A concise plan that is a collection of these, enables sharp focus enabling rapid evaluation, be it human powered or building nimble automated suite(s).

A higher level of what kind of issues to be uncovered at different levels of entities on specific environment(s) would be a strategy, whilst specific issues on specific entities would form a plan. Oh this form of thinking facilitates two kinds of scenarios to be designed: (1) an ‘objectiveScenario’ from an ‘output focus’, that are simple one-liner(s) that outline what issues to uncover for what-entity on which environment (2)an executableScenario from ‘input focus’, that is a set of inputs to stimulate with, for an entity to uncover specific issues for a given environment.

(In this SmartBits, Zulfikar Deen outlines “Operationalising is key “. The video is at the end of this blog)

You need to understand industry like Healthcare where the primary job is not  use a system. For example, if in Healthcare or Finance or if it is a Finance or Back office operations where you got an army of people sitting. They are  actually doing everything in the system approving doing all that stuff. In Healthcare a clinician or a doctor or a nurse their job is not to work on a system,their job is to attend to patients. Whatever system I put in here for them to use it should be  non-intrusive. It should help them but it should be non-intrusive. 

That is the first understanding , whatever system when a partner is developing, you need to understand that this is not a primary tool for them to do that job. Now what is missing piece when I work with multiple partners is, you got a piece of the solution, that is great, it works, fantastic, but that is probably in my view no matter whatever technology architecture you put into the system. It is probably 20% of the effort. 80% of the effort goes in, how do I operationalize .

What do you mean by operationalize? The same example of a doctor, you made a small change, or you delivered some small incremental benefit. How do I let the doctors know? I cannot bring the doctors to the training room. I can not get all of them in one single place, it is not possible , whether they are nurses. So the change we propagated, how do I first of all make it reach them? That is the number one most important difficulty. 

Number two is in terms of support. let us say if they have a problem, now how do the entire chain of support system works in to the place , because when a doctor sees a problem in the application, hopefully it’s not clinically problematic application, if it is a clinically problematic it is going to be very very challenging. But even otherwise if it is raised, then it goes to a local support team, then they need to understand whether it is a problem of networks or application or system or server hardware, diagnose it rightly, then take it to the team internally and understand if it was a network problem, it was a infra problem, or it was actually application problem. Even if it was an application problem, there are ten applications interacting with each other, which part of the application actually created the issue, then finally trying to figure out it may be because of this part of solution then  forward the issue to them, then they need to understand.

For them the challenges of partner is in what sitting, in what situation this real problem occurred. In any clinical situation I cannot recreate this inpatient. I cannot give the patient data to my partner. so it is challenging for them to understand that also. 

Now not only understanding, making sure that support makes the fix or they give a solution it propagates through all through the chain and goes back to that particular end user is extremely challenging. So training is a big challenge, support is a challenge and adoption is a challenge . 

You may roll out n number of systems, how do I ensure, say we have twelve thousand people using a system. How do we ensure it reaches to them? and people have the inertia, so you bring a new what takes them to change. it’s going to be very difficult until it is driven. how we ensure that is we need to drive through the line of business leadership, they need to identify the value for it, they need to tell yes it is important  so by then one small change done by a partner in an application by the time I ensure this is the way for all 12,000 people, it is an enormous task, so that’s why that needs to be understood.

I being on the other side  I would say these are fantastic too, I got a nice little capability on the record,put some button up there, I got some good old report coming up. Why can’t the users use it. This is a challenge, it is extremely difficult to take one and propagate to 12,000 and do it continuously, so that is a critical piece of operationalizing versus developing a solution, 

It may not be about the only change in the software. it is about a process change. For example,we delivered a solution where a patient can do a payment by themselves lets say for  a consultation. Now earlier my internal process are lined in such a way that patient will come they will come to the front desk and then go to the cash counter they paid everything.  They’re all trained very well as a process point of view. Now, I suddenly introduce a system. From my partner point of view it’s easy, I got a payment solution for you, just go pay for it credit card integration, it’s easy, but now how do we train all my staff to know that somebody actually already paid that means they should have the knowledge, how do we change the workflow within the hospital, don’t have to hit the front desk. you don’t have to go to the cashier counter, you can go to the nursing station. How do I change that process? When going to the nursing station, the nurse should know that earlier they will have printed sheet, that they paid for it from the cashier counter, nursing should know, now it may not be the case, they could have been paid online. So they should be able to know how to look up into system that this person has already paid. Now how do I identify this is a person, so there are quite a bit of process challenge.

T Ashok @ash_thiru on Twitter

Summary

A mashup of three articles published as part of SmartQA digest over the last few months that outlines mindsets needed to brilliant code, tips to produce clean code and habits to speed up evaluation

Here is a mashup of three articles published as part of SmartQA digest over the last few months that outlines mindsets needed to brilliant code, tips to produce clean code and habits to speed up evaluation.

Great code is not result of mere unit/dev testing at the early stage. It is really a mindset that is key to producing brilliant code. 10 things to be sensitive to deliver brilliant code outlines ten things that a developer should be very sensitive to enable the delivery of brilliant code.

10 Simple Tips to Clean Code has in detail, simple practices that can ensure that code developed is constantly cleansed. In current times where code is churned out at a rapid pace, it makes great business sense to contain the entropy continually. 

Automation is a key enabler for rapid QA, the limiting function to speed is one’s skills. And this is where good habits come in. 10 Habits to Help You Speed Up Testing outlines these in detail.