Corners are interesting as they are subtle, invisible really. They could be complex with many things that intersect and therefore display an unique behaviour. They may not necessarily be symmetrical at ends, nor be similar to behaviour in the middle.
As a developer focused on solving a problem for typical or generic cases one may not see the interesting extremities. For example we do everything right for a system in normal state, but miss out what happens when it is brought up the first time. This article outlines eight heuristics I discovered when testing a product that we were building, a SaaS platform. The heuristics outlined are based these aspects : Time, Lifecycle, Transformation, Position,Space, Size, Linkages and Limit.
#1 Heuristic based on TIME
The first time when something is used. Thinking from the perspective of time. Doing something on an absolutely fresh system. Creating the first project, registering the first user.Doing the final action, removing an user. First time when a transaction is done on a empty system. Purging a system of content, signifying removal, the end.
#2 Heuristic based on LIFECYCLE
Repetition of system states in terms of cycling through. Starting off, then doing activities and coming to an end. Then restarting and continuing. A workflow that is half done, suspended and then continued to finish. Finish by abandoning it or ending with to a logical closure.
#3 Heuristic based on TRANSFORMATION
Changing something like say formats, views an act of transformation. In the case of UI, this could be relate to responsiveness like reaching the extremities of views? In case of content transformation, reaching the extremities of too large or too small or null content being transformed.
#4 Heuristic based on POSITION
Looking for interesting behavior in case of the elements that are right at start or end. What happens when elements in the middle move to either ends?
#5 Heuristic based on SPACE
The nation of space as when contents close are far, shrunk or expanded, especially when at extremities of too close or distant, too small or large. An example of responsive UI, when screen is shrunk or expanded.
#6 Heuristic based on SIZE
The notion of volume, size when some is really small or extraordinary huge. Say uploading super large files, or really nothing or rally small ones. In case of display, showing tiny/large content/diagram, maybe via zoom?
#7 Heuristic based on LINKAGES
Using notion of linkages like 1-1, 1-N, N-N or thinking in terms of increasing chain length like 1-1-1, N-1-N. Could linkage integrity be violated when N=0 or when chain is long? The notion of propagation especially in longer chains with differential N.
#8 Heuristic based on LIMIT
The most commonly understood that of min/max, the typical extremities of value given an definitive range.
( In this SmartBits, Arun Krishnan outlines “Challenges in testing big-data applications“. The video is at the end of this blog)
“I understand that big data is characterized by three V’s volume, velocity and variety with the data formats classified into different categories of structured, semi-structured data and unstructured data, and these are acquired from a variety of sources. What are some of the challenges or issues that these pose to validation?” Dr Krishnan’s answer to this question is below.
The Three V’s Volume, Velocity and Variety actually depends on who you are. There are 4V’s and 5V’s as well. Some of the definitions around Big Data are volume, variety, velocity but in a true sense all these are relative.
There are people who say 1TB of data and above is Big Data. The best definition as of today is one bit more data than your system can handle. If there is a system with 8 GB of memory and there are 8 GB and 1 bit of data if this can’t be loaded into memory that is the Big data and then it needs to be broken into chunks.
When we talk about Big Data, we need to understand that it is relative. What Big Data is to a retail chain where there is a point of sale data coming in every second or every minute need not be the same for a company which tests the software where the focus is on looking at test results coming in every few minutes or every hour.
HR data from a retail chain perspective is not big data, but from an HR perspective, yes they do, they have a variety of data sets coming in and they got to pull it all together The trick is in bringing all the data together and then get deeper into it. It’s not about the data quantity.
Data are in different forms like structured, semi-structured or unstructured. How we tie it all together and how we gain insights from them, is analytics. Another example is one of my students had been to an internship at an Indian public sector unit, and there he was asked which are the best colleges to hire from. This is a huge amount of data that one could gather. This student did something really simple and straightforward. He took the average scores for every College on performance and he took the average scores on that amount of time that college folks have spent in that organization. Plot the data with Y-axis as performance and X-axis the amount of time spent. Then arbitrarily, take two values one parallel to the x-axis one parallel to the y-axis. It suddenly has four quadrants and interestingly enough all the IIT’s came in the bottom quadrant, which is low retention and low performance. Very simple things can be done in analytics but the idea here is tying these two pieces of data together.
Even for testing it is important to figure out how we can tie real-time data coming in from devices, what we are getting in from server logs, as well as what Developers might be putting as comment, and then use that to infer what would be the issue and then build the test cases.
Intelligent testing is a brilliant set of opposites. Is it finding more bugs or enabling brilliant code from start? Is it about frequent and continuous evaluation or being sensitive and pre-empting issues? Is it an act of doing or a state of mind? In this article I relate intelligent testing to Yin & Yang, where the tension of opposites keeps one in the perfect state of balance enabling one to deliver the very best.
As we mature we see more opposites. Find more bugs or prevent by being sensitive? Automate more or use human smartness to do less? Continuous frequent checks or smart minimal tests? Things that seem contrary, creating tension of choice. It is really not a tussle, it is a perfect state of balance.
What is Yin & Yang? Well it is the really opposites that exist.
The FOUR main aspects of Yin & Yang are :
Let us switch to software testing now. What is Yin & Yang in the context of UNDERSTANDING a SUT?
What is Yin & Yang in the context of DESIGNING test cases?
What is Yin & Yang in the context of EVALUATING the SUT?
What is Yin & Yang in the PROCESS of evaluation ?
What is Yin & Yang in the context of the OBJECTIVE of testing?
Philosophically Zero = Infinity Nothing is everything.
The ability to be empty, to be unattached, to be in the moment, no past or future, mindful and observant enables one to deliver great outcomes.
Yin and Yang are not opposing forces, it is the tension to keep you in a perfectly relaxed state.
( In this SmartBits, Sriramadesikan Santhanam outlines “Enterprise Customers & Quality“. The video is at the end of this blog)
Expectations are getting changed due to the disruptive nature of technology. Most of the people in some domains are on legacy platforms. Moving onto a digital platform definitely changes the role of QA, this needs to be planned. The technology change cannot be done abruptly because this is costing them. Whenever we plan to make any changes to our customers expectations, it has to be aligned to their business programs.
The most important aspect is how successfully QA will
allow running the programs for them. Does it have less risks associated when
they move from legacy
to new modern platforms?
We are talking about business issues occurring on the field. The most important is the field quality. How much defects are we able to reduce in the field quality when we make changes to them?
Want to be a successful leader? Let animals be your guide. Assemble varied skills to build a great team. Let good techniques and methodical action unleash the power. Watch progress and steer continuously. Be confident. Make decisions. Enable each individual to unleash their full potential. Finally enjoy the journey.
Many years ago, we went on a holiday to Masai Mara, a lovely 1510 square km game reserve in Kenya. Here the animals are free in their natural habitat whilst humans are inside open top jeeps. Seeing the Big-5 (Lion, Leopard, Rhino, Elephant and Wildebeest) and others in close quarters was just wonderful. Observing their behaviour, mannerisms at close quarters was not only enjoyable, but educational. Here is what I learnt about leadership from these wonderful creatures.
Lions after a kill
The Lions
A lion is busy eating a fresh kill of wildebeest whilst a cackle of hyenas sit a little distance away patiently waiting for the lion to finish. Once the lion is satiated and moves away to rest, the hungry Hyenas will finish the rest. Their teeth are so powerful that they can crush the bones. At another place a wake of vultures are fighting fiercely to eat the remains of a carcass killed by a lioness. In nature, nothing is left to waste , for each animal is specialised in its contribution to the nature’s eco-system cycle. A wildebeest is a big animal and it is a shame to waste anything at all. The Hyenas don’t mess with the lion nor do the vultures with lioness or with any other animals.
Leadership lesson: It takes multiple skills to get a job done well. Each facet of work requires varied intelligence and power; understanding this and abiding your time patiently is key to being successful. So in your team, who is the Lion(ess), Hyena, Vulture…? Each one of them is important, there is nothing superior or inferior.
Leopard eating his meal
The Leopard
A Leopard had dragged up a young waterbuck up the tree, suspended it precisely on a branch with body resting on branch and legs hanging down. It then went about methodically tearing up the rear with its canines. Though gruesome and sad, it was interesting to see the leopard periodically adjusting the dead animal on the branch as it progressed methodically with its job of eating.
Leadership lesson: The Leopard is a very powerful animal, with its sheer speed and power it can snare any animal. To ultimately accomplish its goal of satisfying its hunger, it must secure the dead animal, which it does by dragging the animal up the tree and then eating it by doing it methodically, tearing from the rear. What do we learn this? In addition to having a powerful team, it is necessary to employ powerful techniques and great processes to ensure that the job is well done. Power is best extracted by using great techniques and then going about it systematically.
The majestic Giraffes
The Giraffe
A herd of giraffes are moving majestically, absolutely elegant movements as they peacefully chomp on leaves off a tree top. Giraffe by virtue of its height and its distinctive coat is indeed a majestic animal. They are not shy at all. One of them walked to our vehicle confidently, stopped, looked at us without any hint of fear and then strutted away. The movements were peaceful and seemingly unhurried. On a different note, it is interesting how a giraffe walks- Both the legs on one side go forward and then the next side go forward, very unlike the other animals where front and rear are in sync.
Leadership lesson: The majestic appearance oozing confidence, fearless attitude, grace and elegance are traits that make a leader successful. And the team will do anything to ensure that the leader stands tall always.
The nimble Zebras
The Zebras
The vast African Savannah is filled with Zebras, beautiful they are with a shiny coat and beautiful stripes of black and white. They are an easy prey for the lions. Their excellent sense of sight and hearing coupled with their acute sense of smell enables them to detect danger which they nimbly respond to, by quickly running away. Their legs are so strong that a just a kick can kill the animal.
Leadership lesson : To survive/course-correct, it requires one to continuously measure (aka smell/watch) and respond nimbly to steer to safe places. Not only is sufficient to know the danger, but it is imperative to have enough power to move away quickly.
Hippopotamus powerful and relaxed
The Hippos
Hippopotamus relaxing in the water seem to be the most lethargic animals. They seem to spend the entire day relaxing in the pool when not eating grass. But do you know that Hippos are the second largest killers of humans? They are not carnivores, they attack only to protect. Their strong bite can kill humans.
Leadership lesson : Each one of us possess enormous power.It is dormant most of the time, the key is unleashing it. A great leader should be able to spot this in others and enable them to unleash it.
The happy Warthogs
The Warthog
These animals were the most interesting. Belonging to the same family of pigs, these herbivores were constantly running, seemingly for the sheer joy of it. Enjoying themselves just running across the vast expanse. They stop suddenly only to recommence in a short while. It was sheer joy to see these “pigs with tusks” run despite no threats from other animals.
Leadership lesson: Explore. Be curious. Enjoy. To unleash your full potential, you need to be unbounded and enjoy the flow.
The Wildebeest
Masai Mara is famous for the crossing where over a million wildebeest cross over from Tanzania to Kenya and vice versa. Our trip was close to the end of ‘crossing season’. Keen to see the action up close, we went to the river Mara where the crossing occurs. And there it was, a group of wildebeests on the other side of river deciding on what to do – ‘cross or not to cross’. Upstream, nearly a kilometre ahead, a crocodile was basking on the rock while a couple of them were gently floating on water, just their tops visible on the surface, a nemesis for these crossing animals.
The wildebeests are still trying to decide, one of them pushes the other into the river, but that guy does not go any further. In fact he turns back. Then they stand still for a few minutes, one of them goes into the water and then turns back. Suddenly some of them run into the water with some running on the banks, coming to a dead stop after some time. They do nothing, each of them just stand still, waiting for the other to decide. None of them take the next step. One of them on bank is fed up and decides that he has had enough of this indecision and heads back to land. He climbs the mud embankment into the high land as he climbs, a few follow suit. One by one, others follow suit. The decision has been made, they are not going to cross now. The animals that were standstill a moment ago due to indecision are rapidly moving away following the first one to the high land.
Leadership lesson: We all suffer from indecision and do possess herd mentality. A good leader takes decision and confidently goes ahead and the team follows. Not all decisions may indeed be right, but taking a decision is indeed superior to not doing anything.
I hope you enjoyed the educational tour of the Savannah! Want to be a successful leader? Let animals be your guide.
Assemble varied skills to build a great team. Let good techniques and methodical action unleash the power. Watch progress and steer continuously. Be confident. Make decisions. Enable each individual to unleash their full potential. And enjoy the journey.
(In this SmartBits, Jawahar Sabapathy outlines “Should I know the architecture to test?“. The video is at the end of this blog)
I think it is a given. You cannot validate if you do not how it is constructed and what it is doing. One has to validate the normal functionality, in the case of vehicle, when the road is good and when it is real bumpy and bad. Hence I need to know under what operating conditions that the vehicle can actually be performing correctly and how it is built. For example if the vehicle has tubeless tyres, we may have to test it differently.
The same is applicable to software construction, so when we deliver on a high availability or scaling, the mechanism that is used must be known so that we can actually simulate those minimum- maximum outlier conditions and see that it is working in those conditions or not. At the least we should know and document it. So a deeper understanding of architecture in deployment is probably more necessary for today’s validation teams.
(In this SmartBits, Sudhir Patnaik outlines “ Transforming test teams “. The video is at the end of this blog)
As a leader running platform engineering organization, industry
changes made us reimagine how we test software platforms. We decided that in
the context of extreme ownership mindset by Scrum team it is best that we
transform quality engineering organization and merge them with Scrum teams.
This required us to look at the entire quality engineering
organization, find out people well-versed in functional, performance testing
and security testing, and automation. In a step-by-step process, we did a shift left of
quality engineering organization, re-skilling
many engineers towards learning the development technologies. We moved one
function at a time into the respective Scrum team, and gave them about a year
to transform themselves into developers. We provided every training that was
needed.At the same time, we also trained developers to embrace functional
testing concepts. If one is the owner of a service, one owns its unit testing,
code review and its functional testing too.
From Quality Engineering perspective this meant that half of
quality engineering organization ended up merging with the Scrum team and
transforming themselves into developers in a period of one year. The remaining
half of the team still stay as a central organization, but largely responsible
for end-to-end and non-functional testing.
They are not executors of end-to-end and non-functional testing scripts, they have transformed themselves to become more creators of tools, frameworks and scripts that are needed to get these done. Then who executes those – “Well it is the Scrum team”!
Good testing is a great combination of intellect, techniques, process and heuristics. What does it take to do brilliant testing ? It requires one to be immersed in the act, be focussed yet unbounded, be keenly observant but non-judgemental, with outcome that are beautiful, the activity so joyful that time stops. A state of flow. What does it take to get there?
As engineers we use good techniques, tools, process and intellect to do things well. So how can we do far better, that is brilliantly? Having possibly exhausted all things “external”, in terms of tools, techniques, process and intellect, it is time we considered the huge “internal” potential that we all possess.
It is about going beyond the intellectual mind, deeper into the sub-conscious to harness the infinite potential. A good start for this would be get into a state of ‘flow’.
So, what is FLOW? Flow is the state when you are immersed in what you are doing, where you are totally mindful. It is when all energies are harnessed fluidly to do brilliant work without tiring or trying hard, becoming an observer, fine tuning actions with extreme agility, when time seems frozen. It is when you accomplish a lot, doing work effortlessly and experiencing absolute joy.
So how can we get into the state of flow? When multiple sensory excitations converge harmoniously, there is a good chance of entering that state of flow. What does that mean? It is engaging the various senses well – colours, picture, visual test, mind maps for the eyes, using pen/pencil, paper for the touch, and maybe background music or talking to oneself for ‘engaged hearing’. Note the keyword is ‘harmonious’.
When we stimulate our creative side with interesting visuals, tactile and possibly sound, it activates us to get into an ‘engaged state, a state of mindfulness.
Exploit visual thinking by using visual text , sketch maps, mind maps to : (1) enable deeper understand the system under test (2) to sketch test strategy (3) jot down test ideas (4) note down observations, questions, suggestions (5) list scenarios (6) record issues.
Exploit the tactile sense by using pen/pencil on paper or finger/stylus on tablet instead of keyboard. The objective is to be write/draw freely rather than be constrained by the keyboard.
If you want to engage your auditory sense, quietly talking to yourself, melodious quiet whistling or if you are music person, then a suitable background song played could enable you to get into the flow.
To ensure that we can perform in the state of peak performance being in a FLOW, it is imperative that we do testing in short sessions. A session may be anywhere between 45-90 minutes. The key is to stay focussed, setup an objective at the start of session and then engage as stated earlier to get into a state of FLOW.
How does this help? When we are fully engaged, in a state of FLOW, it is no more about just left brain centred logical/deductive thinking, or the creative right. It is an expansive multi dimensional thinking brought about by the harmonious stimuli enabling one to become a keen observer and absorb deeply and rapidly. This is when we exploit the infinite power of what we possess, delving into the sub-conscious which is much larger that conscious mind. Interesting questions pop up, ideas germinate rapidly, actions are done quickly, smallest observations captured resulting in brilliant effective testing.
In closing… Test automation allows us to do more, and machine intelligence to do better. It is now necessary for us to delve deeper so that we complement the machine rather than compete, enabling us to be super efficient and effective by being smarter. Get into the state of flow to harness the power of sub-conscious to do brilliant testing.
Summary As software test practitioners we revel in finding bugs, and as managers and engineers we are focused on fixing these. Steven Johnson in his book “Where good ideas come from-The seven patterns of innovation” devotes a chapter on errors as a source of innovation. This article summarises key ideas from this outlining how erroneous hunch changes history, how contamination is useful, how being wrong forces you to explore, how paradigm shifts with anomalies and how error transform into insight.
Brilliant software testing is interesting, kinda like Star Trek, going into the unknown. It requires discipline, creativity, logic, exploration skills, note taking, observation, association, hypothesising, proving and juggling multiple hats frequently. It is not a mundane act of finding bugs and getting them fixed. It is just about scripting and running it to death. It is about exploring the unknown, driving along buggy paths, discovering new ideas, suggesting improvements, enhancing experience, embedding testability and having the joy of creating beautiful software by looking at the interesting messiness.
When I read the book “Where good ideas come from – The Seven Patterns of Innovation” by Seven Johnson, I was delighted to know that one of the patterns was “ERRORS”. An entire chapter on this was a treat to read. As a test practitioner I have always meandered along the paths of anomalies, understood implementation and intentions deeply, and have come up with interesting suggestions to improve and enhance experience. Errors have been my best partner in perfecting what I do and coming up with interesting ideas.
In this article I have summarised key facets from this chapter “ERRORS” outlining how erroneous hunch changes history, how contamination is useful, how being wrong forces you to explore, how paradigm shifts begins with anomalies in the data, how paradigm shifts with anomalies and how error transform into insight.
An erroneous hunch changed history The strange correlation between the spark gap transmitter and the gas flame burner turned out to have nothing to do with the electromagnetic spectrum. The flame was responding to the ordinary sound waves emitted by spark gap transmitter. But because De Forest had begun his erroneous notion that the gas flame was detecting electric signals, all his iterations of Audion involved some low pressure gas inside the device that severely limited its reliability. It took a decade for researchers at GE to realise that triode performed best in vacuum, hence the name vacuum tube. The vacuum tube, the precursor to the stunning electronics that would sweep the world n in the next few decades was born from an error. De Forest watching the gas flame shift from red to white when he triggered a surge of voltage through a spark gap forming a hunch that gas could be employed as a wireless detector could be more sensitive that anything than anything Marconi or Tesla had created to date. An erroneous hunch changed history.
Contamination is useful Alexander Fleming discovered the medicinal virtues of penicillin when the mould accidentally infiltrated a culture of Staphylococcus left by an open window in his lab. Antibiotic was born. A bunch of iodised silver plates left in a cabinet packed with chemicals by Louis Daguerre formed a perfect image when the fumes spilled from a jar of mercury. Photography was born. Greatbatch grabbed the wrong resistor from the box while building a oscillator and found it was pulsing a familiar rhythm. Pacemaker was born.Contamination is useful.
Being right keeps you in place. Being wrong forces you to explore. The errors of great mind exceed in number those of the less vigorous one. Error often creates a path that leads you out of comfortable assumptions. DeForest was wrong about his utility of gas as a detector, but he kept probing at the end of edges of the error, until he hit upon something that was genuinely useful. Being right keeps you in place. Being wrong forces you to explore.
Paradigm shifts with anomalies. Paradigm shifts begin with anomalies in the data, when scientists find that their predictions keep turning out wrong says Thomas Kuhn in the ‘The structure of scientific revolutions’. Joseph Priestly thought a plant would die when kept in a jar depriving it of oxygen, it turned out to be wrong, discovering plants expel oxygen as part of photosynthesis. Being wrong on its own doesn’t unlock new doors in the adjacent possible, but it does force us to look for them. Paradigm shifts with anomalies.
Transforming error into insight Anro Penzias and Robert Wilson thought noise in the cosmic radiation was due to faulty equipment until a chance conversation with a nuclear physicist planted the idea that this may not be the result of faulty equipment, but rather the still lingering of reverberation of big bang. It changed the opinion that the telescope was the problem. Coming at the problem from a different perspective with few preconceived ideas about what the correct result could be can enable one to conceptualise scenarios where the mistake might be actually useful. Transforming error into insight.
Noise free environments end up being more sterile A few decades ago Prof Charles Meneth began investigating the relationship between noise, dissent and creativity in group environments. When his subjects were exposed to inaccurate descriptions in the slides, they became more creative. Deliberating introducing noise forced the subjects to explore more in the adjacent possible enabling good ideas to emerge. Noise free environments end up being more sterile and predictable in their output. The best innovation labs are always a little contaminated.
Error is what made humans possible in the first place. Without noise, evolution would stagnate, an endless series of perfect copies, incapable of change. But because DNA is susceptible to change – where mutations in the code itself or transcription mistakes during replication – natural selection has a constant source of new possibilities to test. Most of the time, these errors lead to disastrous outcomes, or have no effect whatsoever. But every now and then, a mutation opens up a new wing of adjacent possible. From an evolutionary perspective, it’s not enough to say “to err is human”. Error is what made humans possible in the first place.
Mistakes are not the goal, they are an inevitable step in the path of innovation. When the going gets tough, life tends to gravitate towards more innovative reproductive strategies, sometimes by introducing more noise into the signal of genetic code, and sometimes by allowing genes to circulate more quickly through the population. Innovative experiments thrive on useful mistakes, and suffers when demands of quality control overwhelm them. Mistakes are not the goal, they are an inevitable step on the path of true innovation.
Truth is uniform and narrow, error is endlessly diversified. “Perhaps the history of the errors of mankind, all things considered, is more valuable and interesting than of their discoveries. Truth is uniform and narrow; it constantly exists, and does not seem to require so much an active energy, as a passive aptitude of soul in order to encounter it. But error is endlessly diversified.”
Summary Smartness is a brilliant combination of knowledge and skill. Skill is acquired by intense practice. Intense practice happens when you have good habits. In this article I look at SEVEN habits that are key to SmartQA.
Habit #1 Visualise well, see the big picture of the system. Who uses, where, when, using what interface. See the big picture of how it is structured/architected, visualise the flow of control and data and the technologies used(aka tech stack). Appreciate the environments it should run on the dependencies of external systems and changes/modifications done.
Habit #2 Question well. Question to know. Question to know what you do not know. Ask about users and their expectations ask about the how it is constructed, ask about conditions a behaviour should satisfy. Ask, ask… The limit to questioning is unbounded. Note you may not get all the answers, the key is to question.
Habit #3 Prove with facts. Show proof of adequacy of tests, hypothesise potential issues and prove that they may be detected. Understand changes done and prove why an entity(ies) need to be regressed, after all you do not want to more that needed.
Habit #4 Iterate continuously. SmartQA is about experimenting, exploration. It is about iterating with minor changes to understand better, to observe better. Chew well to digest.
Habit #5 Unlearn. Revise what you know. Understanding is not about knowing more, it is also discarding what you know. Chew to digest and spit out the rest.
Habit #6 Empathise. Put yourself in others’ shoes and feel them. Get inside them to understand what they expect, what they will dislike, their constraints, what value they expect of the system. SmartQA is not testing the system technically, it is about delivering value.
Habit #7 Simplify the problem solving. End users use systems to solve their problems. Great software solve them well. Good testing is about ensuring systems do just enough i.e. simply.
