In the current world of rapid development, software is constantly updated with new features, incremental additions and bug fixes. While features (new & incremental) are the focus for revenue generation and market expansion, bug fixes are necessary to ensure that customers stay.
“As product grows, so does regression,
increasing costs & threatening to slow down releases”
So what can be done to solve this challenge? Automate, right? But can we automate all the scenarios right from the early dev tests all the way to end-to-end flow tests? A significant of the latter is pretty difficult, right? And these do consume time and effort to build. Not that all the earlier tests will be completely automated either. Also automation is a catch up game with the software updated continually.
So what else can we do smartly ? Well in addition to whatever we can do via automation to relegate the effort to a machine we can look at two ways to lessen regression (1) Do less (2) Don’t do !
What would be a good way to analyse the impact of change? i.e What other entities may be affected and what aspects of the entities may be impacted? The technique “Fault Propagation Analysis” of HBT (Hypothesis Based Testing) can be very handy.
The first aid uses this technique and is illustrated in this picture:
Given that one of the most preferred approach to efficient regression is automation, how do you ensure the automated scenarios can be kept in sync with the constant updates in the software? Well the logical answer is short and less complex scripts.
So how can we achieve this? By ‘level-ising’ scenarios that implies scenarios are decomposed into multiple scenarios each focused on uncovering a specific type of issue of a specific entity. What we want to do is prevent ‘spaghetti’ scenarios that attempts to uncover multiple types of defects over various entity.
For example a scenario that attempts to validate inputs, validate the UI aspects of an input in terms of say colour or format, validate logical behavior correctness, and compatibility on different devices is a sure short recipe for disaster. Automating this scenario would result in a complex script making this fragile and potentially maintenance heavy.
The second aid titled “Automation fitness analyser” illustrates how to analyse the fitness of scenario(s) for automation so that we can re-test efficiently. Remember this is about ‘Do-less’.
Moving on to the third part of tackling change smartly, the objective is “what scenarios can we avoid executing” especially those that are not yet automated.
This is illustrated in the third aid “Yield Analyser” illustrated below. The gist is to analyse test cases that have continually passed over the last few cycles and use this information to judge what parts of the system may have ‘hardened’ and therefore not execute these scenarios.
In summary it is about “How can we do less to do more”?
Do less regression or Don’t regress and Do less automation maintenance is what “Smart Regression Testing ” is all about. The three aids outlined enable you to smartly accomplish this.
If all regression test scenarios have been automated, then it is real smart use of technology!