( Zulfikar Deen  on “Expectations of owners and users”. The video is at the end of this blog)

Let’s start from an end user. For end-users the solution makes their job easier, more productive and efficient. Any solution which gets delivered to them, they expect them to work hundred percent, there is no full thought process of DevOps and my side of the view here. I can’t have half-baked or DevOps. I got something working for you , let me know how it works, then I will build it better for you, these half -baked  doesn’t work for me. When the solution goes to my end user, it has to be working hundred percent. If there are 10 features that have been planned and if three of them are working, then all the three features must work hundred percent. That’s an expectation.

Secondly, the end users are expecting  and treating the whole solution as a black box, they don’t differentiate whether it’s an application, it’s a front end, it is a network or  it is a backup. If it doesn’t work, it’s of no use. We need to ensure that the whole solution looks at everything and one can’t have a siloed view that it’s my solution.  For example if we are rolling out a partner solution to our end user or to an internal built solution, if it doesn’t work then even if the reason is not related to the application the users would still say this application did not work.  It’s the responsibility of the whole system to ensure that this whole thing is tested properly and taken a holistic view. 

As an IT organizationor or  as a CIO organization when the  partners are dealing with us, they cannot treat IT organization as their customer because we are not their customers. We and the partner together are servicing our business so they need to co-ordinate and work very well with us to meet the expectations of my business users. Often they make a mistake treating IT organization as a customer. We are not their customers. We are  working on behalf of them taking their solution to the end users. Hence, partners need to support us and empathize with our team.

From the business any system that  is rolled out, it has to have value in some form. Whether its improvement in clinical quality or reduction in my operation cost or make my people more productive, There has to be some associated business benefit. 

The problem before and after is , before we take up solution to business, I should be able to prove or I should be able to at least articulate along with the partners how this will help the business in those parameters whether it is clinical quality or compliance.If  it is compliance, It is critical.GST it has to be there, there’s no other choice. It could be compliance or clinical quality or operation efficiency or revenue.I need to be able to prove it is helpful to the business and the system should have the mechanism to show matrix around this.Many times, that’s a key piece we miss in many software.The matrix of the system, whether it is in terms of adoption, in terms of usage, in terms of usability in terms of business metrics. All the matrix should be bid to the system. 

When putting the system in place, when my business asks the questions such as how are we doing? what is our ROI in the last one year? Here, I shouldn’t be scrambling that I got the system in place and people are using but what’s happening is not known . If I can’t figure out or produce some reports to figure out how the system is being used. It’s of no use.  Such aspects need to be baked in the system. What’s my adoption level? What’s my business metrics? they need to be part of the system. So for the end users it is about the solution. For the business users it’s about the adoption and the business metrics out of it.

( In this SmartBits,  Srinivasan Desikan outlines “The evolution of test“.  The video is at the end of this blog)

Earlier days we used to have three different teams – configuration management team to build the product, development that develops and hacks in software and test team that tests the system. The configuration management team will make the build, do whatever checks that are necessary, build it and then give only the binary version to the testing team. This was called as the build ball, they roll with it, then test it and report problems.

Earlier days 30% of the problems were mainly because of the configuration management itself. It was not the developers who were introducing the defects. It was the configuration management team, which was introducing defects by wrong builds. The build was not validated properly, build validation evolution started from the testing team. Configuration management team asked the test team to test whether the build was correct or wrong? The testing team gave them a suite which was called a sanity suite or build verification test and the build team used it, validated the build and gave it to the testing team which resulted in quality improvement.

Whenever there are boundaries there will be more problems lurking around the boundaries. Always the problems occur because of the intersection between the development and configuration management, configuration management and testing and also the boundary between development and testing.

These problems brought a thought process about a new evolution which is called DevOps. Which means starting from the development all the way to the operations, everything should be automated so that there are no boundaries left. If one has to automate something it has to start with test automation, again the evolution started with the test engineers. The build got automated scripts, the binary ball they got automated, the testing that ensures the build is done properly was automated and the functional testing as well as the build of the developers also got automated. Good amount of code that is coming nowadays are auto-created code. Here they check the return values and add more exceptions code to it. Eventually that also got automated.

We are entering the DevOps model where good amount of that is automated and people are really needed only when it breaks and only when more value needs to be added to the automation. We do not deliver any more to code or we do not deliver any more to build. We don’t deliver any more to testing but we really deliver to the purpose of the whole thing, which is the end result. That’s where DevOps is taking us to.

( In this SmartBits,  Arun Krishnan outlines “Challenges in testing big-data applications“.  The video is at the end of this blog)

“I understand that big data is characterized by three V’s volume, velocity and variety with the data formats classified into different categories of structured, semi-structured data and unstructured data, and these are acquired from a variety of sources. What are some of the challenges or issues that these pose to validation?”  Dr Krishnan’s answer to this question is below.

The Three V’s Volume, Velocity and Variety actually depends on who you are. There are 4V’s and 5V’s as well. Some of the definitions around Big Data are  volume, variety, velocity but in a true sense all these are relative. 

There are people who say 1TB of data and above is Big Data. The best definition as of today is one bit more data than your system can handle. If there is a system with 8 GB of memory and there are 8 GB and 1 bit of data if this can’t be loaded into memory that is the Big data and then it needs to be  broken into chunks.

When we talk about Big Data, we need to understand that it is relative. What Big Data is to a retail chain where there is a point of sale data coming in every second or every minute need not be the same for a company which tests the software where the focus is on looking at test results coming in every few minutes or every hour.

HR data from a retail chain perspective is not big data, but from an HR perspective, yes they do, they have a variety of  data sets coming in and they got to pull it all together The trick is in bringing all the data together and then get deeper into it. It’s not about the data quantity. 

Data are in different forms like structured, semi-structured or unstructured. How we tie it all together and how we gain insights from them, is analytics. Another example is one of my students had been to an internship at an Indian public sector unit, and there he was asked which are the best colleges to hire from. This is a huge amount of data that one could gather. This student did something really simple and straightforward. He took the average scores for every College on performance and he took the average scores on that amount of time that college folks have spent in that organization. Plot the data with  Y-axis as performance and X-axis the amount of time spent. Then arbitrarily, take two values one parallel to the x-axis one parallel to the y-axis. It suddenly has four quadrants and interestingly enough all the IIT’s came in the bottom quadrant, which is low retention and low performance. Very simple things can be done in analytics but the idea here is tying these two pieces of data together.

Even for testing it is important to figure out how we can tie real-time data coming in from devices, what we are getting in from server logs, as well as what Developers might be putting as comment,  and then use that to infer what would be the issue and then build the test cases.

( In this SmartBits,  Arun Krishnan outlines “What is big data/analytics?“.  The video is at the end of this blog)

Analytics is a buzzword these days. But quite honestly,  we have been doing analytics for a long time. If you think about it, the animals flight or fight response is analytics. It uses its past experience, like if the grass in Savannah is moving, is it because of the lion or wind?

The reason it has become so buzzwordy is primarily because of digitisation and the explosion in terms of compute power that we now have at our fingertips, and also the ease with which the data gets processed. Twenty years ago grid computing concept was quite prevalent which then slowly morphed into Big Data and now the buzzword is AI. Everybody wants to use that term and so it is essentially a way to look at patterns within data and take meaningful decisions. This is how analytics can be looked at in the current scenario.

We all think and assume that businesses have the best systems, but that’s not the truth. Even the biggest organizations are still very spreadsheet oriented, though there may be specific departments like Sales or Finance that make use of analytics to a larger extent. Again specific Industries like Banking and Finance sector have been using mathematical models and tools to forecast.

There is a lot of talk but it is more than what’s on the ground. Personally I am a big believer that the more we crunch data, the more we can identify patterns and the better outcomes we get, and some companies do it really well.  Classic examples would be Amazon, Google or Facebook. They are good at it. But there are industry segments where there is a lot of scope to use analytics for betterment.

( In this SmartBits, Sriramadesikan Santhanam outlines “QA skills for digital world“.  The video is at the end of this blog)

Skills and competencies are most important. One needs to understand that it is not just the business or it’s not just the technology, it is both. One needs to acquire multiple skills. One needs to understand what the customer is going to use this product for, how is it getting implemented, the expectations of the various end users’, and their perspectives. 

Acquire those skills and competency to understand these. That’s what I advise – try to understand what customers are looking for, what their customers are seeking, and then finally what the end users are looking, for then only one will be able to test from their shoes.

We as QA people need to understand the customer’s program objectives also, that is most important. Only then will be able to align to their expectations and understand why they are making these changes and appreciate the impact/uplift they’re looking out from modern technologies.

The change in skill requirements are: In addition to knowing about technology, to be able test to meet customer program objectives. Many times in the product world, people wouldn’t have seen how people are really using it, are we aligned to them? Testers now have to have good tech/dev skills, and also have business analysis skills too.

( In this SmartBits, Sriramadesikan Santhanam outlines “Enterprise Customers & Quality“.  The video is at the end of this blog)

There is a change in the perspective of quality, measurements/KPIs for QA teams that deliver productised solutions to large enterprise customers. When we develop a product all the KPI’s around the product features are based on the features committed, in terms of code coverage, test coverage, defect density etc.

In a typical product life cycle that is the kind of the KPI we should look at, but when we implement it as a solution, then it is customized to customer’s specific business requirements. So the KPI’s need to be aligned with customer specific expectations. Then when we put it up and roll it out to their end customers, it is a program for them.

How the whole integration is really working, the end-to-end business flows, how the customers look at it from user experience, operational efficiency and ease of use should be the basis of the KPIs now. So definitely it changes from KPIs from program to project to the product. When we start with the product and well begin with the program, KPIs should definitely changes, being in line with customer’s expectations.

( In this SmartBits, Sriramadesikan Santhanam outlines “Enterprise Customers & Quality“.  The video is at the end of this blog)

Expectations are getting changed due to the disruptive nature of technology. Most of the people in some domains are on legacy platforms. Moving onto a digital platform definitely changes the role of QA, this needs to be planned. The technology change cannot be done abruptly because this is costing them. Whenever we plan to make any changes to our customers expectations, it has to be aligned to their business programs.

The most important aspect is how successfully QA will allow running the programs for them. Does it have less risks associated when they move from legacy to new modern platforms? 

We are talking about business issues occurring on the field. The most important is the field quality. How much defects are we able to reduce in the field quality when we make changes to them?

(In this SmartBits,  Jawahar Sabapathy outlines “Should I know the architecture to test?“.  The video is at the end of this blog)

I think it is a given. You cannot validate if you do not how it is constructed and what it is doing. One has to validate the normal functionality, in the case of vehicle, when the road is good and when it is real bumpy and bad. Hence I need to know under what operating conditions that the vehicle can actually be performing correctly and how it is built. For example if the vehicle has tubeless tyres, we may have to test it differently.

The same is applicable to software construction, so when we deliver on a high availability or scaling, the mechanism that is used must be known so that we can actually simulate those minimum- maximum outlier conditions and see that it is working in those conditions or not. At the least we should know and document it. So a deeper understanding of architecture in deployment is probably more necessary for today’s validation teams.

(In this SmartBits, Sudhir Patnaik outlines “Advice for career growth“.  The video is at the end of this blog)

Have a learning mindset. Technology is evolving very rapidly and the general purpose software engineering skill is on its way to becoming extinct. It is best to develop specialized skills or niche skills either in the area of test automation or in the non-functional areas or what we call it as “-ilities” testing- availability, security, scalability performance, vulnerability.  It is good to build specialized skills as opposed to staying on general purpose software engineering skill. In addition one can always become a domain expert. But the domain expertise also is transitioning to other people who are better at performing in those kinds of areas, which are needed for the software.

It is very essential to stay or at least learn technology. Those days are gone where one needs to know only what the product does. More or less, it was QA which was known as a workflow validation expert till today. As more and more logic is getting embedded into the middleware, one needs more code expertise to be able to test the software effectively. In the early 2000, we were probably talking about black & white box testing. The industry has transformed itself from black to white box today. But those who were able to catch up on white-box testing in the early 2000s must be reaping the benefits in today’s world because that is much needed today.

(In this SmartBits, Sudhir Patnaik outlines “Role of human intellect in QA“.  The video is at the end of this blog)

The way artificial intelligence and machine learning is coming into play, there will be a time where a developer doesn’t need to write code. It would be more like telling the chatbot, the business logic and it spits out the code. In fact, to some extent it is happening today, where the template code is ready and one needs to just plug in the business code.

Even in the testing world someday automation is going to be so intelligent where one just point at the software to the URL and it captures everything. There is no need for human intervention and it self-corrects whenever the things change in the website or URL.
Sometimes when we refer to test automation, we unfortunately focus only on UI automation, whereas a big piece of the world actually is platform automation because companies are transforming themselves to be known as platform organizations in the world. There is more technology on the platform side.

The human interface on the UI side will minimize itself and this is where the transformation of the shift is focusing more on the backend side. The front end is going to be driven through intelligent systems mostly. Our intervention as quality engineers on front-end side will get minimized over a period of time.

It is not easy for any computer system or whatever system one builds to understand the piece of code and magically write a unit test code. One can understand the code, can write the computer systems, can write positive and negative test cases, but that may be about 50% of the coverage; the remaining 50% of the coverage on the code still comes from humans. That is where the intellect is still needed. No chatbots can take over.