Marginal gains and the Secrets of high performance

“Unilever had a problem. They were manufacturing washing powder at their factory near Liverpool in the north-west of England in the same usual way – indeed, the way washing powder is still made  today. Boiling hot chemicals are forced through a nozzle at super high levels of pressure and speed out of the other side ; as the pressure drops they disperse into vapour and powder. The problem for Unilever was that the nozzles didn’t wrk smoothly, they kept clogging up.

Unilever gave the problem to its crack team of mathematicians, they delved deeper into problems of phase transition, derived complex equations and after a long time came up with a new design. But it was inefficient. Almost in desperation Unilever turned to its biologists, who had no clue of phase transition or fluid dynamics! Well they solved it!

The biologists took ten copies of nozzle, applied small changes in each and subjected them to failure by testing them. After 449 failures they succeeded.”

From Black Box Thinking –Marginal gains and the Secrets of high performance

Progress had been delivered not through a beautifully constructed masterplan (there was no plan!) but by rapid interaction with the world. A single outstanding nozzle was discovered as a consequence of testing and discarding 449 failures. 

It is not coincidental that biologists chose this strategy – Evolution is a process that relies on a ‘failure test’ called natural selection. 

The strategy is a mix of top-down reasoning and fusing of knowledge they already have with the knowledge that can be gained by revealing the inevitable flaws.

— 

A brilliant chapter titled “The nozzle paradox” from the book “Black box thinking – Marginal gains and the Secrets of high Performance” by Matthew Syed.

This book is a compelling read on innovation and high performance across many industries: sports, healthcare and aviation amongst others, all approached from an unusual starting point – failure.

“Learning from failure has the status of a cliché. But it turns out that, for reasons both prosaic and profound, a failure to learn from mistakes has been one of the single greatest obstacles to human progress. Healthcare is just one strand in a long, rich story of evasion. Confronting this could not only transform healthcare, but business, sports, politics and much else besides. A progressive attitude to failure turns out to be a cornerstone of success for any institution.”

From the extremely moving first chapter to the very end, Matthew Syed tells the inside story of how success really happens and how we cannot grow unless we learn from our mistakes. (From https://www.hpt5.co.uk/blog/2018/11/21/book-review-black-box-thinking-marginal-gains-and-the-secrets-of-high-performance)

---

T Ashok (@ash_thiru)

Summary

“A typical accident takes seven consecutive errors” states Malcolm Gladwell, this notion is reflected in Mark Buchanan’s book “Ubiquity” too. This article dwells upon ‘How do you ensure that potential critical failures lurking in systems that have matured can still be uncovered?’

—

“A typical accident takes seven consecutive errors” quoted Malcolm Gladwell in his book “The Outliers”. As always Malcolm’s books are a fascinating read. In the chapter on “The theory of plane crashes”, he analyses the airplane disasters and states it is a series of small errors that results in a catastrophe. ” Plane crashes are much more likely to be a result of an accumulation of minor difficulties and seemingly trivial malfunctions” says Gladwell. The other example he quotes is the famous accident – “Three Mile Island” (nuclear station disaster in 1979). 

It came near meltdown, the result of seven consecutive errors – (1) blockage in a giant water filter causes (2)moisture to leak into plant’s air system (3) inadvertently trips two valves (4) shuts down flow of cold water into generator (5) backup system’s cooling valves are closed – a human mistake (6) indicator in the control room showing that they are closed is blocked by a repair tag (7) another backup system, a relief valve was not working.

This notion is reflected in the book “Ubiquity” by Mark Buchanan too. He states that systems have a natural tendency to organise themselves into what is called the “critical state” in what Buchanan states as the “knife-edge of stability”. When the system reaches the “critical state”, all it takes is a small nudge to create a catastrophe.

Now as a person interested in breaking software and uncovering defects, I am curious to understand how I can test better. How do you ensure that potential critical failures lurking in systems that have matured can still be uncovered?

Let us look at what we do- We stimulate the system with inputs (correct & erroneous) so that we can irritate latent faults so that they may propagate resulting in failure. When the system is “young”, the test & test cases we come up are focused on uncovering specific (singular) faults. i.e a set of inputs that can irritate singular faults and yield possibly critical failures. This is possible because the “young system” is not yet resilient and therefore even a singular fault bumps it up! We then think that our test cases (i.e. combinations of inputs) are powerful/effective. But these test cases do not yield defects later as the system becomes resilient to singular faults. 

As the system matures we need to sharpen the test cases to irritate a set of potential faults that can create a domino effect to yield critical failures. Creating test cases to uncover singular faults in a mature system may not useful. It is necessary that test cases be at a higher level of system validation (i.e have long flows) and have the power to irritate a set of faults. 

Should we resort to uncovering critical failures only via testing? By creating test cases at higher levels that have the power to uncover multiple types of faults? Not necessarily. We can apply this thought process at the earlier stages of design/code too. Using the notion of sequence of potential errors and understanding what can happen.

If your drive in India you know what I mean … the potential accident due to a dog chasing a cow, which is charging into the guy driving the motorbike, who is talking on the cell phone, driving on the wrong side of road, encounters a “speed bump” , and screech *@^%… You avoid him if you are a defensive driver. Alas we do not always apply the same defensive logic to other disciplines like software engineering commonly enough…

---

Raja Nagendra Kumar in discussion with T Ashok (@ash_thiru on Twitter)

Summary

Raja Nagendra Kumar outlines the role of refactoring, unit testing in producing clean code. He states this very interestingly as “Technical debt is fat, clean code is liposuction” and crisply explains the act of producing clean code.

The video of this “nano learning” smartbits video is available here.

---

Question: In our conversations you said  “Technical Debt is Like Fat, Clean Code is Liposuction ”.  A very interesting phrase it is, could you please elaborate?

This phrase has come out with the intention of producing world-class products from India, where every developer as he gets married to the profession, his duty is to produce products of world-class.

In that context, every day whatever code he writes, he is trying to produce a product baby and as more and more code gets added like the way the baby learns each day.  The large code also will start accumulating a lot of fat. If the future of the baby has to be better, you should know how to grow the baby or constrain the baby in a way and that’s where the clean code practices starts coming. 

As more and more large code starts coming into the product, on one side  product growth happens and on the other side, the clean code  practices should know which is relevant now, then start cutting the fat. The beauty of this approach is that it is not like  doctor operating on somebody, here the professional himself is doing to yesterday’s code whatever he has written is able to remove that code which is not relevant or modify the code to make it scale better. 

So, when anything is growing there is also a fat and that fat has to be transformed in a way where it is taking advantage of that and is leaping forward much faster. Otherwise, then the two options are : living with fat and not being able to run or becoming better professional by identifying the fat on time and trying to run faster. 

What do mean by clean code?

When we are trying to achieve something there are a lot of abstractions coming on the way, which actually no code is communicating to you. For example, you want to write a feature X and you have done it in X way and feature Y comes in, now to position this feature “Y, you need to have a creative thinking rather than fitting that along with X.

Now the product will have both X and Y. Instead of fitting them you need to make sure  what is affecting Y to be there in concurrence with X, the   engineer must listen to these clashes and see how to refactor it so that Y can go smoother. Unless we try to be a listener to our own challenges, what code is speaking, clean code will not come.

Most of the time people try to fit x y z as silos, they may work independently, but not coherently. My meaning of clean code is “ listen to the last code while adding new code” and that’s where people will start talking about what is the need or purpose of refactoring, what the purpose of unit testing is. They’ are not compliance. They are the ways for you to bring out a world-class product from your own daily experiences. 

click to video

---