Why Is Organisational Transformation so Difficult?
Recently I was chatting with a senior leader about their frustration around leading organisational change. No matter how much conversation took place and no matter how many people outwardly agreed the change was required: Why was it so difficult to get those that agreed to execute?
I’ve written several case studies on why transformations fail citing specific examples, but they are mainly from a skills perspective: leadership, intelligence, strategic thinking and influencing. This time I’m going to discuss this social phenomenon in terms of it’s psychology from an individual to organisational perspective, adding further insight as to why 50-90% of strategic initiatives fail and 70% of digital transformations fail.
Let’s explore this now.
Certainty vs Uncertainty
The model below is based on a change curve discovered in 1983 by Doctors of Psychology James Prochaska and Carlo Di Clememti. The model shows a timeline made up of 5 distinct stages that demonstrate the process of attempting to make a change that involves giving something up without replacing it with something else, this can be anything from smoking cessation and weightloss to a career path pivot and organisational transformation. The one thing they all share is moving from a state of certainty to a state of uncertainty. In the context of this article we will focus on leading a population through a transformation, this can be anything from a digital transformation and marketplace pivot to a merger & acquisition.
Each stage of change correlates with a belief system:
Pre-Contemplation: The concept of change or the awareness of the need for change does not exist.
Contemplation: Considering the need to make a change in the near future.
Preparation: Seeking the knowledge to achieve required change.
Action: Implement new-found knowledge required to achieve change.
Maintenance: Sustained successful measurable change.
What should become abundantly clear, and something I find particularly interesting, is when the idea of a transformation is floated, by default it’s asking everyone round the table in a state of Pre-Contemplation to give something up and replace it with something that’s, at best, an abstraction, i.e. relinquish the certainty of how things work now and the sense of control and familiarity that provides with the uncertainty of how things might work, a lack of control and unfamiliarity. Or put another way it’s asking a population to severe ties with the positive psychological anchors they have in place that allows them to live their life as they know it.
The transformation evangelist is already in the preparation phase, possibly even the action phase sharing an idea with an audience in the pre-contemplation phase, at best contemplation phase. There is already a psychological gap of up to 12 months in thought processing. Right out the gate the incentives are mismatched, unless the transformation evangelist can clearly state in specific terms the pro’s vs cons for each stakeholder round the table.
In fact, those at the table might even agree with the concept that change is required, as it might be obvious the organisation is struggling financially, however those being asked to support the change haven’t had time to conceive of their role in the new world and therefore what they individually need to do differently to support the change. Ergo, they will most likely to continue as they always have, as thats the only course of action they consider available to them at this point in time.
Let me give you an example from a large enterprise transformation of how much people, regardless of their rank, can experience a need to cling to certainty.
A senior leader was being challenged to eliminate a process from their area of responsibility as it was no longer fit for purpose. The numbers it produced were neither accurate nor relevant for the new world, yet this senior leader fought tooth and nail to retain this process. Eventually we managed to get to the root cause of their behaviour. They would rather retain a process that was familiar and ineffective than deal with the discomfort and fear of failure associated with adopting a new process they didn’t yet understand, even though it demonstrably worked. What made this particular instance memorable and fascinating was the senior leader admitting they knew their course of action was counter productive, but the need for certainty was so great they felt compelled dig their heels in. With this understanding I was able to give them the tools to overcome their fears.
If you want to learn more about why transformations have such high failure rates do read: Culture: Digital Transformation & Organisational Readiness, which specifically investigates why 70% of Digital Transformation fail.
Rate of Adoption
The model below is a bell curve created by Professor Everett Rogers who discovered the rate of adoption of new technologies in any given population, which he called the Law of Diffusion of Innovation. In the context of this article it predicts what you will witness in terms of how a transformation will be received by your organisation:
We can now correlate what we’ve learned about certainty vs uncertainty with how any given population can be broken down to predict the rate of change adoption.
Innovators: Come up with the idea and present a vision based on future possibilities.
Early Adopters: Lacked the idea, but quick to seize opportunities based on future possibilities.
Early Majority: Get onboard once the benefits have been reliably tested and proven.
Late Majority: Not really interested in the new thing, get onboard to avoid being left behind.
Laggards: Grudgingly participate, opposed to change, likely resistant, possible saboteurs.
This model gives us a snapshot to reinforce just how change averse any given population really is. Reality check: Almost everyone is not ok with, or prepared for, the change. In fact, only 16% of your population will have, at best, a conceptual understanding of new world, never mind be prepared for it in practical terms. Lets imagine a transformation is announced to a population of 100. We can now predict how many will populate each category:
Innovators: 2-3 people will be the transformation evangelists.
Early Adopters: 13-14 people will actively support the idea, agree with the concept, willing to have a go.
Early Majority: 34 only buy-in to something that’s been proven, will initially resist as its still unproven.
Late Majority: 34 are motivated by fear of missing out, but only after adoption by the early majority.
Laggards: 16 don’t care, no matter what you do they cannot be won over. Sabotage likely.
84 of our imaginary 100 people will operate on a spectrum from mild resistance to active sabotage as they believe they expect to lose more than they will gain. Remember, from their perspective they are being asked to give up the tangible and reliable for the intangible and unknowable. This explains why so many people queue up to tell you why something won’t work and so few will tell you why it will work, hence progress typically seeks forgiveness rather than permission. Otherwise, nothing new would ever get done.
A great example of this is the noise around Elon Musk and his acquisition of Twitter. If you take the time to cut through the white noise, deconstruct and understand what he’s attempting to achieve, its not only impressive, but likely to succeed. To learn more do read: High Performance: Twitter & Elon Musk.
If we go back to our example. Our senior leader was outwardly very supportive of the transformation, even eager to participate. As soon as it became evident what this senior leader needed to do differently they began to question: Their role, values & principles, technical competence, ability to lead, dealing with making people redundant, the list goes on. This senior leader was suddenly immersed in so much uncertainty and discomfort they immediately tried to put the lid back on what had become Pandora’s Box. What was conceptually an exciting opportunity was now literally a frightening reality. This manifested as: I am keeping this process, no matter what, even if it doesn’t work.
Fight, Flight or Freeze
Below is my version of a learning curve demonstrated by merging the Dunning Kruger Effect with Noel Burch’s four stages of competence. In any given moment each of us is somewhere on this curve, and it changes from one moment to the next in any given endeavour.
This model shows us Knowledge correlates with competence, and competence comes in four stages:
Unconscious Incompetence: Unaware you lack a required skill, utterly clueless.
Conscious Incompetence: Aware you lack a required skill, in your face reality check.
Conscious Competence: You are learning a required skill, having a go with training wheels on.
Unconscious Competence: Unaware you have a highly developed skill, in the zone and smashing it.
We can see anxiety exists in the middle two. The closer you are to the bottom of Conscious Incompetence the greater the challenge the greater the anxiety, more commonly known as panic. And the closer you are to the top of Conscious Competence the milder the challenge, the milder the anxiety, more commonly known as excitement.
To continue with our example: The mind of our senior leader was in the process of being hijacked by the Amygdala in their Limbic Brain during their reality check. The Amygdala has no capacity for language and can only induce an emotional response. In this case: Severe anxiety. Our senior leader rapidly tipped from unconscious incompetence to conscious incompetence, manifesting as fear. This was an unconscious response to the sheer number of absent skills required to navigate themselves and those in their charge from old to new world.
This is where, I think, things get very interesting. When this stress response occurs in anyone, they default to their programming, regardless of whether it’s effective or not. During a high stress situation: If you’ve learned to lie on the floor and regress to a tantruming two year old because you are well practiced at that, which I’ve witnessed during a large enterprise transformation, thats exactly what you will default to. On the other hand, if you’ve practiced running in to a burning building to save peoples lives like a fire fighter then thats what you will default to. Others have learned to become paralysed with fear and freeze. In other words you will default to what you have trained yourself to do.
There is a superb quote that sums this situation up, which I first heard from a Navy Seal: “Under pressure you don’t rise to the occasion, you sink to the level of your training. Thats why we train so hard.” It’s an adaptation of the original quote by Archilochus, a Greek Philosopher from around 650 BC: “We don't rise to the level of our expectations, we fall to the level of our training.” This is unequivocally the case.
If we combine this model with the previous one we now know with certainty 84% of your organisation will rapidly tip from unconscious incompetence to conscious incompetence. They are being involuntarily tipped in to a situation they have not trained for.
If you want to learn more about this in the context of leadership the please do read: Leadership: The Side no-One Talks About
Organisational Anxiety
The model below is based on a matrix created by Professor of Psychology, Mihaly Csikszentmihalyi who was first to formally investigate the traits associated with high performance, which he collectively called Flow. I have modified it for a business context to correlate organisational states with culture by looking at the relationship between Challenge, defined as: Degree of difficulty to be overcome, and Support, defined as: Available resources. To determine the culture you predominantly operate in you will witness one or more of the following:
These four cultures correlate with specific behaviours that can be measured in any individual, team or organisation:
Anxiety: Missed deadlines, avoidance of interaction, likely negatively impacting employee sickness absence.
Apathy: Absence of enthusiasm or presence of indifference, likely negatively impacting employee retention.
Comfort: Lack of activity and accountability, likely negatively impacting employee productivity.
Flow: Pro-active, self-directed, emphasis on personal responsibility, likely positively impacting all of the above.
We have already learned from the previous models that anxiety directly correlates with a lack of knowledge and skill, i.e demonstrable incompetence. The example of our senior leader gives us insight on how that might manifest in an individual and we now also know that with mathematical certainty 84% of your organisation will spend the majority of the transformation in the Anxiety quadrant, causing inevitable costly delays, stall your project or cause it to fail all together.
If you want to learn more about the correlation between confidence and anxiety, and how to leverage them as a force for good then please do read: High Performance: Confidence vs Anxiety.
Conclusion
To stay relevant in the market place organisational transformations are inevitable, as are the demonstrable effects of them on your people. The reality check is understanding and accepting almost everyone will not be ok with it. So the question becomes: What does a successful transformation look like and how do you achieve it?
A starting point is coming to terms with the fact you cannot fight human evolution, but you can channel it in a meaningful direction. Developing a strategy to leverage the inevitabilities demonstrated in these models and figuring out how to use them to your advantage in the nuance of your business is key. A very small few will respond to the carrot, the rest will respond to the stick.
Those chasing the carrot will go on the journey no matter what. The stick is still about taking your people on the journey, however it’s not about making sure everyone is happy as you all make daisy chains and hold hands while skipping across the beach in the sunshine. We have demonstrated with mathematical certainty this will not happen no matter how hard you try.
The stick is about clearly laying out the facts of the transformation in the form of pro’s vs con’s, i.e. the inescapable reality of what is going to happen whether they like it or not, with each individual responsible for deciding for themselves what will or won’t work for them. Whatever they decide you support them to achieve it. For most the support needed will be in the form of Learning & Development to learn how to operate meaningfully during the transformation, then build the skills required to perform well after the transformation, this includes senior leaders. For some it will be support to exit and find a different new world. You cannot make an omelette without smashing eggs.
As an ex-Paratrooper I find it fascinating leaders in the business world never train for worst case scenario, something we accept as the norm, especially when a transformation can easily be perceived as worst case scenario for at least 84%, and the same leaders are then bewildered as to why it’s not working.
This begs the question: What impact would high stress scenario training have in terms of transformation success and failure?
If your organisation is about to embark on a transformation of some kind and most, if not all, of what I have written here is new or surprising to you then you are already on the back foot. These principles will help get you on the front foot:
Accept this is what your leadership salary is actually paying you for. Own it.
Be honest and upfront about what is happening, good or bad, you will be respected for it.
Avoid hiding behind legal jargon, otherwise you will be considered weak and untrustworthy.
Determine the scale of impact on your organisation using the Law of Diffusion of Innovation.
Speculate who might actually end up in each category.
Identify the support they need to navigate the transformation.
Accept it’s likely not everyone will want to stick around to find out if it works.
Manage your relationships with those staying and going in ways that benefit them.
If you would benefit from support to set your transformation up for success, or get an ongoing transformation back on track, ensuring you take a strategic approach, then please do schedule a call with me by putting a 60mins in my diary at a time that suits you. We can discuss your situation and options over an eCoffee.
Best Wishes
Kenny
