With the advent of quantum mechanics, the whole world of classical physics and traditional theories was rattled. It even prompted someone like Einstein to say in disbelief [1] that "God does not play dice." Classical mechanics had given rise to unprecedented applications of predictive linearity in economics and the gumption to apply it to predict human behavior using a set of psychological tests. As a result, the world order was all set to follow a set pattern of rules.

All was well until we started hearing about a certain uncertainty principle and a thought experiment involving a cat ^[2]. The two gentlemen, Heisenberg and Schrodinger, along with a host of other physicists postulated that you cannot determine the position, direction and speed of a (quantum) particle with precision and a cat locked in a radioactive box can exist in two states (dead or/and alive), until an observer opens the box and determines the state.

Schrodinger's and Heisenberg's postulates showed that there is an uncertainty that cannot always be predicted through equations and laws, unlike in classical physics.

Today, the laws of physics deal with probabilities, not certainties. Now, this seems very familiar to the business environment, where the world order surely cannot be predicted with the same old cocky certainty. Time and again, we have seen economic, political, environmental, and certainly behavioral predictions go awry. So, where does this lead to the debate of data-driven insights versus intuitive probabilities?

Increasingly, data-driven lead indicators are ushering in a new era in decision-making by pre-empting risks. In most cases, decision-makers see no harm in following these insights, except when they get uneasy vibes. And this is especially true when it comes to determining future actions of humans.

Like all living things, and unlike machines, humans are driven by one of the most enigmatic yet unpredictable of all earthly things: the mind. And it is programmed to make conscious decisions based on Intuition. However, the era of data-driven decision-making ties down a decision maker's analysis to data extraction and mining, multivariate algorithms, and actionable insights. While this approach proves valuable for machines, it often falters when the uncertain element of the human mind is involved.

Why Intuition Cannot be Overruled

My grandparents never had insurance. And today, many generations later, a significant number of people in the world still don't have insurance - not because they cannot find a suitable insurance policy or for a lack of data to prove the value of insurance, but because their intuitions prevent them from trusting the insurance model. Although data and insights have proven that insurance isn't an unnecessary expenditure, insurers continue to face challenges in removing distrust from the minds of the uninsured.

Intuition also plays a strong role in the minds of many business leaders. A case in point is Travis Kalanick, CEO of Uber, who faced criticism while introducing surge or dynamic pricing ^[3]. Data showed that customers dislike uncertainty in pricing and welcome discounts, but not price surges. But Travis took an intuitive decision to pioneer surge pricing in taxis. Today, surge pricing continues to grow revenues for Uber and is now accepted by customers worldwide. In this case, Travis’s Intuition proved predictive models can sometimes mislead you.

Last year, a leading financial firm tried to capture the outcomes of a human-driven event - the FIFA World Cup. The firm used 200,000 probability trees and one million simulations to determine that England and Belgium would fight for the 2018 World Cup. Not only did the predictions go awry ^[4], but they fared worse than the 2014 predictions.

Decisions Based on Intuition or Insights?

While these examples showcase the ups and downs of decision-making using data and Intuition, the approach to predicting the future is more complicated. If we have every piece of data before deciding, we can just use machines with an appropriate algorithm. In most circumstances, however, we need to deal with ambiguity, and that is where human judgment and experience come into play.

As we discussed earlier in the thought experiments, it’s ultimately the observer and their apparatus of measurement, which determines if the cat is dead or alive. Prior to the observation, the cat was in a superstate of being both dead and alive. We are aware of how the dynamic human mind manipulates our thoughts and actions. As a result, HR functions cannot solely rely on predictive insights to solve and counter issues related to attrition, training, engagement, and more. I recommend that we arrive at decisions by using three elements: predictive analytics, Intuition, and experience.

Decision-makers must not feel pressured by insights. They must take a step back, listen to their Intuition, and rewind their experience to ensure that those insights do not mislead them. I am certain that there will be times when Intuition does not agree with insights. In these cases, trust your gut - but do understand the risks and pitfalls of deciding based on Intuition. Speak to your team and trusted advisors and ensure that they are aware of the risks that come with intuitive decision-making. And most importantly, don’t beat yourself up if your Intuition fails you.

Or at least hope that the era of quantum supercomputing is well within our time and narrows down the probabilities by finding those missing variables that will make the world order more deterministic.

[1] https://www.stmarys.ac.uk/news/2014/09/physics-beyond-god-play-dice-einstein-mean/

[2] https://www.nationalgeographic.com/news/2013/8/130812-physics-schrodinger-erwin-google-doodle-cat-paradox-science/

[3] https://www.wired.com/2013/12/uber-surge-pricing/

[4] https://www.theguardian.com/football/2018/jul/14/investment-banks-predict-world-cup-failures

[5] https://www.cio.com/article/3221621/6-data-analytics-success-stories-an-inside-look.html

This point of view article originally published on Forbes.com. Forbes, the No. 1 business news source in the world, is among the most trusted resources for senior business executives, providing them with the real-time reporting, uncompromising commentary, concise analysis, relevant tools and community they need to succeed at work, profit from investing and have fun with the rewards of winning. Forbes reaches an audience of 931,0558 for their print edition, and 3,600,000 for their online edition.

Click here for the article.