Optimising financial processes

Posted on:

Why So Many Data Science Projects Fail to Deliver


The business value of data is as seductive as it is compelling.

We all want to be “data driven” in our operational execution, transformation and process improvement.

But data strategies are much harder to deliver than we expect or than they should be. Some of the challenge is a cognitive bias towards the belief that “once we have the data”, everything else should be easy. A perfect example of the Dunning-Kruger effect.

More and more companies are embracing data science as a function and a capability. But most have not been able to consistently AND rapidly derive business value from their investments in big data, artificial intelligence, and machine learning.

The MIT Sloan Management review published some interesting research recently that aligned with some of our own experiences.

The authors identify 5 common challenges.

  1. The Hammer in Search of a Nail
  2. ​Unrecognized Sources of Bias
  3. Right Solution, Wrong Time
  4. Right Tool, Wrong User
  5. The Rocky Last Mile

The authors comment “It’s not exactly rocket science to observe that analytical solutions are likely to work best when they are developed and applied in a way that is sensitive to the business context“. 

This, in our experience, is ironically, the closest thing to “rocket science” of the whole endeavor. This is why we refer to “Business Science” rather than “Data Science”. 

Of all of the challenges, the lack of deep business process domain knowledge by data scientists has a nuclear impact. And this problem runs deep and affects a number of other barriers to success;

  • Simplicity in “Insight to Action” – data insights need to be clear and simple (and that in itself is HARD!) to drive effective decisions and action. It is very hard to achieve this without deep process knowledge.
  • The Need for Speed & Pareto Principle – the 80/20 rule is alive in well in business. “Time to Value” is a critical component of the value itself! 
  • Buy vs Build – Building effective data driven insights from scratch can take many months, sometimes years, for these reasons above. Where possible, consider buying (or preferably, renting) packaged data insight capabilities with embedded process knowledge and expertise> Ensure there is no consulting, customization or configuration effort required, as this defeats the “Time to Value” purpose. 

The MIT SMR article summarizing the research by Mayur Joshi, Ning Su, Robert Austin and Anand Sundaram can be found here .. 

It is a 10 minute read, perfect for the “commute” between bedroom and home office!