Guiding leaders to greatness

Data strategy & BIInnovationMachine learning

Innovating With Data in a Rapidly Changing World

Businesses are too focused on their desired data end-state being advanced analytics and machine learning. With increasingly rapid change and radically fluctuating consumer behaviours, historical data is a poor guide to the future. Partnering to establish data-sharing ecosystems is the future of assessing innovation success.

01 June 2021 • 5 min read

Photograph: Hossein Beygi/Unsplash

“It’s incredibly frustrating – we know all the use cases, we have the data, but we just can’t figure out how to get value from them.” This statement may surprise some readers; weren’t companies supposed to be valued by their information portfolios by 2021? Since Gartner’s prediction in 2017, I suspect most leaders with significant data repositories available have expressed similar feelings to the exasperated Chief Technology Officer who recently said this to me.

I believe popular analogies between data and various scarce, valuable commodities have resulted in executives using a narrow frame of reference when pursuing data-driven innovation. Leaders have far greater opportunities to deliver more sustainable, and increasingly valuable, products and services if they share rather than hoard data across their value chains.

The new oil

One of the most popular commodity analogies is that ‘data is the new oil’. Consequently, most leaders conceive of data initiatives as using a finite, extractable resource that provides fuel for generating business value through its exploitation.

With the enormous valuations of the Big Tech companies in the US (the likes of Facebook, Amazon, Apple, Netflix, Alphabet and Microsoft), ambitious management teams have no shortage of inspiration. Moreover, these companies provide access to the same tools they have developed, ranging from open-source tools and libraries to commercial, enterprise-grade platforms and infrastructure.

Popular analogies between data and various scarce, valuable commodities have resulted in executives using a narrow frame of reference when pursuing data-driven innovation.

However, replicating their success has not been straightforward. In the same year as their information portfolio prediction, Gartner research indicated that  85% of big data projects fail, and companies have not had any more success becoming data-driven in the intervening period.

The ‘data as oil’ analogy has helped leaders understand that one of their key challenges is that data needs to be ‘refined’ to be useful. Consequently, there is widespread awareness of the practical challenges around this ‘refining’ process, including a lack of talent and data culture. The comparison between oil’s environmental impact and ethical and societal concerns about the use of data has also provided a helpful framework for public conversation.

Data isn’t a store of value

Nevertheless, there are deep-rooted issues with the analogy. While well-constructed arguments outline how data is a non-rivalrous, non-fungible experience good, there is a consistent assumption that data acts as a store of value and utility, indeed even more so than oil or similar commodities: “Data is essentially infinitely durable and reusable, whereas oil is a finite resource. Data being compared to oil implies that data has no value after use and decreases in utility like in the case of oil.”

For most businesses, this isn’t the case. And it is based on the mistake of conceptualising data as a commodity rather than information. Instead, let’s consider an older concept of data as forming a map of a specific territory. Or, to put it another way, the territory is the reality on the ground (pun intended), while the map is a less detailed abstraction that enables you to comprehend the entire territory. As a practical example:

  • The territory is your customers and their actions to purchase goods or services;

  • The map is the abstracted data about your customers and their transactions (e.g. products viewed, the price paid and demographics) from which you draw insights.

The typical lesson is that ‘the map is not the territory’. As an abstraction, our map is necessarily flawed:

  • Data is always a reduction of the actual thing, and not all critical information is captured;

  • The data may be incorrect without us realising it;

  • We may be misinterpreting the data.

By necessity, we must use maps, but we should keep in mind George Box’s aphorism that “All models are wrong, but some are useful”. However, our challenge is not only that our maps are wrong, but that the territory itself is also rapidly changing.

Unpredictable territorial change

The Coronavirus pandemic accelerated existing trends by a decade: “At the beginning of 2020, approximately 16% of retail was transacted via digital channels. Eight weeks after the pandemic reached the US, that number leapt to 27%… and it’s not going back. We registered a decade of e-commerce growth in eight weeks.”

Fundamentally, all insight-generating models, from basic statistics to highly complex machine learning algorithms, use historical data to provide observations or predictions we can apply to the present or future. Therefore, as the territory changes, your map will increasingly lead you astray, regardless of its initial quality. For example, in a recent Bank of England survey, over a third of UK banks reported a negative impact on machine learning model performance due to Covid.

Leaders should be preparing for a future where rapid, unpredictable changes in the territories they operate are the rule, not the exception.

Ironically, in the same survey, the risk appetite for machine learning and data science projects also experienced an equivalent positive impact. In doing so, respondents may be revealing their complacency that the disruption of Covid is a one-off. Instead, climate and other environmental changes are poised to have a far more disruptive, and devastating effect on our societies and day-to-day lives. Reading the recent Intergovernmental Panel on Climate Change (IPCC) Report, leaders should be preparing for a future where rapid, unpredictable changes in the territories they operate are the rule, not the exception.

Therefore, rather than working with an infinitely reusable, durable store of value, leaders should approach data initiatives with the mindset of improving and maintaining a rapidly fraying, ageing map.

Partnering to disrupt

What does this mean for the frustrated CTO I quoted earlier who was struggling to obtain business value from their data, despite knowing all the use cases?

Fundamentally, their challenge is one of scale; customers use Google and Facebook’s ever-expanding services every day, linked by a single unique ID. Interactions with most other companies (even the best at collecting and managing data) are decidedly more limited in both their frequency and volume of data captured.

To compensate for their sub-scale ‘maps’, leaders should not jealously guard their own crumbling fragments but work with partners to accelerate the updating and development of shared maps that drive innovation across their value chains. There are two linked opportunities for leaders to consider, which I have illustrated with examples from the automotive industry:

1) Partner to maximise the breadth, and value, of existing data

The CTO’s challenge is the data they possess is not relevant for their use cases or out of date by the time they can use it. There will be other organisations within the same value chain that suffer the same challenge. For example, within the automotive industry, tremendous opportunities are beginning to be explored by OEMs, retailers, finance companies etc., as they combine their data to deliver value for customers throughout a fragmented, rapidly changing, end-to-end lifecycle.

2) Partner to remake the territory and pioneer data collection

Rather than wait to be disrupted by the climate emergency, Leaders should take advantage of the $12 trillion sustainability opportunity. As Gen Z enters the workforce and business world with a world-view focused on sustainability, the likelihood of disruptive new business models to address climate change will multiply.

For example, despite evidence that Gen Z is the most environmentally conscious, a recent NTT DATA survey of UK car buyers revealed that only 35% of 18 to 25-year-olds intend for their next vehicle to be electric or hybrid, considerably less than those over 55 years old. Rather than being counter-intuitive, this highlights a model of private car ownership that is ripe for disruption. OEMs and other market players need to ask how to enable young buyers to access more expensive electric vehicles, and give those who have rejected car ownership a reason to reconsider their stance.

Data will be both a key enabler for these alternative business models and inform subsequent business strategies. As the territory changes, data about the performance of these new models will be a crucial input for decision making. Those who can most effectively map the emerging value chain(s) with their partners have the opportunity to obtain a significant commercial advantage. Moreover, those who can effortlessly, and safely, share trusted data will emerge as the partners of choice for organisations exploring this new territory.

Data strategy & BIInnovationMachine learning

Discover more in

Data strategy & BI