More Details On The 8 Productive Tensions Of Innovation - 4. Accept Data Or Ignore It

Q - Do you follow what data is telling you or ignore it?

This looks at how data is used in decision-making. In recent times there has been explosion in data use as shown by the likes of Facebook, Amazon, Netflix, Google, etc., ie all are data-driven. Data comes from digital platforms, wireless sensors, apps, mobile phones, etc.; computer is becoming increasingly faster and more powerful; it is getting cheaper to capture, store and process data.

Need to be careful how you use data in decision-making  As data is historical and in the past, it is not necessarily an accurate prediction of the future in an uncertain world, ie what captivates customers now may not continue to do so; data is fallible; sometimes you can mis-use data to oversimplify the situation; many people consider data as objective and trustworthy, yet it can be a blocker as pathbreaking innovation is inherently contrarian - its evaluation requires nuance and interpretation, not simply deferring to data.
"...data just tells you what happened in the past. It doesn't tell you anything that will happen in the future..."
Ted Sarandos as quoted by Christopher Bingham et al, 2022a

Another limitation of being data-driven is becoming selective in what you use, ie to support your point of view. An example of this is only collecting, analysing and using data collected from existing customers.

Data forecasting is more accurate in stable environments that unpredictable ones.

Need to be like scientists, ie develop new paths by exploring promising ideas, usually unsupported by data, that can lead to radical changes in their fields of knowledge.
"...the pursuit of breakthroughs requires an approach to data that is fundamentally more sceptical than deferential..."
Christopher Bingham et al, 2022a

Need to be wary of cognitive bias, ie look for data that supports your existing prejudice and point of view.

However, need to realise that the appropriate use of selective data can encourage innovations, ie
"...a more nuanced approach than simply deferring to data..."
Christopher Bingham et al, 2022a

Data and its interpretation can have inherent limitations, eg it can limit creative thinking as the data may not support it. Good examples of 'defying' the data are:

- Albert Einstein and his thought experiments, eg light affects gravity (initially there was no data to support the statement; it took decades for experiments to provide the data to prove this)

- Galileo and his work, eg helicopters (his drawings of helicopters was a fairly accurate for what was developed centuries later; however, he had no data to support his designs).

Einstein not only re-defined physics - he developed an entirely new way of thinking about the universe; he overthrew the very foundations of the process of scientific thought.

This process can be applied to business. For example, when Steve Jobs introduced the Macintosh it was based on his theory of technology, not data, ie
"...In the early 1980s, there was no data suggesting overwhelming unmet demand for cute desktop computers..."
Christopher Bingham et al, 2022a

Similarly with the development of the iPad, ie
"...a product that people couldn't yet imagine but would wanted once they saw it..."
Christopher Bingham et al, 2022a

At the same time, do not underestimate the power of data analytics and empirical observations. Both have the power to transform organisations, industries, communities, society, etc. They are most useful and powerful in stable environments. However they less useful when talking about new-to-the-world innovations and/or fundamentally pathbreaking or disruptive ideas in environments that are uneven, erratic, disorganised, uncertain, unpredictable, etc.

It is about
"...looking beyond 'what is' to focus on 'what is possible'......We can ignore much of what is observable and define something fundamental that works better......breakthroughs in business rarely fit the available data..."
Christopher Bingham et al, 2022a

Need to actively engage with data, but don't cede decision-making to data only.

Need to use quantitative data analysis like cost benefit, etc  in conjunction with qualitative analysis, etc eg projections, intuition, passion, logic, insights, creative thinking, etc.

The below diagram shows how to treat data, ie when to ignore or accept it.

Diagram - accept v. ignore (When should we defer to the data and when should we ignore data?)

(source: Christopher Bingham et al, 2022)

"...we live in an age of data analytics. But stunning advances typically begin with a big beautiful theory or transformative paradigm for which supportive data does not yet exist. In the meantime, leaders should create virtual barriers within the organisation between the data used (or ignored) in the innovation process and the data used in everyday operation..."
Christopher Bingham et al, 2022a

Being data-driven can result in doing the same thing continually.

If a product and/or service is new, there would be little or no available information about potential customers, its value chain, etc; thus any market research would be of limited benefit.

Furthermore, there is a need for a healthy scepticism about the data and insights derived from it when looking at potentially disruptive and new-to-the-world innovations. This can be achieved by creating
"...a 'data barrier' between the innovation process and the data used in everyday business......The reasoning behind a data barrier: supplying data during the innovation process would tend to promote conformist idea generation and lower the chances of creating new-to-the-world ideas..."
Christopher Bingham et al, 2022a

This is similar to the 'ethical barrier' to handle conflicts of interest within an organisation, eg for journalists and business managers in the media industry.
Furthermore, appoint a 'data heretic' whose job is to be the devil's advocate around use of data; they need to be familiar with the data to understand its shortcomings and limitations.

Usually, new-to-the-world ideas, etc will initially be less profitable than mainstream business operations; when this happens, there is managerial pressure to remove resources away from new-to-the-world ideas and back to mainstream operational areas; this needs to be resisted to encourage innovation.

Both quantitative and qualitative analysis are legitimate; it is a combination of art and science, ie
"...marshalling multiple perspectives improves the accuracy and thoroughness of the information used in decision-making about innovation..."
Christopher Bingham et al, 2022a

Engaging with data is consequential
"...it constrains or enables different types of innovation. Relying on extensive data analytics will help propel incremental innovation but will also tend to discourage pathbreaking innovation......(the latter) requires an approach that embraces the wisdom of ignoring, instead of deferring to, data..."
Christopher Bingham et al, 2022a

Summary

How active engagement with data enables breakthrough innovation