The Organisational Void Between Data and Decisions

If we can’t tell the difference between a good decision and a lucky outcome, we can’t learn,

and we definitely shouldn’t automate.

Over the last decade, organisations have invested heavily in data platforms, analytics teams

and, more recently, AI. The intent has been consistent and well-meaning: to help leaders

make better, faster decisions.

Yet, in practice, many organisations find themselves asking an uncomfortable question -

often directed at the CDO. With more data, more dashboards and more sophisticated tooling

than ever before, decisions ought to be easier and more informed. So, why aren’t we seeing

tangible business value?

I’ve come to believe the issue isn’t technology, talent or even data quality. It’s something

more structural and harder to see. We have learned how to build data platforms. We have

not learned to capture how decisions are made.

In many organisations, data now sits in a void - positioned between technology and the

business, expected to influence outcomes but rarely empowered to shape how decisions are

formed, evaluated or learned from.

This article explores how we ended up here, why so many well-intentioned efforts stall, and

what this gap means as organisations rush to automate decisions they don’t yet fully

understand.

How we got here: when data became too technical

A useful place to start is “the quants” of the 1980s. When it was proven that purely

quantitative, algorithmic approaches could generate extraordinary returns in trading firms,

they became a secret, competitive advantage. Soon, every major investment bank and

hedge fund was building quant teams. The advantage was in the skilled people, not the

empowering technology.

Organisations sought to scale quant-style thinking outside domains where outcomes were

directly attributable. Processing billions of rows of data started to become financially

accessible, and code libraries offered tempting models that could be quickly deployed to

deliver returns across many industries. But, in scaling the quants, they misunderstood the

decisive ingredient, increasingly replacing human judgement with scaled technology.

Vendors and consultants keenly leant into this race for competitive advantage through

technology platforms and transformations. Their natural C-suite partner was the CIO or CTO

who already owned the infrastructure, ERP and other systems of record where transactions

were created and stored.

Abstracting these into a platform configured for analysis and decision support, therefore,

became a business case value assessment aligned to technology-style metrics: volume of

data processed, number of dashboards built, time saved or efficiency gains.

Nobody ever measured the quants on such metrics.

Bridging Judgement and Technology

This abstraction was essentially data warehousing, and specialist modelling techniques

made the data more usable, in granular context or for trends over time. Graphical data

visualisation tools made presenting these insights increasingly informative and engaging.

Defining suitable models and styles of presentation required specialist attention and

connection to business decision-makers.

Because the CIO rarely needs such commercial business relationships or success metrics, a

common response to handle these challenges is to implement a dedicated data function, run

by a Chief Data Officer (or similar title). With this dedicated team creating what seems like

insightful information, it feels aligned to decision-making. But this structure is now creaking.

Despite the expectation of decision support – applied judgement – the data function, as a

subset of the CIO, has often inherited technology-style success metrics.

The CDO role was introduced as a response to growing investment in data, with an explicit

expectation for delivering business value. Its P&L influence is exercised indirectly, through

platforms and pipelines, governance and literacy.

All of these are structured to provide better information for business decision-makers who

own the outcomes that generate revenue, remove cost or change operational processes and

structures.

The void is clear. The CDO is using platforms that are typically in the CIO’s budget and

building capabilities that others may use to hit their own targets. Consequently, the data

function ends up being judged on outcomes it cannot control and constrained by structures

designed for processes owned by someone else.

If the CDO is held to CIO-style metrics and never gets the chance to influence how decisions

are made, they will constantly be walking the tightrope over the void.

Decisions create, or destroy value

Data has no value until a decision is made using this superior input. Featuring in better

Business Intelligence (BI) is helpful, but merely reporting on past performance does not

meet the investment expectations of appointing a CDO.

Decisions feature applied judgement, acceptance of trade-offs, consideration of other

options, assumptions and the expected level of success against given metrics.

Unless we’re sifting intent from good fortune, we won’t learn what a good decision is.

Learning requires reasoning, not just end results – it requires us to ask why, and not just

what.

As a CDO, when those decisions go well the P&L function gets the credit. When they don’t,

the data investment can be questioned. It’s an uncomfortable, political and fragile position.

Yet the CDO is rarely positioned with the authority and accountability to challenge how

decisions are made.

It’s tempting to assume this gap exists because organisations haven’t yet built the right tools

or processes.

A more reasonable explanation could be that organisational performance systems reward

outcomes, not explanation. Senior business leaders feel they are in place because of their

acquired expertise. They are not rewarded on the “why” of a decision, only for demonstrating

the “what” based outcomes of their successful leadership. Making decision reasoning explicit

feels difficult and time-consuming when you’re operating at speed. To them, it must seem

like unnecessary bureaucracy and oversight without an obvious return.

In that context, it’s hardly surprising that judgement remains implicit. The absence of

decision reasoning isn’t an oversight; it’s a by-product of structures designed to demonstrate

and protect experienced authority, not to interrogate it.

But, if we can’t tell the difference between a good decision and a lucky outcome, we can’t

learn - and we definitely should not automate.

This problem is now manifesting through the innovations in AI and the urge to implement

automation. There’s a strange parallel to the rush to invest in data infrastructure 20 years

ago. Then, we devalued the importance of human judgement applied by the quants and

focused on technological scale. Now, we’re tasking AI with interpreting decisions we made

by looking only at gathered historical data when we haven’t ourselves considered or

captured why and how they were made. Who can tell if AI has correctly inferred what we did,

and will apply the same judgement going forward?

To use a simple marketing example: an AI trained on historical pricing decisions may not

distinguish between “we made this level of discount for this lapsed customer segment to

ensure they came back” from “we had some money available and passed it back to

customers in cheaper prices”. But the outcomes of inferring the wrong intent and automating

at scale could wipe out the promotional budget with no realisable customer impact.

Suddenly, the thinking that led to the “why we’re deciding to do this” has become as

important as the demonstrable “what happened” outcome. But it’s context we often don’t

have available.

Before your organisation automates another decision, pause and ask: can you articulate why

you made that decision the last 10 times? If not, you’re just scaling ambiguity at machine

speed.Correctly identifying the void

The uncomfortable implication is this: the CDO stepped into a void that was framed as a

data problem, when the real gap lay in how decisions are made, justified and learned from.

The role was never given authority over that terrain, only the expectation to influence it from

the side.

Now, as organisations rush to automate decisions through AI, a second void is opening with

the same characteristics: high expectation, unclear ownership, and little appetite to make

reasoning explicit.

Where the barrier to entry is human language and a prompt box, the barrier to entry for AI

tools is so much lower than the data platform, exacerbating the peril of the void for the

unwary organisation.

For CDOs, this raises a more fundamental question than tools, platforms or maturity models.

Not how to deliver more data but how, or whether, they are positioned to influence the

decisions that data is supposed to inform.