By Davide Russo - February 8, 2021
Original Article: https://www.datacenterdynamics.com/en/opinions/cost-decision-latency/
Tools alone do not make a data-driven organization. What separates those who aspire to be data-driven from those who actually are comes down to how quickly decision-makers can get the answers they need to make informed decisions.
Today, there’s been an explosion of innovation focused on reducing data latency and giving companies access to information in near real time. But despite advances in the availability, access, and analysis of data, enabling analysts to generate insights faster than ever, companies still struggle to be truly data-driven in their decision making.
The true measure of a data-driven organization is in its decision latency. With more data available than ever before, and more ways to understand and analyze it quickly, the bottleneck is no longer with the infrastructure — decision latency is buried within the business process itself.
Decision latency is the amount of time it takes for a team to make a decision in response to a business change. For even the most sophisticated teams, the time between noticing a troubling change, formulating questions, reaching an answer, and taking action can be days, and in some cases weeks. Analysts and business leads must find and process the relevant information, generate numerous queries to get to an answer, and present the insights in a clear, actionable format.
Decision latency should be a charter metric that leaders track. The companies that thrive in today’s economy, especially in periods of extended uncertainty, use data as a competitive advantage to adapt rapidly to changing conditions and continuously optimize their business. Unfortunately, with few exceptions, decision latency often increases as companies grow. The longer it takes to get the answers and make truly data-driven decisions, the more missed opportunities can pile up. This latency can be the cause of increased risk in the business, with critical errors in product, targeting, or engagement going unnoticed under a pile of unanswered questions and complex data— resulting in lost revenue, disappointed customers, missed first-mover advantages, and more.
Most damaging to a data-driven culture, a delay between business requests and actionable insights forces business teams to continue as usual, making decisions based on best guesses and instincts, and undermining the goal of building a truly data-driven culture.
In theory, the growing complexity, dimensionality, and speed of data available to businesses today should make decision-making easier. But these major advances in data engineering, storage, and compute are both the poison and the cure for decision latency.
With more data available than ever before, data teams find themselves facing a new problem: knowing where to look in the data and where to focus their attention. While much has been invested in building faster databases and infrastructure to solve this dilemma, an incrementally faster database will not meaningfully reduce decision latency.
The problem is, the rate at which an analyst can point and click through today’s analytics tools is a snail’s pace compared to how quickly new data is generated. Analytics tools are overly complex, requiring decision makers to rely on analysts and data scientists to deliver answers and context from their data. And even with new tools designed to enable self-service analytics, it’s still on the data organization to prepare and maintain the data, which often requires custom efforts for each new metric or project.