The antidote to big data
Make better decisions with data lakes and modern data architecture
Gandhi Swaminathan | July 8, 2015
Today’s businesses face a major challenge when it comes to making critical decisions using actionable data: lack of data integration across line of business (LOB) applications. Many of these businesses are limited by their operational infrastructures and forced to use intuition to relate metrics across a host of data islands.
So what happens when there's a lack of data integration in your business?
First and foremost, you won’t have a 360-degree view of customers to build a 1:1 journey, which is critical in today’s era of personalization and digital marketing. Second, you’ll see limited or no opportunity to discover the unknown customer segments and unknown market prospects. But most significantly, you will be bound to rely on intuitive methods to find causation from correlation in a never-ending quest of realizing ROI from your business initiatives.
As a decision maker, you can’t afford to lose potential and existing customers nor miss the opportunity to discover unknown business potential. All these factors greatly influence business value and your bottom line.
The antidote to this challenge is modern data architecture, empowered by the commoditization of data, algorithms, and cloud infrastructures.
Modern data architecture offers you an opportunity to seamlessly integrate and uncover unknowns that will shape the future of your business. Data lakes enable such integration, facilitating deeper analytics and the ability to store “all the data all the time”; address business needs; and enable more rapid, agile decision making with commodity hardware.
“Allowing people to do more with data faster and driving business results: that’s the ultimate payback from investing in a data lake to complement your enterprise data warehouse.”
Data lakes in practice
Last fall, I was working on a project with a large, multinational software company. The company evolved one of its products to a cloud offering. Within a couple of years of the offering's release, the executives at the company observed revenue stagnation despite acquiring new customers. They turned to data to validate their observation and understand the cause.
To understand this scenario, we needed:
- Customer data (when did you acquire, where did the lead come from, special offers et al.)
- Financial data (subscription, regularity of pay, frequency of pay)
- Service usage data (number of users, frequency of usage)
- Feature usage data (what features are being used and in what frequency)
- Support tickets’ data (what are the issues, how long did it take to resolve the issue, satisfaction of resolution)
Each of these data points resided in its own LOB application and there was no simple way to integrate. For example, the customers were identified in the support system and the CRM system using different keys, stopping the company from building a 360-degree view of its customers.
In addition to these challenges, the service and feature usage information had to be derived from ~15 terabytes of semi-structured machine data generated by operational infrastructure. A traditional data warehouse was challenged to manage the volume and veracity of data in this scenario.
Modern data architecture, using a data lake approach, significantly addressed data integration challenges. Gartner Research Director Nick Heudecker explains, “The idea is simple: instead of placing data in a purpose-built data store, you move it into a data lake in its original format. This eliminates the upfront costs of data ingestion, like transformation. Once data is placed into the lake, it’s available for analysis by everyone in the organization.”
A data lake helped my client answer critical business questions with agility, revealing never before seen insights about service and feature usage. The company confirmed its observations and analytics from integrated data, and was able to understand the causation of revenue stagnation and influence the right decisions to correct it.
As a result of the initiative, the company reduced its mean time to resolution (MTTR) by 80%; helped customers better capacity plan so they “buy what they need”; and provided transparent metrics on service and feature usage, improving customer loyalty and reducing churn.
Integrated data is a strategic asset that can be hugely influential for your bottom line and the future of your business. Wide-spread adoption of a data lakes methodology for integration is influencing industry leaders such as Microsoft to facilitate easy implementation of modern data architecture. Integrating your data assets using modern data architecture will empower the decision-making capabilities to pave a path of unseen analytical insights, enhancing your business value.
Gandhi Swaminathan is a solution principal leading Seattle’s Advanced Analytics practice at Slalom. He specializes in marketing analytics, customer analytics, and modern data platforms to build rich analytics capabilities.