Optimizing a data platform to grow the business
snowflake and slalomAt a glance
What we did
- Snowflake data platform optimization
- Data architecture assessment
- Roadmap development
- Scalable data platform framework
An open-ended assignment
A fast-growing, global pharmaceutical company had just completed a migration to Snowflake and wanted to optimize its use of the data platform. Its objective was to sustain growth while keeping technology costs in check, and to put the right operations and processes in place to make the technology sustainable.
The data in question was enterprise data that supports its operations in a variety of use cases. For example, the marketing team uses this data to identify healthcare organizations and providers who might prescribe their products.
It was important to the client to build a user-friendly, self-service platform where anyone in the organization could log in and understand the data quickly, without waiting for help from IT. Speed to market is a key business priority, and accessible data can give the company an edge on the competition.
The client approached us to analyze how it was operating with Snowflake and to share industry best practices from our experience. Specifically, the team wanted us to challenge them. When we put the contract in place, there was no specific scope; they wanted us to find areas where they could improve and give them the roadmap to execute on.
The client specifically wanted us to challenge them. They needed a roadmap for the next three years that would help them grow as fast as their business was growing—and keep working for them.
Weeks from start to finish
Interviews with stakeholders
Hours of interviews conducted
Years in the implementation roadmap
Under the microscope
In just three weeks, with one full-time developer and one part-time architect, we conducted an in-depth assessment of the client’s current Snowflake data platform architecture, workflows, process models, tools, and services to identify its pain points and the areas where it could optimize.
We interviewed all business units currently using Snowflake to identify how they used the data, what their challenges were, and how they supported other users.
During the assessment, we identified and documented their current state processes, interactions, gaps, and future directions. Then, we delivered:
- Future state requirements, use cases, and solution patterns. We provided industry best practices and points of view on architectures, patterns, processes, and tools to support known and unknown use cases.
- Recommendations and next steps. We provided detailed guidance on how to address knowledge, skill, and/or technical gaps between current and target states.
A clear picture of the ideal state
It’s often challenging for organizations to scale their data platforms sustainably. Our client asked us for a roadmap for the next three years that would help the platform grow as fast as its business was growing, and would keep working for them.
We proposed many recommendations as part of the roadmap, the top three being:
1. Center of excellence
We provided operating models to establish a Center of Excellence (COE) group that would support the enterprise with current and future data use cases. The COE will help the client expand the use of data beyond the commercial side of the business to other functions such as Finance and HR.
2. Architecture frameworks
We laid out the steps to enable self-service user onboarding, maturing to architecture-as-a-service, and improving application information security. Architecture-as-a-service was particularly appealing to the client because it assured strategic guidance, governance, and quality assurance across all data processing that it does now and in the future.
3. Platform optimization
We gave the team an extract, transform, load (ETL) framework, which functions as ETL-as-a-service, to optimize the current state of their data model.
ETL is used to read data from a database, transform it into another form, and finally load it into another database. The ETL framework will help our client deal with increasingly large quantities of data.
ETL-as-a-service takes the client’s data, whatever format it’s in and from whichever data source, and brings all that information together to make it accessible to the client for all its business use cases.
Ready to scale
Once we delivered the roadmap, the client team had a clear understanding of how they would use their platform in the future, and they were ready to get started with the implementation.
Our goal is never to create a dependency on our resources, and we succeeded in handing off the project to the client and preparing its teams to execute on the vision. The company is now ready to automate new user onboarding and other tasks, meeting the demand for access to its data platform and sustaining the growth of the enterprise.
The company is now ready to meet the demand for access to its data platform and sustain the growth of the enterprise.
A template for next time
We observed that this kind of assessment was valuable not only for the current client, but that we would likely get similar requests from clients in the future. With that in mind, we templatized the evaluation and interview questions, and created a reusable framework from the outcome of the evaluation.
The next time we’re asked to do a project like this, we’ll be able to get started right away and deliver results quickly, at a lower cost to the client.
