6 steps to start your DataOps practice
DataOps is the orchestration of people, process, and technology to accelerate the quick delivery of high-quality data to data citizens. When done right, DataOps creates business value because users know what data they have, can trust the quality and its meaning, and use it without violating compliance or privacy laws.
1. Assess your organization’s DataOps maturity
DataOps could be transformational to an existing organization and established processes. The intention of DataOps is to automate many existing manual tasks and streamline the data pipeline creation process. Whether your organization is starting to develop a basic DataOps practice or sustain a more developed practice, it is important to baseline your team’s ability to deliver business-ready data fast and make a plan for improvement that aligns with creating business value.
DataOps success begins with cataloging your data assets by capturing metadata and assigning policies to data classes, assessing and scoring data quality, and leveraging tools for integrating data (as opposed to spreadsheets, tribal knowledge, or hand coding). Once your team’s maturity level has been defined, the goals and objectives should be to improve capabilities across as many DataOps dimensions as possible.
2. Align your DataOps mission to high priority business initiatives
DataOps teams should focus on aligning the delivery of necessary data with the value it can bring to the business. Ask the question: How much money could be saved or earned if this information were made available quicker?
For example, if a bank could quickly determine a fraudulent event was occurring, how much money could be saved every hour? If a retailer could determine inventory stock positions in seconds, how many online shoppers could be converted to sales by minute? It is important to begin by focusing on a problem statement or business initiative that is capable of delivering instant value back to the business.
3. Assign a small dedicated squad with top skills
Identify and assign an initial squad of highly skilled, open-minded resources. Remember, DataOps is a transformational initiative. You want a team that is willing to think differently from how things are being done today. Limit the team to about six to seven people. Include a DataOps project lead, a developer with DevOps toolchain experience, a data engineer, a data steward or quality analyst, a data governance and privacy representative, a business initiative focal, and representatives from the team consuming data to meet the business initiative.
4. Run a DataOps Kickoff Workshop
With your “Ace” team identified, assign DataOps pre-reading and hold a half to one-day workshop to kick off the project. In your workshop, describe the problem statement and why this business initiative is a priority. Agree on the critical data elements required to solve the problem. Have the team map out the data pipeline that will be required to deliver the data set. As you explore the mapping, make sure to identify known bottlenecks that create delays in delivering the data (the “As-Is” state). Problem solve ways to use automation and collaboration to reduce those bottlenecks.
Before you leave the room, ensure you have the inputs needed to prototype a data pipeline based on the desired “To Be” state. In order to maintain motivation and commitment from your team, keep your pilot project to a limited scope. Target no longer than six weeks or three sprints and define metrics to measure progress.
5. Execute a Pilot Project
The goal of the pilot project is to measure and baseline your team’s ability to identify data assets, gain access and understand their meaning, and integrate and deliver information to the business. By capturing telemetry associated with the process itself, you can verify if your “To be” process is reducing bottlenecks.
Measure how quickly data was delivered throughout the project and capture the impact it had on the business. At the completion of the pilot, assess what worked well and which areas could use some improvement. If the first iteration took two weeks (14 days) to deliver the data, what would it take to reduce it to 10 days? Or one day? What business value would that create? And then define your next pilot.
Establishing a DataOps practice, requires just that – practice. Use the pilot project’s success to expand and grow the DataOps skills and organization. Promote its success to recruit more teams to participate in the DataOps practice. Share lessons learned and start to build your own DataOps Center of Excellence. Good news spreads – let the success fuel more success.
Learn more about the IBM DataOps Program
IBM is here to help you on your path to a DataOps practice with a prescriptive methodology, leading technology, and the IBM DataOps Center of Excellence, where experts work with you to customize an approach based on your business goals and identify the right pilot projects to drive value for your executive team. Reach out for a consultative discussion and to schedule an IBM DataOps Garage Workshop.
Accelerate your DataOps learning and dive into the methodology by reading the whitepaper Implementing DataOps to deliver a business-ready data pipeline.