Blogs

Time-Saving Data Refining Just for Application Developers

Delight application end users with refined, ready-to-use data

Program Director, Analytics Platform Marketing, IBM

When application developers define their responsibilities, they may list activities such as gathering requirements, coding to meet those requirements, developing test plans, and completing projects on a timely basis in accordance with their commitments. They typically do not include the requirement of provisioning good data to end users of their applications. But without good data, most applications have little value.

Refining data for its intended use

As a result, a frequently unspoken responsibility for application developers is to make good, refined data readily available to end users through capabilities built into the application. Refining or preparing that data involves making the data fit for its intended use (see figure), and may include the following functions:

  • Bringing data into alignment with standards
  • Choosing the master data from inconsistent or overlapping alternatives
  • Finding, integrating, and shaping data
  • Protecting personally identifiable information that shouldn’t be visible to unauthorized users
  • Resolving conflicts

 
Time-Saving Data Refining Just for Developers – figure

Invoking cloud-based data refinement with minimal time investment in business application development for access to the right data

 
Because data refinement may not be on the developer’s own priority list, all this discovery and preparation needs to be achievable without weighing down the developer with lots of additional work or slowing down the development process. If including refined data as part of a new application were simple, everyone would do it. But in the past, it hasn’t been easy for three key reasons:

  • Developers themselves haven’t had quick access to data. Making a request to IT and waiting for the data to be delivered could slow down development. Making certain assumptions about the data and then continuing to build functionality into the application was often easier—and more sensible.
  • The process of creating refined data—including data cleansing, data masking, or data profiling—required a significant amount of development time.
  • The downstream problems that could arise from developing an application that is not data aware—such as legacy, incomplete, or unreliably sourced data that end users may acquire through the application—typically did not end up on the developer’s desk. Instead, IT had to handle the complaints.

Developing enhanced applications efficiently

A new IBM approach to data access and data refinement makes it easy for developers to address data issues as they build their applications, without slowing down delivery cycles. The approach uses readily available, cloud-based data access and refinement services that can be built into any cloud application. Because application programming interfaces (APIs) are a preferred tool for application developers, the services are offered as APIs that developers can use to enrich their applications and improve the quality and usability of data that businesspeople receive.

The IBM® Bluemix™ cloud-based platform as a service (PaaS) offers IBM DataWorks APIs with functions that can be called up as part of any process. Examples of IBM DataWorks services include loading data, cleansing addresses, profiling data, and classifying data to prepare data for use in applications.

The IBM Watson™ Analytics application embeds IBM DataWorks, and built-in data refinement capabilities enable end users of Watson Analytics to easily and quickly get the data they need to analyze. They spend less time searching for data and more time doing what they were hired to do—analyzing data to help the business. Developers of other applications can build the same type of easy access to appropriate data into their own applications.

Have you faced the challenge of baking easy access to appropriate data right into an application you’re building? Please share your thoughts and experiences in the comments.

Resources