Why big data is a big deal for innovating in finance

Client Technical Leader for UK Public Sector, IBM UK Ltd.

Financial services organizations are striving to better understand their customers in hopes of improving service and managing risk. I was honored to be the keynote speaker at the Innovate Finance big data event, at which I discussed the technology trends I am seeing in big data and analytics. I talked about capabilities that can help us address challenges in the finance industry, and I particularly highlighted some challenges and opportunities for startup companies innovating in the finance sector.

Though we’re all generating lots of data in many forms, the most recent annual IBM Institute of Business Value survey tells us that in the real world of big data, most benefits are derived from speed: speed of insight followed by action. That series of studies also tells us that 55 percent of big data efforts issues in financial services focus on customer-centric opportunities. Risk management and fraud comes second, with 23 percent of projects, and after that new business models, which accounts for 15 percent. From both these observations, we see three themes directing technology:

  • IT
  • Data are never going to be in one place
  • Data gravity and value

Let’s briefly consider how each affects innovation in the financial services industry.

Two-speed IT

Reporting and business intelligence are proven and mature in our industry, but such projects are often slow for IT to deliver and can be difficult to change. Furthermore, expectations have shifted in a world that is now always on and that has become universally networked: Business users want access to data to get insight themselves—now.

However, in the highly regulated financial services industry, such self-service for access and use must be governed and ethical. Unless we get security sorted, we will not achieve the promise of big data.

Data will never be in one place

One area of focus for big data in financial services is building a richer view of client and counterparty, including their activity, from more data sources, both internal (logs, records, documents and the like) and external. There is no shortage of open-source technology for startups to use, but the challenge lies in engaging with banks. Gaining traction in a regulated space is often difficult, and to a degree often underestimated by startups.

What is needed is governed provision of data from sources, as well as controlled access for consuming that data to business users and data scientists alike. We know that 70 percent of project effort is spent preparing and refining data, a figure we can reduce by harmonization and automation.

Users will then have access to common data, being able to use their technologies of choice to explore and discover insights that help in taking advantage of opportunities, and overcoming challenges, that arise. Startups and banks thus must understand how to expose held data, managing and governing their provision. Furthermore, these organizations must learn how to push insight to those who need it even while data are being generated, not simply looking back historically.

Data gravity and value

Data gravitate toward existing data repositories, for interesting comprehensive stores of information attract more processing and more data. So how can we build value?

Use of a hybrid cloud presents opportunities to innovate through analytics, interacting with existing operational systems of record and systems of engagement. Beyond offering agility, its key design considerations are securing and moving data between bank premises and the cloud.

For startups and innovators, the dilemma is between building something that works for today but that will be a shackle tomorrow versus architecting things properly and missing the market. At IBM’s Bluemix Garages, one of which is here in London on Level 42, engineers, developers and coders from both startups and larger enterprises can collaborate with IBM’s team of techies. This platform as a service (PaaS) features more than 200 IBM and third-party services. Moreover, access to is free, allowing you to experiment and assemble services before you commit.

Not only is a wide range of big data services on Bluemix, from Hadoop and NoSQL to supercharged data warehousing and the Internet of Things, but also patterns and services are available to integrate cloud services into existing systems of record that are on premises, to aid DevOps and to provide security and management.

Big data innovation’s role in financial services is to look beyond access to technology and good people with an eye to succeeding by being the best at understanding and exploiting data. Take the first step and sign up to explore Bluemix at