A big data–enabled modern recommendation engine for retail banking

Director for Watson and AI applications, IBM

The global banking industry is at the brink of disruption. Consumers’ affinity toward fintech products and new banking alternatives such as peer-to-peer lenders—including MoolahSense in Singapore and RateSetter in Australia—are driving them away from traditional brick-and-mortar retail bank branches and automated teller machines (ATMs). Consumers are more likely to use alternative banking services such as fintech products when legacy financial services aren’t meeting their needs and are not providing service up to the mark. As banks are challenged to grow revenues and contain costs, few resources are available for product innovation, which brings them to great risk of being disrupted and experiencing revenue erosion.

Millennials will be the most active users to move towards modern Fintech and digital banking. They will be most likely walking away from using traditional and legacy banking systems. As per PWC studies Millennials form 25% of the workforce in the US and by 2020 they will form 50% of the global workforce.  FICO consumer research reckons that 83 percent of millennials use credit cards to support their lifestyles.  As can be seen from these numbers, if this generation moves away from banks it will pose profound threat to revenues and profits of banking industry.

The banking industry has to tap into millennials in the same way as entities such as Amazon and Netflix are doing so. One great way of tapping into this generation is by using big data to drive a modern recommendation engine. A big data–enabled recommendation engine is similar to what industry leaders such as Amazon, Netflix and Pandora are using to recommend next-best offers based on purchase history and the purchase histories of similar customers.

The backbone of recommendation engines

Big data technologies together with data science provide the means to better profile the customer. Data science–based collaborative approaches and content-based filtering are being extensively used to offer the most appropriate product or bundle of products at any given time to customers.

Retail banks have to increasingly rely on marketing and product innovation to grow the total value of their average customer. Simultaneously, banking customers are using more channels, particularly online and mobile, to access banking information and transact in real time, thus increasing the volume and variety of data collected and the number of sales opportunities. This data, when consolidated and analyzed longitudinally, cannot only be used to better profile the customer and segment the market, but also develop advanced models on the most appropriate product or bundle of products to offer at any given time.

In the past, the cost of storing data for all customers and customer interactions has prevented that data from being actively analyzed to drive real-time offers. A consequence of that ineffective marketing is that it depletes customer sentiment and drives down return on investment (ROI). Product cross-sell and up-sell efforts have had diminishing returns, as offers are based on novelty or popularity, not tailored to customers.

The data science solution pillars

The solution is composed of data ingestion and machine learning to gain insights from historical customer-and-product interactions, as well as a framework for leveraging a search engine to engage customers by delivering real-time recommendations. Using a big data solution enabled by Apache Spark and Apache Hadoop transition data from specialized silos to a central data store that collects not only transactions, but also clickstreams, service logs, social data and so on, in full fidelity is possible.

Banks can find data using Apache Solr search and build models using Apache Pig, Spark and ecosystem analytics tools such as R. Machine learning models can be tested and experiments can be run on real, large data sets to iterate and improve processing. This approach can significantly shorten timelines to operationalization and production of the recommendation engine use case. A recommendation engine needs to automate analysis with increased yields while taking care of the following processes: 

  • a profile in real time
  • Suggest a next-likely product to be offered
  • Generate dynamically a corresponding scorecard based on various business rules
  • Track dynamically an offer history to avoid repeating offers
  • Capture event data showing customer interactions—including channel, time, intermediate behaviors—and not just transactions in real time
  • Generate automatically relevant marketing notifications and promotions 

Despite the uncertainties and challenges, significant opportunities are available for retail bankers to redefine their business and operating models, unlock new value and provide innovative customer propositions. A big data–enabled recommendation engine offers one such initiative and is expected to be realized primarily by forward-looking banks that gain a first-mover advantage. Learn more about how banks are using big data technologies to build new products and services by attending IBM Insight at World of Watson 2016, 24–27 October 2016, in Las Vegas, Nevada.

Register for IBM Insight at World of Watson 2016