About
Leveraging deep learning (LSTM) to analyse sequence of events that have occurred in a customer's life time in order to anticipate their financial needs and offer personalized recommendations
Innovation presentation
In our continuous quest to enhance customer experience and tailor our products/services to individual needs, we have developed an innovative AI-driven solution known as the Event Sequence Mining Engine for Predicting Next Best Action.
The AI engine leverages the power of Deep Learning (LSTM) to analyse time series led sequence of events happened in customer life to anticipate customers financial needs and provide them with personalized product recommendations.
The bank has over 60 million customers conducting more than 480 billion interactions annually across various bank products and platforms, whether it's routine interactions like booking a cab or buying groceries, or significant life events such as Childbirth, marriage, buying a home etc.
These interactions and sequence in which these interactions happen, leave behind a rich tapestry of insights about the customer, essentially capturing the financial journey of each customer.
The objective is to analyse each customer's life journey captured in the form of events that happened over a period overlaid with their persona and life stage and recommend the most suitable product for cross sell.
Conventional machine learning models lack the ability to analyse intricate patterns involved in event based sequential data and further fail to capture the full context of customer behaviour. As a result, recommendations tend to be less targeted and lack the necessary context for driving meaningful engagement with customers.
Uniqueness of the project
We have developed an in-house Deep Learning based Event Sequence Mining Engine capable of understanding and deciphering intricate patterns in sequential event based data. It can uncover a set of sequences of events with a high probability of leading a customer to take a desired action, such as cross-selling a Personal Loan.
Deployed a one of its kind ensembled unique three layers analytical architecture solution (first time ever implemented in the BFSI industry)
1. Event identification using Transaction Classification Layer: Any customer interaction on the bank’s platforms (e.g. UPI/Netbanking/Digital/Credit Card) gets captured in form of unstructured logs. Using NLP techniques (TFIDF, Bag of words) , categorised these interactions into event (Medical, Child birth, large jewellery etc) and stored in customer level centralised event repository using Postgres & Airflow Dag.
2. Mining of Sequence of Event using Sequence Analyser Layer – To identify relevant high probability sequence of events, team explored numerous existing techniques from different domains, including machine translation and text summarization. These techniques included Apriori-based Generalized Sequential Pattern Identification (GSP), SPADE (Sequential Pattern Discovery Using Equivalence), and pattern-growth methods like FreeSpan.
However, we encountered runtime and accuracy challenges when dealing with a massive amount of data exceeding 20 TB. Additionally, interpreting the sequences generated was cumbersome since they were represented as vectors, requiring significant effort for analysis.
To address these challenges, we developed an in-house sequence mining algorithm leveraging PrefixSpan, which searches for sequence prefixes and extends them to find complete patterns. By employing efficient hyperparameter tuning, we strategically pruned dominating or excessively frequent events, resulting in sharper, more concise and relevant sequences.
3. Deep learning Based Bi-directional LSTM Layer to identify high probability sequences of events : At the core of the solution lies a deep learning Bi-directional LSTM (Long Short-Term Memory) model to mine customer level sequence of events present in timeseries order. Overlaid with Forward Neural Network layers (ANN) to analyse static attributes like customer’s income, profile, demographic & location etc.
In summary, this innovative ensemble approach significantly enhanced the model's performance, marking a successful experiment with a blended deep learning architecture.
This unique Deep learning-based Sequence Mining Engine is a pioneering solution that harnesses AI to analyse raw customer transaction data, providing personalized product recommendations based on intricate event sequences. It's a game-changer in delivering tailored financial services and enhancing customer engagement.
