top of page

Loan Fraud Detection via Audio Analysis

Technology has grown exponentially integrating with nearly every way of living and profession. One of the important areas of focus of technology is Banking, which is pivotal to safeguard money and also the development of wealth. Since it involves money, a bank has to keep up its constant communication and notification with the client, to showcase transactions and insurance & loan offerings, which is usually done by calls. But, many miscreants with some functional knowledge of this have been scamming people and corporations by posturing as the banks and stealing money and also their identity in certain cases.



Banking Industry



Our clients are call centers and retail associates of a Singaporean bank which has branches and operations globally, who need a system to tackle thousands of fraudulent calls they attend and resolve manually every year. Most of these calls occur as low interest loan requests. These calls not only waste the time of the clients but also induce financial losses which are even more negatively impactful to the functioning of the company. The client wanted us to create a model to analyze the voice patterns used by the scam callers, and also create a method to auto-detect and prevent these calls with relative ease.



Various challenges should be tackled for a very serious issue like this. Here are some important challenges that are faced by our model:

  • Constant Updation of the Base - The scammers are notoriously known for modifying the way they talk to the victims, so the changes have to be made to the base data frequently to prevent detection failure.

  • Proper analysis of accents - Since, most of these scammers talk from different countries their accents have to be understood well by the model.

  • Proper phrasal analysis - Sometimes, the scam callers might use different phrases and words with major grammatical errors, this has to be identified properly.



Solution summary:

  • Creating an archive of data from pre-recorded & pre-identified fraudulent calls from various sources.

  • Identify hidden algorithmic patterns based on this data and train a ML model on it.

  • Use the trained ML model, along with few business heuristics to flag a scam call during an ongoing conversation.

  • Create scorecard based on these patterns, to rate calls for potentially scam for later manual quality check.

With the challenges in mind, we arrived at the following steps for a solution :

  • A sample set is created by utilizing pre-recorded calls by the clients and various sources, which are then processed and analyzed to build a voice classifier model.

  • Commonly used inflections, voice patterns, keywords are also identified and processed as an additional supporting business layer for final prediction.

  • A scorecard is generated to mark the variations and mistakes made by the scam caller during a scam call.

  • The ML predicted score is then compared with the business rules set by the banks and client to identify potential and recurring fraudsters.

  • Custom vocal analysis is implemented to identify the intent of the caller, and screen it using the questions used by these fraudsters (identified by the scores and datasets).



Our model prevents various complications faced by this issue, and formulates a veritable solution for clients to safely screen and disregard these harmful scam calls. This system creates not only a safer functioning of the business but also proposes a smoother and streamlined functioning of the calls attended by the clients. We can also prevent data theft using a separate identity check model.


After a successful implementation of our product for 2 business units for 4 locations in about 3 months, we received below feedback statistics from the client:

  • 56% increase in accuracy by just the initial 220 fraudster call sample set trained model than the previous business rules.

  • ~94% model accuracy observed on trial set against business rules, and 82.4% accuracy observed on realtime 3 month successful live run with manual quality check (after adding phrasal inflow implementation).

Ready to put AI to work for your business?

Make a plan and understand your ROI before you start implementing AI. 
Don’t fall into the trap most companies fall into. 
Take the first step—Get in touch today.

bottom of page