Harmoney and DataRobot Drive Innovation in Australasia’s Personal Loan Market

Harmoney challenges the big banks with marketplace lending

With a total value of more than $116B, Australia and New Zealand’s personal loan market is dominated by six banks, which fund loans from deposits, and two global credit card companies. Harmoney, Australasia’s leading marketplace lender, challenges the banks’ dominance using marketplace lending, an arrangement where investors seeking to earn a return invest money which is then lent to creditworthy borrowers. Personal loans comprise about 4 percent of the banks’ balance sheets while generating about 16 percent of net profits. These high net profits indicate that opportunities exist for entrepreneurial companies able to deliver greater efficiency to the market. Harmoney’s platform delivers this efficiency and by matching borrowers with lenders without the need for a financial intermediary, ensures everyone gets a better deal.

By understanding the information that is truly predictive of default risk [with machine learning], Harmoney has reduced the number of questions we must ask borrowers.

— Andrew Cathie, Chief Data Scientist, Harmoney

Improving the efficiency of credit risk modeling

While Harmoney is creating short-term disruption, it is planning for long-term success. Effective risk management is critical to this success.

Traditional lenders, such as the banks, assess credit default risks by asking prospective borrowers a list of questions. The fact is, the longer the list of questions a prospective borrower must answer, the higher the drop-out rate. So, assessing risk this way introduces a different risk — driving away prospects who refuse to waste their time answering questions they see as needless but who represent good business opportunities because they have low propensity to default.

Eschewing the industry norm of a long, paper-based process, Harmoney is innovating the credit application process at the heart of its business to deliver accurate credit risk analysis.

Harmoney discovered that machine learning models can be highly effective when assessing risk of default. As Andrew Cathie, Chief Data Scientist at Harmoney explained, “By understanding the information that is truly predictive of default risk [with machine learning], Harmoney has reduced the number of questions we must ask borrowers.”

Harmoney’s innovation delivers a number of outcomes. By asking only the necessary questions, the company accelerates borrowers through the credit application and reaches a decision faster. Machine learning improves the accuracy of credit risk assessments, and correlates directly with increased profitability to lenders by reducing defaults.

This accuracy also benefits borrowers. Currently, the banks generate net interest margins (NIMs) on personal loans of about 12 percent, while credit cards generate about 15 percent. In comparison, for the first quarter of 2017, US banks generated average NIMs of about 3.1 percent. By using machine learning to generate highly accurate predictive models, Andrew and his team created a new Credit Risk Scorecard that Harmoney uses to lower the price of loans it offers expressed as the interest rate individual borrowers are charged based on their risk score.

By creating demonstrably more accurate risk assessments based on each individual’s circumstances, Harmoney is more efficient than the large incumbents, resulting in better value for borrowers and a low default risk for lenders while simultaneously providing momentum for the company to win an increasing share of a highly competitive market.

Harmoney’s pace of innovation demands greater productivity

Andrew’s team of data scientists has found it necessary to change the way it works in order to deliver the machine learning models necessary for Harmoney to continuously innovate in the personal loan market.

Andrew has a team of four people responsible for analytics. But, as Andrew explained, “Our day-to-day work involves supporting many different applications as well as maintaining our data warehouse and ensuring our company completes the compliance reports required by our industry regulator. Although we understand the value of predictive analytics, the reality is that the time we can dedicate to it is limited.”

Modeling involves a large amount of data work, including exploratory “thread pulling,” which is experimental. For the team, using traditional tools – both open-source and vendor-supplied – was very time-consuming. Frequently, this work was interrupted when urgent situations arising in other areas of Harmoney’s business demanded their attention. “When we returned to a modeling project, we would inevitably go back over ground already covered to pick up the threads, an unplanned consequence of the tools we are using,” said Andrew.

Additionally, when using these tools, it was easy for the team to be distracted by technical details of data science and to lose sight of the real goal which should always be making the best use of the company’s data to improve the business and better serve the lenders and borrowers who connect across Harmoney’s peer-to-peer money marketplace. “In the legacy modeling environment our fundamental challenge was that we were not getting enough done,” Andrew concluded.

Harmoney’s requirements for an enabling technology

Andrew and his team took time to consider the process they followed as they developed machine learning models. “The real trigger point was our realization that there must be a better way for us to deploy models from the realm of data science and into business operations,” said Andrew. “This was a big issue for us, as once our data science team had produced a model that could deliver an advantage for our company, we hit roadblocks which delayed getting our new model into production where they could be put to work to create value for our business.”

Andrew continued, “Once we had validated a model’s performance in our data science environment, moving it to run in our Salesforce production architecture was difficult. First, we had to convert manual code developed in SAS or R or Python into a format that would execute in a production environment. Then we had to test and validate the converted model to ensure that in the translation process we had not introduced errors that undermined the validity of its predictions. Then we needed to be sure that the converted model was capable of performing to the level required by our business where thousands of customer loan applications would need to be scored every hour in real-time.”

Realizing that they required a single integrated environment that supported both model development and model deployment, Andrew and his colleagues began their market research. “This is how we identified DataRobot as a platform that could be shared by both our data science and our IT engineering teams as a means to create great models and to accelerate their deployment into our operational systems,” Andrew recounts. “In fact, DataRobot ticked so many boxes in our evaluation that we stopped searching and immediately moved to deploy the product.”

DataRobot at work at Harmoney

Harmoney can be viewed as a technology company, as about 60 percent of its staff are engineering and data science professionals. Technologists and business professionals work closely together, with business professionals frequently generating great ideas for improving the company. As Andrew explained, “Our business experts tend to be highly non-prescriptive – they can see areas for improvement and they trust the technical specialists to create innovative solutions. From a technical perspective, this creates great freedom.”

“When the business identifies an area for improvement, we typically do some initial investigative analysis of customer behavior in our data warehouse,” continued Andrew. “We can then move into predictive modeling with DataRobot. We ensure that we engage our engineering teams early so they can identify where and when models can best be folded into their strategic projects. We discuss how customers will see the improvements that we are planning, for example, asking ‘Is it best that we deploy this advance as one or as two models?’ We then investigate how the business will consume the results of the models. In this way we build confidence as we move into deployment knowing our models are going to work for us and for our customers. Our machine learning models have behaved superbly and with great consistency.”

“Model deployment in our legacy environment was complicated and it could take us three to four months to complete the transition. With DataRobot, model deployment is trivial and can be achieved in a few mouse clicks. This is a huge enable for our business. Working at a higher cadence at the interface between data science and IT engineering creates faster returns on our investment in machine learning.”

Automated machine learning is a great enabler — it allows data scientists to deliver more valuable work than they could otherwise achieve. At Harmoney, DataRobot is not replacing data scientists. In fact, I believe automation will increase demand for good, business-focused professionals.

— Andrew Cathie, Chief Data Scientist, Harmoney

Protecting Harmoney against systemic fraud

Although fraud has not been a significant issue for Harmoney, Andrew and the executive team are alert to the possibility that their company could become the target of systemic fraud.

Andrew welcomes DataRobot’s continual broadening of the range of models that can be created with automated machine learning. “We already have identified one application that could take advantage of a model using multiclass classification, and we see machine learning models that use anomaly detection being a valuable tool in our protective armoury.”

Advice for Companies Considering Adopting Automated Machine Learning

Automated machine learning with a product such as DataRobot isq uite a change from traditional approaches to predictive analytics. Prepare to be adaptive and change the way you work.Take the time to understand this technology, see what it does well,and give it the freedom to do its best for your company.

— Andrew Cathie