Model Interpretability

What is Model Interpretability in Machine Learning?

A machine learning algorithm’s interpretability refers to how easy it is for humans to understand the processes it uses to arrive at its outcomes. Until recently, artificial intelligence (AI) algorithms have been notorious for being “black boxes,” providing no way to understand their inner processes and making it difficult to explain resulting insights to regulatory agencies and stakeholders.

Some models, like logistic regression, are considered to be fairly straightforward and therefore highly interpretable, but as you add features or use more complicated machine learning models such as deep learning, interpretability gets more and more difficult.

Why is Model Interpretability Important?

When using an algorithm’s outcomes to make high-stakes decisions, it’s important to know which features the model did and did not take into account. Additionally, if a model isn’t highly interpretable, the business might not be legally permitted to use its insights to make changes to processes. In heavily regulated industries like banking, insurance, and healthcare, it is important to be able to understand the factors that contribute to likely outcomes in order to comply with regulation and industry best practices.

Model Interpretability + DataRobot

DataRobot includes several components that result in highly human-interpretable models:

• Model Blueprint gives insight into the preprocessing steps that each model uses to arrive at its outcomes, helping you justify the models you build with DataRobot and explain those models to regulatory agencies if needed.
• Prediction Explanations show the top variables that impact the model’s outcome for each record, allowing you to explain exactly why your model came to its conclusions.
• The Feature Fit chart compares predicted and actual values and orders them based on importance, allowing you to evaluate the fit of a model for each individual feature.
• The Feature Effects chart exposes which features are most impactful to the model and how changes in the values of each feature affect the model’s outcomes.

DataRobot works to ensure that models are highly interpretable, minimizing model risk and making it easy for any enterprise to comply with regulations and best practices.