Meet the Challenges of Assured Compliance and Governance with DataRobot 9.0

As you grow and optimize your business with AI, you must deliver models you can trust and defend over time. This requires you to proactively implement safety best practices and apply the highest governance standards within your ML lifecycle. The problem: this is really difficult – even for expert teams.

Slowing down machine learning (ML) projects to collect documentation manually can cause friction, block other projects, and delay ROI. So although model documentation is a best practice all ML teams should embrace, it can be painful to put into practice.

It’s also hard to handle knowledge loss from turnover or changes in tech stacks as data science teams evolve. With little notice, you might lose the context for how a critical model operates, leaving you ill-prepared for the future.

Every business needs the ability to audit and explain the models they have deployed, plus regulatory guidance across many industries requires strong documentation. Yet keeping up with regulations means making sure data scientists and model risk teams are in lockstep, or you’ll delay or need to retrace steps in your deployment process.

DataRobot 9.0 can help you meet these challenges. We’re expanding our governance and compliance offerings to be inclusive of all models across your organization, and ensure every business critical ML asset is governed properly. The three key capabilities are:

• Extending our existing automated documentation offering, which is unique to us, to include almost all custom models, even models built or deployed outside of DataRobot.
• New ML Flow Import and extensible features for model documentation, to enhance your model governance and flexibility.
• Bias Mitigation, so you can quickly increase your ability to deliver safe and sound models.

Save Time by Automatically Creating Compliance Documentation 

If you have undocumented models running throughout your organization, on different clouds or platforms, you’re exposing your team to additional risk, especially if there’s knowledge loss across your team. Until now, we offered full model compliance documentation capabilities and insights for models running in DataRobot. These save data scientists hours of work: with the touch of a button, they automatically document any model’s behavior for compliance.

Now, you can just as easily automatically create compliance documentation for externally hosted models. DataRobot 9.0 helps you:

• Quickly connect to a range of systems and document your portfolio of models, on almost any infrastructure – and all without any code or infrastructure changes.
• Bring together all the information known about a model, from multiple systems and frameworks.
• Customize compliance documentation to adhere to enterprise or industry-specific requirements.

Whether you’re working in a highly regulated industry, or simply practicing good model governance, this feature helps ensure all models tied to business critical applications are managed and reported on.

Enhance Governance and Compliance with MLflow Integration

In many enterprises, there’s not just one way of experimenting and building models. While such freedom can be an enabler, it’s also a problem when critical information is stored across multiple tools.

DataRobot 9.0 adds MLflow import and extensible features for model documentation. This enhances your model governance and flexibility, and helps your AI builders save time and increase quality.

MLflow integration lets data scientists sync metadata they have created and bring key pieces of data, benchmarks and statistics from MLflow to DataRobot. This can then instantly be used to assist with model governance, documentation, and reviews. If there is another source besides MLFlow, the new import API can be used to bring in metadata from there as well.

Model documentation can be easily extended with other datasets, charts or tables relevant to your models, such as scenarios generated from a Notebook. This means you retain flexibility and can continue to build in the way you think is best, while keeping DataRobot AI Platform as the central location to help you manage, monitor, document, and govern your suite of models.

Ensure Fairness and Correct Model Discrimination with Bias Mitigation

Bias is a risk factor in any model you will deploy for customer decisions. And with governments exploring bias regulations, organizations are increasingly interested in understanding model discrimination based on features such as race, gender, or income. It’s therefore critical to detect when bias exists in your AI models, measure it, and have real strategies to fix it.

DataRobot 9.0 introduces Bias Mitigation, so you can quickly increase your ability to deliver safe models. It includes the ability to choose from multiple strategies for squashing bias in a model to ensure fair treatment for the classes you care about. So whether you’re driven by regulatory pressures, customer and supplier expectations, or are simply designing ethical and fair AI systems, you can now prevent bias from being an unknown risk as you deploy new models.

Equip Your Data Scientists with the Tools They Need

It’s vital to help your organization document models, uphold high ethical standards, and stay ahead of bias regulations. With DataRobot 9.0, you can equip your data scientists with these tools, and accelerate model documentation, manage risk, and fix bias – all while retaining existing freedoms and flexibilities, and saving time.

To learn more and see how these new DataRobot 9.0 features can benefit your organization, watch our Compliance and Governance in Production session.

About the author

Brian Bell Jr.

Senior Director of Product, AI Production, DataRobot

Brian Bell Jr. leads Product Management for AI Production at DataRobot. He has a background in Engineering, where he has led development of DataRobot Data Ingest and ML Engineering infrastructure. Previously he has had positions with the NASA Jet Propulsion Lab, as a researcher in Machine Learning with MIT’s Evolutionary Design and Optimization Group, and as a data analyst in fintech. He studied Computer Science and Artificial Intelligence at MIT.

Meet Brian Bell Jr.