Speed as a Dimension of Trusted AI
The speed of model scoring directly impacts how it can be used in your business process. Find out how the speed of predictions influences their trustworthiness.
How Speed of Predictions Influences Their Trustworthiness
Speed in machine learning can refer to two time periods: how long it takes to build a model and how long it takes to use a model to score a prediction. Although expediting model development time is a key concern for getting a model into use and delivering value, the focus for model performance is instead on the time to score a prediction.
What Types of Prediction Scenarios Take Place in Machine Learning?
There are two most typical paradigms for model scoring.
The first is called a batch prediction. New records are sent as a “batch” for the model to score all at once. This may happen at regular intervals, for example daily, weekly, or monthly. The size of the batch in batch scoring is extremely relevant. Infrastructurally, there are massive differences between scoring a few hundred records with numeric and categorical data and doing so for gigabytes of data. The speed to score might be more or less significant in your model selection. If the process runs monthly and takes an hour, shaving off ten minutes is unlikely worth an accuracy tradeoff. If the process runs daily on a large dataset, using a computationally less intensive modeling approach that returns values faster might be the desired approach.
Increasingly, you might require predictions returned in real time. An example is in digital advertising. Sometimes the span of a click is all you have for allocating a particular ad to a particular user. For real-time prediction scoring, milliseconds might weigh in on your model selection.
Speed Is Just Pieces of the Puzzle
Speed is only one dimension of model performance that directly contributes to the trustworthiness of generated predictive models. The full list includes the following:
