Product Clustering Techniques in Demand Forecasting

Demand forecasting is a common Time Series use case in DataRobot. Using historical sales data, together with data related to product features, calendar of events, and economic indicators, we can produce forecasts of future demand. We can then use these forecasts for inventory and supply chain planning, making overall operations more efficient.

For some companies, a common issue in demand forecasting is the abundance of product lines, or stock keeping units (SKUs). The problem is that it is not an easy process to produce accurate forecasts for every SKU when there are so many of them. A demand forecasting model for a typical retailer of consumer products might have thousands or tens of thousands of SKUs to forecast.

To improve the performance of such demand forecasting models, we can use several modeling techniques. All of these techniques center around product clustering, where product lines or SKUs that are “closer” or more similar to each other are clustered and modeled together.

In this blog post, we describe these strategies. We can apply more than one of these techniques in Demand Forecasting use cases.

1. Clustering by product group

The most intuitive way of clustering SKUs is by their product group. It is straightforward that different types of products have different demand patterns. For example, the demand for expensive jewelries is relatively low and highly dependent on gold prices. Fashionable daily accessories, such as necklaces and earrings, typically have consistent demand patterns and higher transaction volumes. Sales patterns for themed jewelries exhibit strong seasonality and have peaks over certain events, such as Valentine’s Day and Mother’s Day. Modeling product groups with similar nature of demand together can help the model better capture the driving factors, resulting in better forecasting accuracy. It is also not hard to prepare such datasets by simply using the product group as the filter.

2. Clustering by sales profile

Building models per product group is a simple and straightforward starting point. However, it is possible that products in the same product group might exhibit significantly different sales patterns or profiles. Plotting the historical sales can help visualize the pattern, identify obsolete or zero-inflated items, and create better sub-clustering. A side benefit of doing so is the ability to spot potential data issues, as well as gain insights from interesting sales patterns that the business unit might not be aware of.

Plotting historical sales patterns might require some coding skills if we are talking about plotting thousands of SKUs. However, it shouldn’t be too hard to achieve with Python or R. Some BI tools, such as Tableau or PowerBI, can easily perform these tasks as well.

3. Using performance clustered models in DataRobot

In performance clustered models such as the one above, series data is clustered according to the accuracy that can be estimated for each series using a base model. In this case, an eXtreme Gradient Boosting model (XGB) is used as the base model. After several iterations of XGB estimation, series that have similar model performance are then assigned to individual clusters. This model has five clusters.

The information on how many clusters the model created and how many series were in each cluster is in the coefficients section. Complete mapping of Series ID to the cluster numbers is available to download using the Export option in the same window.

The number of clusters to be used is a hyperparameter in performance clustered models and can go up to 20.

4. Using Series Insights

Alternatively, we can also study individual series using the Series Insights tab. Series Insights tells us how each series is performing based on a selected metric (e.g., SMAPE) across different backtests.

From here, we can discover series that have similar model performance and/or product group or sales profile. Product clustering can be formed using a mix of model-driven and business-driven insights. We then build individual DataRobot projects based on this product clustering.

Finally, DataRobot has blueprints that can handle both cold-start and warm-start series in the same model. Generally, these blueprints use a two-stage approach:

1. The main effect model is built, which works well on averaged derived features.
2. The known-in-advance features (if available) are used to account for series effects for zero history records, and series effects are used for partial history.

In summary, we can use product clustering techniques to improve model performance in demand forecasting use cases. Using DataRobot, we can quickly identify series that tend to perform similarly, and we can use a mix of model-driven and business-driven product clustering to optimize our demand forecasting models. For more information on DataRobot’s Automated Time Series offerings, visit DataRobot Community for self-learning materials and webinar recording.

About the author

Wei Zhong Toh

Customer Facing Data Scientist, DataRobot

Wei Zhong is a Customer Facing Data Scientist (CFDS) at DataRobot, located in Singapore. His areas of expertise include public sector, healthcare, and telecommunications. Wei Zhong holds a MSc in Statistics and a BSc (First Class Honours) in Computational Biology, from the National University ofSingapore.

Meet Wei Zhong Toh

---

Nina Xing

Customer Facing Data Scientist

Nina is a Customer Facing Data Scientist at DataRobot, based in Singapore. Nina has data science expertise in fintech, insurance, manufacturing, and retail industries. Prior to joining DataRobot, Nina worked in the financial market and specialized in portfolio performance and risk management. Nina is a Chartered Financial Analyst and holds a bachelor’s degree in Mathematical Sciences from Nanyang Technological University in Singapore.

Meet Nina Xing