{"id":1702,"date":"2019-11-12T22:42:34","date_gmt":"2019-11-12T20:42:34","guid":{"rendered":"http:\/\/beeeye.com\/?p=1702"},"modified":"2019-11-12T22:42:34","modified_gmt":"2019-11-12T20:42:34","slug":"machine-learning-model-deployment-lessons","status":"publish","type":"post","link":"http:\/\/beeeye.com\/machine-learning-model-deployment-lessons\/","title":{"rendered":"Machine Learning Model Deployment Lessons"},"content":{"rendered":"

Model deployment<\/span><\/h3>\n

Working on a highly accurate machine learning model is just the first part of the complex process of putting a model into the production system. Model deployment, and specifically deploying a machine learning model can be a highly challenging task which becomes even harder when you are dealing with credit risk models.<\/span><\/p>\n

It\u2019s extremely interesting to learn from the vast experience of others in this field and to take into account their insights when deciding on your approach toward model deployment. In this blog post we\u2019ll go through some of the lessons published in the last <\/span>KDD<\/span><\/a> conference in a paper by Pablo Estevez, Themistoklis Mavridis and Lucas Bernardi. References to this paper appear in numerous sources. One which resonated nicely with what we do is <\/span>here<\/span><\/a> in the <\/span>blog of Adrian Colyer<\/span><\/a>.<\/span><\/p>\n

150 Successful Machine Learning Models<\/span><\/h3>\n

The paper focuses on the experience in deploying machine learning models by <\/span>booking.com<\/span><\/a>. Based on their experience in deploying real life machine learning models they try to draw some insights and conclusions which can benefit others in achieving good results when deploying (and developing) their own models. We found this paper and blog post highly relevant as the challenges that the team in booking.com encountered, are very common to other teams as we see daily with customers we work with. It doesn\u2019t even have to be in the same industry. Some challenges in machine learning model development are so fundamental that they are prevalent in almost any use case.<\/span><\/p>\n

In addition, gaining experience with many model development and deployment projects provides much more substance to team\u2019s expectations from any technology or platform they may be using.<\/span><\/p>\n

Managing a portfolio of models in production<\/span><\/h3>\n

One of the first things that you encounter in that paper is the fact that they are managing 150 models in production at the same time. Of course not all models handle the same capacities of predictions, however the sheer number of concurrent models to manage is impressive. With our customers we\u2019ve found that the range of active models in the credit risk area can range anywhere between around 10 to a little over 100. Unlike any other industry, managing models in the financial world requires some substantial collateral work that comes with each model:<\/span><\/p>\n