Meson Workflow Orchestration and Scheduling Framework for Netflix Recommendations
Netflix’s goal is to predict what you want to watch before you watch it. They do this by running a number of machine learning (ML) workflows every day. Meson is a workflow orchestration and scheduling framework that manages the lifecycle of all these machine learning pipelines that build, train and validate personalization algorithms to help with the video recommendations. Netflix development team recently wrote about Meson framework and how it helps with the ML pipelines. One of the goals of Meson is to increase the velocity, reliability and repeatability of algorithmic experiments while allowing engineers to use the technology of their choice for each of the steps in the pipeline. Some of the technologies that play an important role in machine learning pipelines at Netflix include Spark MLlib, Python, R and Docker.…
Link to Full Article: Meson Workflow Orchestration and Scheduling Framework for Netflix Recommendations
Sharing
