Netflix Open Sources Polynote, Opening up Machine Learning Workflows for All

The movie-streaming site shared the information on its tech blog.
Fabienne Lang

Media giant Netflix shared its new and freely available multi-language programming notebook, Polynote. The company has done this to help streamline the process of machine learning and data science development for others. 

Polynote integrates with Apache Spark and supports Scala, Python, and SQL.

Netflix shared all the open-source information via their tech blog.


Netflix and Polynote

Polynote was created to allow data scientists and AI researchers to integrate Netflix's JVM-based machine learning system with Python machine learning, as well as visualization libraries. 

Netflix said: "On the Netflix personalization infrastructure team, our job is to accelerate machine learning innovation by building tools that can remove pain points and allow researchers to focus on research. Polynote originated from a frustration with the shortcomings of existing notebook tools, especially with respect to their support of Scala."

They continued, "At Netflix, we have always felt strongly about sharing with the open source community and believe that Polynote has a great potential to address similar needs outside of Netflix."

Polynote is a polyglot system, meaning that every cell in a notebook can be read in multiple languages.

Additionally, the system offers configuration and dependency setup, which is saved in the notebook, at the same time as this data exploration is enabled with Matplotlib — a Python 2D plotting library — and Vega, the visualization grammar.

To make matters simple, Polynote uses a symbol table that gives insight into the notebook's internal state. Each user can set their own dependencies for each notebook through the configuration system. 

Most Popular