Making Machine Learning Applications Fast and Simple with ONNX

So, you’ve trained your machine learning model and now you want to put it into production? Possibly, you even would like to deploy a microservice, where you can expose its prediction logic through an HTTP interface. If you’ve ever gone through this process, you’ve probably encountered two problems:

• Moving Python-based machine learning models around is difficult, as the model is not fully self-contained and needs to be packed with all of its dependencies.
• Making a prediction for a single input from an HTTP request can be slow using a batch-optimized Python-based model, possibly too slow.

In this talk, you will learn how we solved this problem in a dynamic-pricing project using the Open Neural Network Exchange (ONNX) framework, where we complied our model into a statically typed, fully self-contained computational graph that we could invoke using an optimized runtime, which allowed us to reduce single-row inference time by 99%, while also drastically simplifying our model management.

You will be introduced to the ONNX ecosystem, consisting of the specification itself, as well as converters, runtimes, optimizers, and visualizers. You will also learn how to convert scikit-learn-based modeling pipelines to the ONNX specification, using sklearn-onnx and onnxmltools. Finally, you will learn how to implement ONNX converters for your custom machine learning transformers and estimators.