Contents


Introduction

Machine learning remains a hot topic and currently we may just witness the beginning though many applications are around for quite some decades now - despite the AI winter. Machine learning frameworks are often programmed in C++ or CUDA nowadays and provide a wrapper for some of the programming languages that are used in this field. Python is the most dominant script language used for “rapid development” and prototyping, then there is Julia and some people are still using R and other even slower languages. However, it is a bit different in deployment/inference scenarios.

General Thoughts

With rather well established ecosystems, why on Earth would someone choose Rust for machine learning applications. Let’s have a look at some practical considerations regarding using Rust for machine learning and exclude the fan boy scene. The Rust Machine Learning Book looks rather poor and the majority for crates seem to be wrappers and less native implementations. Are we learning yet seems to provide a somewhat up to date state of the ecosystem description.

Rust has extremely nice safety nets to prevent really big fuck ups and errors. Yes, perhaps many of these could be achieved using C++ as well if people would stick close to the standard library and enforce extremely strict compiler settings but that hardly ever happens outside industries where it is really vital. Especially for the “web development crowd” and related fields where sloppy programming is not just tolerated but widely accepted, Rust is a decent choice to enforce to prevent many of these issues. Most likely strict typing and proper memory management takes care of a lot of fuck-ups. Moreover, there are real business benefits in avoiding slow script languages such as Python (yes there is cython…) when it comes to scaling debugging.

However, there are some known issues:

  • lack native accelerator support (GPU, NPUs , FPGAs, …)
  • many crates seem to be experimental/lack rigorous testing/evaluation
  • focus on “classical machine learning” due to a lack of accelerator support

While there are many good applications for using Rust, most of us have to ask ourselves what could be reasonable applications for machine learning in Rust today (not in 5 - 10 years or so). There are applications in computer vision like strand-braid but most likely Rust is used in some web application or gateway setting already and more and more web applications use some form of machine learning to do various things. A gross generalization might be that we have some form of API written in Rust and the data received by this API is subjected to some form of machine learning applied at it or that the data originates from a different source and is streamed out. At some point data has to arrive at the ML module via some API. Based on this we could distinguish two basic use cases (pure data storage and ML applied later excluded):

  • pure inference based on input data
  • input data provided used for updating and existing model

A combination of these two cases is possible as well but let’s keep it somewhat simple for now.

Most likely the best use-case for machine learning with Rust is applying pure inference of classical algorithms which are available in Rust on input data. All other cases require at least the usage of a wrapper or some API with a module programmed in C++ or Python. Using wrappers might actually increase development complexity and therefore should be evaluated carefully.


These thoughts might be considered as a “sane approach to machine learning with Rust” ;).


Software Libraries

Like with many crates there seem to exist may small ones that basically do the same but nothing is remotely mature. Given that SmartCore and linfa are at least somewhat usable the question arises why people involved don’t pool their resources and build at least one production grade machine learning library in Rust. linfa however seems to be under active development whereas SmartCore’s development looks to be in a ‘bug fix only’ stage.

Programmed in Rust/Native Crates:

  • automl
    • very basic AutoML based on smartcore
  • candle
    • minimalist ML framework
  • egobox
    • optimization tool box
  • juice
    • deep learning framework programmed in Rust with some GPU (CUDA/cuDNN) support
  • linfa
    • an attempt to rewrite something like scikit-learn in Rust
  • MOSEC
    • framework agnostic inference engine similar to TF-serving or Triton with a strong Python API
  • rstats
    • statistics crate with some interesting linear algebra functionality
  • rusty-machine
    • deprecated machine learning library
  • smartcore
    • an attempt to rewrite something like mlpack in Rust
  • tokenizers
    • Hugging Face’s tokenizer library
  • tract
    • inference engine for tensorflow and ONNX models with some optimization for e.g. ARM based devices

Wrappers

  • darknet-rust
    • bindings for the Darknet framework
  • paddle-sys
    • bindings for the Paddle inference engine (programmed in C)
  • tch-rs
    • Rust bindings for libtorch