The time needed to create AI-enabled Services with machine learning has been decreasing in recent years. Still, when it comes to putting models in production, developers typically spend weeks or even months learning about the infrastructure need to operate analytical or operational machine learning systems. With serverless machine learning (ML)…

TL;DR Programmers know data types. A data type for a variable in a programming language defines the set of valid operations on that variable — invalid operations will cause an error, either at compile time or runtime. But what is a feature type to a programmer new to machine learning…

Testing feature logic, transformations, and feature pipelines with pytest — TL;DR Operational machine learning requires the offline and online testing of both features and models. In this article, we show you how to design, build, and run test for features. The source code for the examples in this article are available here on github. Introduction In 2020 in our MLOps with…

An overview of Google Colossus, Facebook Tectonic, Microsoft ADLSv2, HopsFS — TL;DR: A new class of hierarchical distributed file system with scaleout metadata has taken over at Google, Facebook, and Microsoft, and provides a single centralized file system that manages the data for an entire data center, scaling to Exabytes in size. …

TLDR; Machine learning models are only as good as the data (features) they are trained on. In Enterprises, data scientists can often train very effective models in the lab — when given a free hand on which data to use. However, many of those data sources are not available in…

TLDR; Hopsworks Feature Store interacts with the new Python and Scala/Java Software Development Kit (SDK) available in Hopsworks 2.0. The new SDK builds on our extensive experience working with Enterprise customers and users with Enterprise requirements. With this new release, we consolidate multiple libraries with a single one. …

TLDR; The feature store is a data warehouse of features for machine learning (ML). Architecturally, it differs from the traditional data warehouse in that it is a dual-database, with one database (row-oriented) serving features at low latency to online applications and the other database (column-oriented) storing large volumes of features…

TLDR; Hopsworks provides support for machine learning (ML) experiments. That is, it can automatically track the artifacts, graphs, performance, logs, metadata, and dependencies of your ML programs. Many of you already know about platforms like MLflow, so why should you read about Hopsworks Experiments? Because you do not have to…

Feature Stores have become the key piece of data infrastructure for machine learning, connecting models to their data. They manage the whole lifecycle of features: from training models, to batch inference, to providing low-latency access to features by online-applications for online inference. This article introduces the Hopsworks Feature Store, the…