Introducing MLRun Community Edition

MLRun Community Edition (CE) is the out-of-the-box solution of MLRun for AI and ML orchestration and model lifecycle management.

MLRun CE can be installed directly on your Kubernetes cluster or even on your local desktop. It provides a complete, integrated MLOps stack that combines MLRun’s orchestration power with Nuclio’s high-performance serverless engine, along with additional tools for data storage, monitoring, and more.

In this blog, we’ll explain how MLRun CE works and recommended use cases, and share how one of our users leverages MLRun CE for experiment and model tracking.

MLRun CE is ready to use out of the box. It is designed to simplify the entire lifecycle of LLM and ML projects, and provides a robust solution for complex MLOps needs (see examples below).

Under the Hood

By easily installing the MLRun CE Helm chart on your Kubernetes cluster or local desktop, you get a powerful, integrated environment for development. The platform is built on two cores: MLRun for MLOps orchestration and Nuclio for serverless computing.

MLRun is the MLOps orchestration framework that automates the entire AI pipeline, from data preparation and model training to deployment and management. It automates tasks like model tuning and optimization, enabling you to build and monitor scalable AI applications. With MLRun, you can run real-time applications over elastic resources and gain end-to-end observability.

Nuclio is a high-performance serverless framework that focuses on data, I/O, and compute-intensive workloads. It is the engine that powers the real-time functions within MLRun. Nuclio allows you to deploy your code as serverless functions, which are highly efficient and can process hundreds of thousands of events per second. It supports various data sources, triggers, and execution over CPUs and GPUs. It also supports real-time serving for generative AI use cases.

Key Integrations for a Complete MLOps Solution

MLRun CE easily integrates with several other tools. It includes an internal JupyterLab service for developing your LLM code and supports Kubeflow Pipelines workflow for creating multi-step AI pipelines. It also works with Kafka and TDengine for robust, real-time and batch model monitoring, and provides built-in support for Spark and Grafana for data processing and visualization.

Data and developer users of MLRun CE can benefit from:

• Open-source MLOps Solution: MLRun CE is an open-source MLOps platform that you can quickly install on your Kubernetes cluster or local desktop by deploying the mlrun-ce chart.
• Integrated MLOps Workflow: MLRun CE combines the MLRun orchestration framework with the Nuclio serverless engine to provide a complete MLOps solution. This allows users to seamlessly automate tasks from data preparation and model training to deployment and monitoring. This integration eliminates the need for teams to stitch together disparate tools, saving time and effort.
• Rapid Production Deployment: The platform allows you to take your code from a Jupyter Notebook or your local IDE to a scalable k8s cluster with minimal changes. This significantly shortens product development, enabling faster iteration and business impact.
• Scalability and Efficiency: With Nuclio as its serverless engine and MLRun as an MLOps orchestrator, MLRun CE can automatically and elastically scale resources based on demand. This ensures your workloads, whether batch or real-time, run efficiently, reducing computation costs. It’s particularly useful for resource-intensive tasks like LLM fine-tuning or inference.
• Real Time and Batch Model Monitoring: MLRun CE includes a real-time and batch model monitoring solution, based on the fully out-of-the-box integration with Kafka and TDengine.  You can track models, compare results and performance metrics, and detect data drift or anomalous behavior. It also supports automated alerts for model exceptions, enabling proactive maintenance and ensuring continued model reliability.

Seamless Integrations: The platform integrates with a wide range of popular open-source tools, including Kubeflow Pipelines for workflow management, and Spark and Grafana for data processing and visualization. This open architecture gives you the flexibility to use the tools you already know and love.

The following are the components that get installed when installing MLRun CE

1. MLRun
2. Jupyter notebook
3. Kafka & TDengine
4. Minio
5. Nuclio (for realtime functions)
6. Grafana & Prometheus
7. MPI
8. Minio
9. Spark Operator

The picture below describes the relations between them. MLRun is the orchestrator and deploy function by using MLRun, Nuclio, Spark and MPI jobs runtimes. Grafana is used to monitor usage, Jupyter for out-of-the-box development platforms and Minio, MySQL & TDengine to store data.

MLRun CE can be used for a wide variety of MLOps use cases. In particular:

• MLRun UI for project management – Track project experiments, artifacts, model performance, and manage project members and secrets
• Batch jobs for retraining processes and user experimenting
• Experiment tracking and results tracking, including duplication and debugging
• Real-time serving for generative AI use cases
• Alerts mechanism for threshold results like data drift 
• Real-time functions for model inferences, realtime data processing and more
• Runtime resource management and scaling
• Models artifactory that allows users to track datasets and model artifacts by tags, labels and results

Spotlight: How One of Our Users Manages Their Experiments and Models with MLRun

One of our community users has adopted MLRun CE as their MLOps platform to deploy, track, and manage their ML training experiments and models. MLRun CE is deployed across Kubernetes environments.They run two main types of ML workflows run through MLRun CE. The first is manually triggered training jobs. MLRun CE runs the training function, logs metrics and datasets, and registers the model for deployment on edge devices. 

The second is automated periodic insight models, such as drift detection functions that compare recent data against training distributions and generate alerts when anomalies occur.

The team relies on MLRun CE’s full set of components: project management, batch functions, experiment tracking, model monitoring, and alerts.

With MLRun CE, their data science teams can:

• Run experiment tracking
• Ensure model integrity
• Duplicate experiments
• Easily debug experiment logs
• Deploy models easily and quickly
• Manage resources and scale

Check out these resources for more information:

• Documentation: MLRun Docs, Nuclio Docs
• Quick Start: MLRun basics tutorial
• Cheat Sheet: MLRun cheat sheet
• GitHub: MLRun Repository
• Community: Join our Slack for support, discussions and feedback