awesome-kubernetes/v2-docs/mlops.md

description

description
Curated, AI-ranked MLOps resources for the 2026 Cloud Native architect: top-tier tools, guides and references (AI).

Machine Learning Ops (MLOps) and Data Science

!!! tip "Nubenetes V2 Elite Portal" You are browsing the AI-Curated V2 Elite Edition. Looking for the exhaustive list of references? Check out the V1 Historical Archive.

!!! info "Architectural Context" Detailed reference for Machine Learning Ops (MLOps) and Data Science in the context of AI.

CICD

Containers

• (2021) towardsdatascience.com: From DevOps to MLOPS: Integrate Machine Learning Models using Jenkins and Docker [PYTHON CONTENT] [COMMUNITY-TOOL] — This technical guide bridges DevOps and MLOps by demonstrating how to containerize machine learning models using Docker and orchestrate their integration pipelines with Jenkins. It provides an architectural map for automating CI/CD routines for smart microservices, ensuring reproducible builds and robust test suites.

Cloud Platforms

AWS

SageMaker

• (2022) aws.amazon.com: MLOps foundation roadmap for enterprises with Amazon SageMaker [NONE CONTENT] [ADVANCED LEVEL] 🌟🌟🌟🌟 [ENTERPRISE-STABLE] — A detailed blueprints roadmap by AWS outlining standard enterprise implementation strategies for SageMaker. Addresses resource provisioning, continuous training cycles, multi-account governance, and secure networking architectures required for heavy-duty financial and healthcare deployments.
• (2022) aws.amazon.com: Promote pipelines in a multi-environment setup using Amazon SageMaker Model Registry, HashiCorp Terraform, GitHub, and Jenkins CI/CD [HCL CONTENT] [ADVANCED LEVEL] 🌟🌟🌟🌟 [ENTERPRISE-STABLE] — An advanced infrastructure-as-code deployment blueprint for multi-environment (Dev, Stage, Prod) MLOps promotion. Orchestrates SageMaker Model Registry with HashiCorp Terraform, GitHub Actions, and Jenkins, demonstrating an enterprise-grade, highly-secured CI/CD model delivery loop.

Azure

Model Serving

• (2022) bea.stollnitz.com: Creating batch endpoints in Azure ML [PYTHON CONTENT] 🌟🌟🌟 [COMMUNITY-TOOL] — Demystifies Azure ML batch endpoint configurations, highlighting the differences between batch and low-latency real-time managed endpoints. Covers execution environments, partition configurations, storage connections, and scaling parameters needed to serve heavy computational batch datasets efficiently.
• (2021) youtube: Deploy Convolutional Neural Network (CNN) on Azure with Python | Deep Learning Deployment | MLOPS [PYTHON CONTENT] 🌟🌟🌟 [COMMUNITY-TOOL] — A practical video tutorial showing step-by-step preparation, containerization, and hosting of a Convolutional Neural Network (CNN) on Azure container endpoints. Demonstrates writing score scripts, declaring environment dependencies, and triggering predictions via REST APIs.

Flyte Managed

• (2024) Union Cloud [NONE CONTENT] [ADVANCED LEVEL] 🌟🌟🌟🌟 [ENTERPRISE-STABLE] — A managed enterprise platform powered by Flyte, designed to orchestrate complex machine learning and data engineering workloads. It delivers serverless operational abstraction, dynamic scaling, robust isolation structures, and unified lineage tracing across multi-cloud environments.

Data Engineering

Streaming

Kafka

• (2021) towardsdatascience.com: Schemafull streaming data processing in ML pipelines [PYTHON CONTENT] [ADVANCED LEVEL] 🌟🌟🌟 [COMMUNITY-TOOL] — Technical analysis of schema-driven streaming pipelines using Apache Kafka and Apache Avro in Python. Demonstrates how strict schema enforcement prevents downstream ML model ingestion errors. Crucial for designing real-time feature stores and maintaining strong structural contracts across distributed data microservices.

Deployment

Kubernetes Orchestration

• (2022) bodywork-ml/bodywork-core: Bodywork ⭐ 436 [PYTHON CONTENT] 🌟🌟 [COMMUNITY-TOOL] — Bodywork acts as a pipeline orchestrator and deployment tool focused on shipping machine learning systems directly into Kubernetes. While currently seeing low developer activity, it remains a valuable conceptual blueprint for running serverless, stateful, and batch-oriented ML pipelines.

Distributed Systems

Compute Engines

Ray

• (2026) ==Ray== [C++ CONTENT] [ADVANCED LEVEL] [DOCUMENTATION] 🌟🌟🌟🌟🌟 [DE FACTO STANDARD] — Ray is the premier distributed execution framework for scaling compute-heavy AI and Python workloads. It provides low-overhead, dynamic actor execution models, powering distributed training (Ray Train), hyperparameter tuning (Ray Tune), and model serving (Ray Serve) at enterprise scale.

Experiment Tracking

Visualization

• (2024) github.com/aimhubio/aim ⭐ 6154 [PYTHON CONTENT] 🌟🌟🌟🌟 [ENTERPRISE-STABLE] — Aim is an open-source, highly responsive experiment tracking and visualization dashboard for machine learning. It provides a robust query language and a user-friendly UI to compare thousands of metrics, hyperparameters, and logs across deep learning runs.

Generative AI

LLM Ops

AWS (1)

• (2023) towardsdatascience.com: Deploying LLM Apps to AWS, the Open-Source Self-Service Way [PYTHON CONTENT] [ADVANCED LEVEL] 🌟🌟🌟🌟 [ENTERPRISE-STABLE] — Presents a self-service architectural framework for deploying LLM microservices to AWS with open-source infrastructure-as-code tools. Outlines the provisioning of specialized GPU-backed instances, serverless scaling mechanics, and custom embedding cache deployments to balance performance with operating costs.

System Design

• (2023) ==huyenchip.com: Building LLM applications for production== [NONE CONTENT] [ADVANCED LEVEL] 🌟🌟🌟🌟🌟 [DE FACTO STANDARD] — A seminal, highly-cited framework for deploying Large Language Model applications in production environments. Addresses technical hurdles such as context window management, prompting reliability, latency optimization, cost-efficiency trade-offs, and structural output sanitization. Essential reading for modern generative AI architects.

Infrastructure

GPU Orchestration

Kubernetes Operators

• (2024) ==catalog.ngc.nvidia.com: NVIDIA GPU Operator - Helm chart 🌟🌟🌟== [GO CONTENT] [ADVANCED LEVEL] 🌟🌟🌟🌟🌟 [DE FACTO STANDARD] — The official NVIDIA GPU Operator Helm Chart coordinates all physical driver configurations, container engine runtimes, device plugins, and monitoring layers on Kubernetes. This is the industry-standard approach to automated provisioning of GPU compute capabilities across massive cloud and on-premise clusters.

Kubernetes

Architectural Patterns

• (2021) towardsdatascience.com: A Kubernetes architecture for machine learning web-application deployments [YAML CONTENT] [ADVANCED LEVEL] 🌟🌟🌟 [COMMUNITY-TOOL] — Outlines a highly resilient architectural blueprint for deploying machine learning models on Kubernetes. Discusses containerizing model APIs, managing resource limits, utilizing ingress controllers, and decoupling frontend services from computational inference backends. Offers concrete patterns for scaling web apps backed by heavy-weight deep learning payloads.

Artifact Registration

• (2023) artifacthub.io: mlflow-server [YAML CONTENT] [ADVANCED LEVEL] 🌟🌟🌟 [COMMUNITY-TOOL] — Official or community Helm Chart designed to bootstrap a highly available MLflow tracking and registry server inside a Kubernetes cluster. Streamlines configuring databases, AWS S3 / MinIO backend stores, and ingress mechanisms required for cloud-native model lifecycle management.

Component Engineering

• (2021) itnext.io: Building ML Componentes on Kubernetes [GO CONTENT] [ADVANCED LEVEL] 🌟🌟🌟 [COMMUNITY-TOOL] — A deep dive into structuring modular machine learning pipeline components inside a Kubernetes cluster. Focuses on orchestrating stateless compute workloads, defining clear volume interfaces, and managing persistent training artifacts. Highly relevant for architects planning custom infrastructure abstractions over vanilla K8s primitives.

Deployment Guides

• (2023) dev.to/pavanbelagatti: Deploy Any AI/ML Application On Kubernetes: A Step-by-Step Guide! [YAML CONTENT] 🌟🌟🌟 [COMMUNITY-TOOL] — Hands-on guide showing how to package, deploy, and scale diverse machine learning applications using Kubernetes manifests. Focuses on establishing proper ingress routing, service definitions, CPU/GPU resource constraints, and continuous monitoring sidecars within a native cluster environment.

Kubeflow

• (2026) ==kubeflow== [GO CONTENT] [ADVANCED LEVEL] [DOCUMENTATION] 🌟🌟🌟🌟🌟 [DE FACTO STANDARD] — Kubeflow is the leading cloud-native open-source MLOps suite designed to construct, deploy, and run modular machine learning workflows on Kubernetes clusters. Provides a comprehensive platform for managing Jupyter notebooks, workflow pipelines, and highly optimized inference deployments.
• (2021) infracloud.io: Machine Learning Orchestration on Kubernetes using Kubeflow [NONE CONTENT] 🌟🌟🌟 [COMMUNITY-TOOL] — Explores the practical orchestration of complex ML workflows using Kubeflow pipelines on Kubernetes. Outlines the underlying architecture, components (e.g., pipelines, notebook servers, metadata), and strategic advantages over non-containerized distributed ML setups.

Learning Roadmap

Courses

• (2023) freecodecamp.org: MLOps Course – Learn to Build Machine Learning Production Grade Projects [PYTHON CONTENT] 🌟🌟🌟🌟 [ENTERPRISE-STABLE] [GUIDE] — A complete, production-grade syllabus designed to guide engineers through model training, automated deployment strategies, CI/CD, and system monitoring. Emphasizes building production-ready architectures with clear structural boundaries rather than pure algorithmic modeling.

Machine Learning

Computer Vision

Instance Segmentation

• (2023) github.com/CASIA-IVA-Lab/FastSAM ⭐ 8364 [PYTHON CONTENT] [ADVANCED LEVEL] 🌟🌟🌟🌟 [ENTERPRISE-STABLE] — Fast SAM offers a highly optimized, CNN-based real-time alternative to Meta's Segment Anything Model. By sacrificing minimal accuracy, it reduces latency and computation footprints, which is critical for edge deployments and microservice image APIs.

Databases

In-database ML

• (2024) postgresml/postgresml 🌟 ⭐ 6800 [RUST CONTENT] [ADVANCED LEVEL] 🌟🌟🌟🌟 [ENTERPRISE-STABLE] — An extension that integrates machine learning directly inside PostgreSQL, written in Rust. It enables developers to train and run real-time inference using classic models or LLMs natively through SQL, entirely bypassing external ETL and API pipeline latency.

Document Analysis

OCR

• (2024) ==github.com/VikParuchuri/surya== ⭐ 20801 [PYTHON CONTENT] [ADVANCED LEVEL] 🌟🌟🌟🌟🌟 [DE FACTO STANDARD] [LEGACY] — Surya provides multi-lingual document OCR and accurate layout analysis powered by deep learning. It delivers high-fidelity reading and structuring of dense scientific papers, tables, and financial layouts, serving as a lighter, open substitute for legacy systems.

Information Retrieval

RAG Pipelines

• (2024) github.com/decodingml: Real-time news search engine using Upstash Kafka and Vector DB ⭐ 140 [PYTHON CONTENT] [ADVANCED LEVEL] 🌟🌟 [COMMUNITY-TOOL] — A hands-on repository and architecture blueprint illustrating a real-time semantic news search engine. Combining Upstash managed Kafka for event streaming with a Vector Database for indexing, it shows how to build low-latency RAG systems.

Large Language Models

Fine-tuning

• (2023) ==github.com/meta-llama/llama-recipes== ⭐ 18353 [PYTHON CONTENT] [ADVANCED LEVEL] 🌟🌟🌟🌟🌟 [DE FACTO STANDARD] — Meta's core repository for scaling LLM deployments. It offers highly robust templates for PEFT (Parameter-Efficient Fine-Tuning) such as LoRA, model quantization, and optimization strategies that enable low-latency inference setups inside microservices frameworks.

Model Life Cycle

AWS (2)

• (2021) towardsdatascience.com: From Dev to Deployment: An End to End Sentiment Classifier App with MLflow, SageMaker, and Streamlit [PYTHON CONTENT] 🌟🌟🌟 [COMMUNITY-TOOL] — Walks through a complete MLOps workflow deploying a sentiment classifier. Synthesizes MLflow for parameter and metric tracking, AWS SageMaker for hosting scalable compute environments, and Streamlit for rapid front-end application prototyping. Highlights the synergy of multi-cloud and hybrid-SaaS tooling.

Enterprise Patterns

• (2022) Machine Learning in Production. What does an end-to-end ML workflow look like in production? (transcript) 🌟🌟🌟 [NONE CONTENT] 🌟🌟🌟🌟 [ENTERPRISE-STABLE] — A granular transcript and architectural review of an end-to-end production ML pipeline workflow. Identifies key infrastructure challenges such as reliable data ingestion, continuous model validation, distributed caching, and metadata lineage tracking.

Model Serving (1)

API Development

FastAPI

• (2021) towardsdatascience.com: Deploying An ML Model With FastAPI — A Succinct Guide [PYTHON CONTENT] 🌟🌟🌟🌟 [ENTERPRISE-STABLE] — A step-by-step technical implementation guide utilizing FastAPI for low-latency ML model serving. Highlights the benefits of asynchronous request handling, built-in Pydantic data validation, and automated OpenAPI schema generation. Demonstrates how to package the application with Docker to establish a robust microservice baseline.
• (2021) towardsdatascience.com: Step-by-step Approach to Build Your Machine Learning API Using Fast API [PYTHON CONTENT] 🌟🌟🌟 [COMMUNITY-TOOL] — Practical guide outlining the architectural components needed to design an enterprise-ready FastAPI wrapper for pre-trained machine learning models. Highlights exception handling, asynchronous inference configurations, and the construction of deterministic, typed request/response contracts using Pydantic.

Architectural Patterns (1)

Infrastructure Selection

• (2024) axelmendoza.com: The Ultimate Guide To ML Model Deployment In 2024 [NONE CONTENT] 🌟🌟🌟🌟 [ENTERPRISE-STABLE] — Comprehensive blueprint detailing contemporary paradigms of ML serving, contrasting serverless, dedicated clusters (like K8s), and edge processing. Helps infrastructure architects navigate hardware acceleration, pipeline containerization, security policies, and real-time observability structures.

Kubernetes (1)

KServe

• (2022) thenewstack.io: KServe: A Robust and Extensible Cloud Native Model Server [GO CONTENT] [ADVANCED LEVEL] 🌟🌟🌟🌟 [ENTERPRISE-STABLE] — Comprehensive technical exploration of KServe (formerly KFServing) on Kubernetes. Covers dynamic autoscaling (scaling down to zero via Knative), standardized ingress protocols (v2 data plane), advanced traffic routing, model validation steps, and canary rollout orchestrations.

Microservices

• (2021) cloudblogs.microsoft.com: Simple steps to create scalable processes to deploy ML models as microservices [PYTHON CONTENT] 🌟🌟🌟 [COMMUNITY-TOOL] — Provides an enterprise-grade methodology for packaging machine learning models as distinct, containerized microservices. Focuses on automated CI/CD validation loops, lightweight interface design, and scalable deployment targets on Azure Kubernetes Service (AKS). Solves the organizational silo problem by treating the model as an isolated API.

Orchestration

Airflow

• (2021) towardsdatascience.com: Build Machine Learning Pipelines with Airflow and Mlflow: Reservation Cancellation Forecasting [PYTHON CONTENT] [ADVANCED LEVEL] 🌟🌟🌟 [COMMUNITY-TOOL] — Describes the convergence of Apache Airflow DAG orchestration with MLflow execution and model-tracking servers. Uses a concrete business use case (reservation forecasting) to illustrate scheduled dataset ingestion, remote training validation, and model artifact logging.

Flyte

• (2022) mlops.community: MLOps Simplified: orchestrating ML pipelines with infrastructure abstraction. Enabled by Flyte [NONE CONTENT] 🌟🌟🌟 [COMMUNITY-TOOL] — Highlights Flyte's capability to isolate infrastructure concerns from data science domains via clean abstract interfaces. Discusses how Flyte's structural containerized execution guarantees clean, versioned deployments without requiring data scientists to directly configure complex Kubernetes parameters.

Frameworks

• (2024) zenml.io: ZenML [PYTHON CONTENT] 🌟🌟🌟🌟 [ENTERPRISE-STABLE] — ZenML is an extensible MLOps pipeline framework designed to decouple data engineering and machine learning workflows from physical target infrastructure. It integrates with major cloud stacks and allows reproducible local executions to scale to production environments effortlessly.

Workflows

• (2024) ==github.com/Netflix/metaflow 🌟== ⭐ 10128 [PYTHON CONTENT] 🌟🌟🌟🌟🌟 [DE FACTO STANDARD] — Metaflow is Netflix's human-centric framework designed for building and managing production-grade data science pipelines. It seamlessly integrates local development with enterprise-scale cloud infrastructures, handling data caching, model versioning, and compute scaling automatically.

Workshops

Infrastructure (1)

• (2022) ML Platform Workshop ⭐ 445 [PYTHON CONTENT] 🌟🌟🌟🌟 [ENTERPRISE-STABLE] — A hands-on technical workshop repository showcasing the design of an end-to-end Machine Learning Platform. Demonstrates real-world integration of model registries, tracking servers, and deployment mechanisms under production-like conditions. Excellent educational resource for learning the architectural glue of modern MLOps frameworks.

---

💡 Explore Related: AI | AI Agents MCP | ChatGPT

🔗 See Also: About | Postman