KServe

Spec sheet

What it does

KServe is an open-source platform for serving generative and predictive AI models on Kubernetes.Read 5 Oct 2026 · kserve.github.io

Purpose

KServe is a Kubernetes platform for serving generative and predictive AI models across multiple frameworks.Read 5 Oct 2026 · kserve.github.io

Integrations

Documented integrations include Istio, Knative Serving, Envoy Gateway, Envoy AI Gateway, and the vLLM runtime.Read 5 Oct 2026 · kserve.github.io

Security

The Istio integration documents mutual TLS, automatic certificate management, and fine-grained access-control and authorization policies.Read 5 Oct 2026 · kserve.github.io

Support

Users can get help through the documentation, the #kserve Slack channel, GitHub Discussions, and community meetings.Read 5 Oct 2026 · kserve.github.io

Deployment API

KServe provides a Kubernetes custom resource definition that handles autoscaling, networking, health checks, and server configuration for model deployments.Read 5 Oct 2026 · kserve.github.io

Frameworks

Documented serving runtimes include TensorFlow Serving, Triton, Hugging Face, LightGBM, XGBoost, AutoGluon, PMML, scikit-learn, and PaddlePaddle.Read 5 Oct 2026 · kserve.github.io

Install options

KServe can be installed with Kustomize components or Helm charts, and can serve predictive models alone or alongside separate LLM and model-caching components.Read 5 Oct 2026 · kserve.github.io

Installation

KServe documents installation with Kustomize components or Helm charts and supports multiple deployment modes.Read 5 Oct 2026 · kserve.github.io

Known limitation

The optional LocalModel component currently supports InferenceService workloads; support for LLMInferenceService is planned for future releases.Read 5 Oct 2026 · kserve.github.io

LLM API

The Hugging Face runtime uses vLLM and provides OpenAI-compatible API endpoints for generative tasks including text generation, embeddings, and reranking.Read 5 Oct 2026 · kserve.github.io

LLM features

Its generative AI features include vLLM and llm-d backends, OpenAI protocol compatibility, GPU acceleration, model caching, KV cache offloading, and request-based autoscaling.Read 5 Oct 2026 · kserve.github.io

Predictive AI features

It supports TensorFlow, PyTorch, scikit-learn, XGBoost, and ONNX, with routing, InferenceGraph deployments, explainability, monitoring, and scale-to-zero.Read 5 Oct 2026 · kserve.github.io

Predictive model features

Predictive serving supports canary rollouts, inference pipelines and ensembles, scale-to-zero, explainability, and monitoring features such as payload logging and drift detection.Read 5 Oct 2026 · kserve.github.io

Project affiliation

The site says KServe has joined CNCF and identifies it as a Series of LF Projects, LLC project.Read 5 Oct 2026 · kserve.github.io

Protocols

KServe documents gRPC support for its V1 and V2 inference protocols, including binary tensor data and streaming responses.Read 5 Oct 2026 · kserve.github.io

Requirements and limits

KServe runs on Kubernetes; its optional LocalModel component currently supports InferenceService workloads, while support for LLMInferenceService is described as planned for a future release.Read 5 Oct 2026 · kserve.github.io

Rollouts

KServe supports canary rollouts and A/B testing.Read 5 Oct 2026 · kserve.github.io

Standard API

It provides a Kubernetes Custom Resource Definition that abstracts autoscaling, networking, health checks, and server configuration.Read 5 Oct 2026 · kserve.github.io

Who it is for

The administrator guide covers production management of both predictive and generative inference workloads on Kubernetes.Read 5 Oct 2026 · kserve.github.io

Where Specifiction places it

• Model Hosting#12 of 256.9