Modelplane Modelplane docs
Version
main v0.3 Latest v0.2 v0.1

ModelDeployment Custom Resource

On this page
This document is for an unreleased version of Modelplane.

This document applies to the Modelplane main branch and not to the latest release v0.3.

Deploy a model to the fleet, from a single pod to disaggregated prefill and decode.

Concept guide: Deploy a Model →

#Metadata

API version
modelplane.ai/v1alpha1
Kind
ModelDeployment
Scope
Namespaced
Short names
md

#Example

Manifest
apiVersion: modelplane.ai/v1alpha1
kind: ModelDeployment
metadata:
  name: qwen3-8b
  namespace: ml-team
spec:
  replicas: 2
  template:
    spec:
      engines:
      - name: qwen3-8b
        members:
        - role: Standalone
          nodeSelector:
            devices:
            - name: gpu
              count: 1
              selectors:
              - cel: |
                  device.capacity["gpu.nvidia.com"].memory.compareTo(quantity("20Gi")) >= 0
          template:
            spec:
              containers:
              - name: engine
                image: vllm/vllm-openai:v0.23.0
                args:
                - --model=Qwen/Qwen3-8B

#Spec

# replicas required integer 1–10

How many ModelReplicas to fan out to. Each replica is a complete serving instance scheduled to one InferenceCluster.

# template required object

Template for the ModelReplicas this deployment fans out to. Everything but the replica count lives here, mirroring a Kubernetes Deployment’s spec.template.

# metadata optional object
# labels optional map[string]string

Labels to set on the ModelReplicas and ModelEndpoints this deployment composes, alongside the labels Modelplane manages. Use them to organize and select your own resources (kubectl get modelreplica -l tier=prod) or to route to a subset of endpoints from a ModelService. Keys under the modelplane.ai/ prefix are reserved.

# spec required object
# clusterSelector optional object

Optional label selector to filter InferenceClusters. If omitted, all ready clusters are candidates.

# matchLabels optional map[string]string
# engines required object[] 1–8 items
# copies optional integer 1–64 default: 1

How many identical copies of this engine to run per ModelReplica. A fixed number, sized once per deployment; scaling happens by adding ModelReplicas (spec.replicas), never by varying copies. Maps to the composed Deployment’s or LeaderWorkerSet’s replica count, or a Grove PodCliqueSet’s podCliqueScalingGroups[].replicas (its spec.replicas stays 1). Defaults to 1.

# members required EngineMember[] → 1–2 items
# name required string 1–63 chars

Identifies the engine within the deployment. Becomes part of the composed workload names, so it must be a DNS label.

# phase optional enum: Prefill | Decode

The engine’s phase in a PrefillDecode deployment, Prefill or Decode. Set only when serving.mode is PrefillDecode, where exactly one engine takes each phase.

# modelCacheRef optional object

Reference to a ModelCache in the same namespace. Optional for single-node engines; required for any engine that spans multiple nodes (a Leader with one or more Workers), since every pod in the gang mounts it.

# name required string ≥ 1 chars

ModelCache resource name in the same namespace.

# serving optional object

How the deployment is served from the cluster edge to its engines. Unified (the default) fronts the engines with a Service. PrefillDecode serves prefill and decode from the two engines marking those phases, with inference-aware routing that sequences prefill then decode. Omitted means Unified.

# mode required enum: Unified | PrefillDecode default: Unified

Unified serves prefill and decode on one engine. PrefillDecode splits them across two engines, each marking its phase as Prefill or Decode.

# tolerations optional object[]
# effect optional enum: NoSchedule | NoExecute

Taint effect to match. Empty matches all effects.

# key optional string

Taint key to match. Empty with operator Exists tolerates every taint.

# operator optional enum: Exists | Equal default: Equal

Exists matches any taint with the key (ignoring value); Equal matches key and value.

# value optional string

#Status

# replicas optional object
# ready optional integer
# total optional integer

#Subresources

# scale subresource

A scale subresource, so kubectl scale and replica-based autoscalers can drive it.

# specReplicasPath .spec.replicas

JSONPath to the desired replica count in spec.

# statusReplicasPath .status.replicas.total

JSONPath to the observed replica count in status.

#EngineMember object

One member of an engine’s gang — a role (Standalone, Leader, or Worker) with its hardware and pod template.

# nodeSelector optional NodeSelector →

The per-node device request for this member’s pods: what devices each pod needs from its node. The scheduler matches it against a candidate pool’s InferenceClass devices (surfaced on InferenceCluster status.gpuPools) and places the member on a pool that satisfies it, preferring one pool for the whole engine and splitting members across pools only when no pool satisfies them all. claim: DRA requests also become DeviceRequests in the ResourceClaim the member’s pods bind GPUs through. A GPU request’s count is the GPUs per node. Omitted, the member claims no devices and schedules onto its engine’s pool - a coordinator-only leader. At least one member per engine must carry a nodeSelector, and at least one member’s requests must resolve to a claimable (claim: DRA) device; an engine that matches only synthetic devices leaves its pods nothing to claim, so the scheduler treats such a pool as ineligible and the deployment reports InsufficientCapacity.

# role optional enum: Standalone | Leader | Worker default: Standalone

The member’s role in the engine. Standalone is a lone pod; a Leader coordinates and serves while its Workers join it. Defaults to Standalone.

# template required PodTemplate →

Pod template for this member’s engine pods. A curated subset of PodTemplateSpec.

# worker optional object

Settings for a Worker member. Valid only on a member whose role is Worker.

# nodes required integer 1–63

How many nodes this member spans - how big the engine is. Each node runs one worker pod, so the engine’s gang spans 1 (the Leader) plus this many worker pods. Defaults to 1.

#NodeSelector object

The per-node device request for this member’s pods: what devices each pod needs from its node. The scheduler matches it against a candidate pool’s InferenceClass devices (surfaced on InferenceCluster status.gpuPools) and places the member on a pool that satisfies it, preferring one pool for the whole engine and splitting members across pools only when no pool satisfies them all. claim: DRA requests also become DeviceRequests in the ResourceClaim the member’s pods bind GPUs through. A GPU request’s count is the GPUs per node. Omitted, the member claims no devices and schedules onto its engine’s pool - a coordinator-only leader. At least one member per engine must carry a nodeSelector, and at least one member’s requests must resolve to a claimable (claim: DRA) device; an engine that matches only synthetic devices leaves its pods nothing to claim, so the scheduler treats such a pool as ineligible and the deployment reports InsufficientCapacity.

# devices required object[] 1–16 items
# count optional integer 1–64 default: 1

How many matching devices a node must have. For a GPU request this is the per-node GPU count.

# name required string 1–63 chars

Name of this request. Mirrors a DRA DeviceRequest name; carried through to the ResourceClaim.

# selectors required object[] 1–8 items
# cel optional string 1–10240 chars

A DRA CEL expression evaluated against one device. Reads device.driver, device.attributes[""]. (typed), and device.capacity[""]. (a Quantity), with quantity() and semver() helpers, e.g. device.capacity[“gpu.nvidia.com”].memory.compareTo(quantity(“141Gi”)) >= 0.

#PodTemplate object

Pod template for this member’s engine pods. A curated subset of PodTemplateSpec.

# metadata optional object

Metadata applied to the member’s pods. Useful for labels and annotations that control cluster-level features like service mesh injection. Label keys under the modelplane.ai/ prefix are reserved.

# annotations optional map[string]string
# labels optional map[string]string
# spec optional object

Pod spec for this member’s engine pods.

# containers required object[] 1–1 items
# args optional string[]
# command optional string[]
# env optional object[]
# name required string
# value optional string
# valueFrom optional object
# configMapKeyRef optional object
# key required string
# name required string
# optional optional boolean
# fieldRef optional object

Reference a pod field via the downward API, e.g. status.podIP, metadata.name, or metadata.namespace.

# apiVersion optional string
# fieldPath required string
# secretKeyRef optional object
# key required string
# name required string
# optional optional boolean
# image required string ≥ 1 chars

Container image.

# name required string ≥ 1 chars

Container name. The container named “engine” is the inference engine.

# imagePullSecrets optional object[]
# name required string