NVIDIA GPU Operator for Kubernetes

By Kokil Thapa | Last reviewed: September 2026

Running GPU workloads on Kubernetes without a unified install path leaves you patching drivers on every node, chasing mismatched container runtimes, and debugging opaque scheduling failures. The NVIDIA GPU Operator for Kubernetes packages that work into a single controller-driven flow: it detects GPU hardware, installs matching drivers, configures the NVIDIA Container Toolkit, and registers the device plugin so pods can request nvidia.com/gpu resources. If you already understand GPU scheduling on Kubernetes, the operator is the production layer that makes those scheduling rules usable on real hardware. This guide covers architecture, Helm installation, verification, and the failures I see most often on bare-metal and cloud GPU nodes.

What is the NVIDIA GPU Operator for Kubernetes and what does it install?

The operator follows the same pattern described in Kubernetes operators that extend the API. It watches custom resources and reconciles a stack of DaemonSets and controllers on GPU-capable nodes. You do not SSH into each worker to install drivers when the operator is configured correctly.

At a high level, the stack has four layers that must align before a pod receives a GPU:

• Host driver — kernel module matched to the GPU generation and host OS.
• Container runtime hook — NVIDIA Container Toolkit integration with containerd or Docker.
• Device plugin — exposes nvidia.com/gpu to the kubelet and scheduler.
• GPU Feature Discovery (GFD) — labels nodes with GPU model, memory, and MIG profile data.

The primary custom resource is ClusterPolicy. It defines driver version, toolkit settings, MIG strategy, and whether the operator manages the driver or only the upper stack. Optional components include the DCGM exporter for metrics, the GPU validator for health checks, and the vGPU manager when you run virtual GPU profiles.

For teams building inference pipelines, this stack connects directly to patterns in serving ML models with GPU on Kubernetes and tools like KServe model serving on Kubernetes. The operator does not replace your ML framework. It makes the node ready so PyTorch, TensorRT, or CUDA base images start without custom init containers on every deployment.

Core components at a glance

The operator expects a supported Linux host OS and a container runtime the toolkit can patch. On bare metal, I treat GPU nodes like any other specialised worker: dedicated taints, separate node pools, and documented upgrade windows. That mirrors how I handle specialised workers during Linux system administration engagements where uptime matters more than bleeding-edge driver versions.

How do you install the NVIDIA GPU Operator on a Kubernetes cluster?

Helm is the supported install path for most clusters in 2026. The operator ships from NVIDIA's chart repository and targets Kubernetes 1.27+ on supported distributions including upstream Kubernetes, OpenShift, Rancher, and major cloud managed offerings with GPU instance types.

Prerequisites checklist

1. GPU hardware visible on the host (lspci | grep -i nvidia on bare metal).
2. Container runtime is containerd or CRI-O with a standard config path.
3. Node OS matches the operator's supported matrix (Ubuntu 22.04/24.04 is common).
4. Helm 3.x installed on your admin workstation.
5. NVIDIA GPU nodes labelled or tainted so only GPU workloads land there.

If you run containerd without the toolkit, review NVIDIA Container Toolkit for Docker and GPU containers first. The operator installs the toolkit for you, but understanding the runtime hook helps when debugging mount or library path errors.

Helm install commands

helm repo add nvidia https://helm.ngc.nvidia.com/nvidia
helm repo update

kubectl create namespace gpu-operator

helm install gpu-operator nvidia/gpu-operator \
  --namespace gpu-operator \
  --set driver.enabled=true \
  --set toolkit.enabled=true \
  --set devicePlugin.enabled=true \
  --set gfd.enabled=true \
  --wait

For OpenShift, use the certified operator bundle from OperatorHub instead of raw Helm. The reconciliation model is the same, but RBAC and SecurityContextConstraints differ.

Common Helm values for production

Pin the driver version when you need reproducible clusters across regions. Pre-installed drivers on cloud images may require driver.enabled=false so the operator only manages toolkit, plugin, and GFD.

helm upgrade gpu-operator nvidia/gpu-operator \
  --namespace gpu-operator \
  --reuse-values \
  --set driver.version="580.65.06" \
  --set driver.kernelModuleType=open \
  --set mig.strategy=mixed \
  --set dcgmExporter.enabled=true \
  --set node-feature-discovery.enableNodeFeatureApi=true

Label GPU nodes before or after install. A typical pattern uses nvidia.com/gpu.present=true applied by GFD, plus a custom taint:

kubectl taint nodes gpu-node-01 nvidia.com/gpu=present:NoSchedule

Pair that taint with tolerations on GPU workloads. See taints and tolerations in Kubernetes for the scheduling mechanics. Without taints, CPU-only pods may land on expensive GPU nodes and waste capacity.

For local testing before production, Minikube vs Kind for local Kubernetes compares options. Neither fully replaces a real GPU node, but Kind with GPU passthrough helps validate manifests cheaply.

How does the NVIDIA GPU Operator differ from manual GPU driver setup?

Manual setup means installing drivers with apt or runfile, configuring containerd by hand, deploying the device plugin manifest, and repeating the process on every new node and every kernel upgrade. The operator centralises that into declarative config with rolling reconciliation.

Cloud providers often ship GPU-optimised AMIs with drivers pre-installed. In that case, disable operator-managed drivers and let the operator handle only toolkit, plugin, and GFD. Mixing a host driver installed outside the operator with an operator driver DaemonSet on the same node causes failures that look like random CUDA version mismatches.

The official NVIDIA GPU Operator documentation lists supported platforms and version matrices. Cross-check against your Kubernetes minor version before upgrading production clusters provisioned with Kubespray for Kubernetes deployment.

How do you verify GPU pods work after deploying the operator?

Verification has three layers: operator health, node capacity, and a running CUDA workload. Skip any layer and you will misdiagnose scheduling versus runtime problems.

Step 1: Confirm operator pods are ready

kubectl get pods -n gpu-operator
kubectl get clusterpolicy cluster-policy -o yaml

All DaemonSets on GPU nodes should show desired equals ready. Driver pods run privileged and may take several minutes on first boot while modules compile.

Step 2: Check node GPU capacity

kubectl describe node gpu-node-01 | grep -A5 Capacity
kubectl get nodes -L nvidia.com/gpu.product,nvidia.com/cuda.driver.version

You should see allocatable nvidia.com/gpu matching physical GPUs or MIG slices. The Kubernetes scheduler uses that allocatable value when binding pods. Wrong counts usually mean the device plugin failed registration.

Step 3: Run a CUDA smoke test pod

cat <<'EOF' | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
  name: cuda-vectoradd-test
spec:
  restartPolicy: OnFailure
  tolerations:
  - key: nvidia.com/gpu
    operator: Exists
    effect: NoSchedule
  containers:
  - name: cuda-vectoradd
    image: nvcr.io/nvidia/k8s/cuda-sample:vectoradd-cuda12.5.0
    resources:
      limits:
        nvidia.com/gpu: 1
EOF

kubectl logs cuda-vectoradd-test
kubectl delete pod cuda-vectoradd-test

Successful logs show vector addition completed on the GPU. If the pod stays Pending, inspect events for insufficient nvidia.com/gpu. If it CrashLoopBackOff, read debugging CrashLoopBackOff in Kubernetes and check driver pod logs first.

Set explicit resources.limits.nvidia.com/gpu on every GPU container. Requests should match limits for extended resources. Read Kubernetes resource limits and requests if your pods get OOM-killed despite free GPU capacity—CPU and memory starvation still happens on GPU nodes.

Export the resulting pod spec or node labels through a JSON formatter when you paste diagnostics into tickets. Clean JSON saves time when you hand issues to platform or hardware teams.

What are common NVIDIA GPU Operator failures and how do you fix them?

Most failures fall into driver conflicts, runtime misconfiguration, or scheduling mismatches. The operator surfaces them through pod events and ClusterPolicy status conditions rather than silent partial installs.

Driver already installed on the host

Symptoms include driver DaemonSet crash loops and messages about existing NVIDIA modules. Fix by either removing the host driver and letting the operator manage it, or setting driver.enabled=false in Helm values. Never run both.

Container runtime not patched

Pods start but fail with libcuda.so or unknown device errors. Check toolkit DaemonSet logs and confirm containerd was restarted after the hook was applied. The NVIDIA Container Toolkit install guide explains the expected runtime class and config snippets the operator generates automatically.

Device plugin not registering

Nodes show zero allocatable GPUs. Verify the device plugin pod is running on that node and kubelet logs contain no plugin registration errors. The upstream Kubernetes device plugin docs describe the registration handshake the NVIDIA plugin must complete.

MIG profile mismatches

When MIG is enabled, resource names change to sliced profiles like nvidia.com/mig-1g.5gb. Your pod limits must reference the exact extended resource the plugin exposes. Mixed MIG strategy on heterogeneous clusters needs node selectors on GFD labels.

Upgrade and rollback strategy

Upgrade the operator chart during a maintenance window. Driver upgrades unload modules and restart GPU workloads on that node. Cordone the node, drain GPU jobs, upgrade, validate with the CUDA sample pod, then uncordon. For cluster-wide issues, Helm rollback is faster than manual driver cleanup on each host.

Broader diagnostic patterns live in Kubernetes troubleshooting field guide and Kubernetes performance tuning. GPU-specific latency often traces to PCIe bandwidth, power limits, or sharing too many pods per GPU—not operator bugs.

Teams building ML platforms on top of this stack often pair the operator with Kubeflow ML pipelines on Kubernetes and GitOps controllers. The operator handles node readiness; pipelines handle experiment lifecycle. That separation keeps infrastructure upgrades from blocking data science releases.

If your organisation needs GPU-backed inference integrated into a business application rather than raw cluster ops, see how AI integration and automation services approach production API design. GPU clusters are expensive. Most SMB teams in Nepal and abroad should validate workload fit before committing to dedicated hardware at Rs 400,000–800,000 per GPU server (~USD 3,000–6,000).

For reference architectures on production clusters I've helped maintain, browse the project portfolio including platforms like Adventure Third Pole Trek where reliable background job processing mattered as much as raw compute. GPU nodes add similar operational discipline: backups, monitoring, and upgrade playbooks before you scale.

Key Takeaways

• Install the NVIDIA GPU Operator for Kubernetes with Helm and a ClusterPolicy that matches your driver strategy—managed or pre-installed.
• Taint GPU nodes and add tolerations so only GPU workloads consume expensive accelerators.
• Verify with node allocatable counts, GFD labels, and a CUDA sample pod before promoting ML workloads.
• Disable operator driver management when cloud AMIs already ship a matching driver to avoid module conflicts.
• Pin driver versions in Helm values for reproducible multi-region clusters and simpler rollback.
• Pair the operator with DCGM metrics and documented drain-and-upgrade procedures for production GPU nodes.

People Also Ask

Does the NVIDIA GPU Operator work with containerd?

Yes. Containerd is the default target for most upstream Kubernetes clusters in 2026. The operator deploys a toolkit DaemonSet that patches containerd configuration and sets the NVIDIA runtime handler. CRI-O is also supported on OpenShift and some security-hardened distributions.

Can you use the GPU Operator without Helm?

NVIDIA publishes static manifests and OpenShift Operator Lifecycle Manager bundles. Helm remains the most common path for vanilla Kubernetes because values files are easy to store in Git and diff during upgrades.

How many GPUs can one Kubernetes node expose?

The device plugin exposes one extended resource unit per physical GPU or per configured MIG slice. A node with four A100 GPUs shows four nvidia.com/gpu allocatable units unless MIG splits them into smaller advertised resources.

Is the GPU Operator required for Kubernetes GPU workloads?

No. You can manually install drivers, the container toolkit, and the device plugin. The operator is recommended when you operate multiple GPU nodes and want declarative, version-controlled reconciliation instead of repeated SSH bootstrap.

Run GPU Workloads With Confidence

The NVIDIA GPU Operator for Kubernetes turns GPU node bootstrap from a fragile manual checklist into declarative infrastructure. Start with a pinned Helm release, validate nodes with a CUDA smoke test, and document your taint and upgrade playbook before production ML traffic lands. When you need help connecting GPU-backed services to a Laravel API, eCommerce pipeline, or internal automation workflow, contact the team or explore related guides on the blog and the homepage. Solid GPU infrastructure pays off only when applications actually use it—plan the software path alongside the operator install.