VCF 9.1 for Production AI: An Architect’s Playbook

This is Part 7 of an architect’s read of VCF 9.1. Parts 1–6 covered the field guide, fleet and instance HLD/LLD, lifecycle, networking, security, and storage. This one is Private AI specifically, how to design production AI infrastructure on VCF 9.1.

AI is the workload class that defines the 9.1 release. Broadcom’s positioning, the analyst commentary, the engineering investment, and the customer conversations all centre on it. Broadcom’s preview of the Private Cloud Outlook 2026 report puts the data point on the table: 56% of surveyed organisations are running or planning to run production inference in a private cloud, while public cloud use for production inference dropped 15 percentage points year-on-year to 41%. 62% are very or extremely concerned about generative AI infrastructure costs. 36% report AI is driving new data protection, privacy, and security requirements.

That data isn’t why architects care. Architects care because AI workloads break the assumptions baked into every other workload class east-west traffic explodes, identity becomes ephemeral, GPUs cost six figures and need accountability, data sovereignty is suddenly architectural, and model weights become first-class assets that require their own lifecycle and recovery design.

VCF 9.1 responds to all of that, but the response is scattered across networking, security, storage, observability, and ecosystem partnerships. This piece pulls it together into one workload-class playbook.

Why AI changes the design conversation

From the architect’s seat, AI workloads bring five structural changes versus general-purpose VM workloads:

East-west traffic widens dramatically. Multi-host training and inference pipelines push GPU-to-GPU and GPU-to-storage flows that dwarf typical VM east-west traffic. Lateral security inspection that worked at 3 Gbps per host (the pre-9.1 IDS/IPS ceiling, covered in Part 5) becomes the bottleneck. IDPS Turbo Mode’s 9 Gbps per host is not a feature update it’s a precondition for inspecting AI traffic.

Identity moves between ephemeral and durable workloads. AI pipelines mix containerised pods (ephemeral, IP-rotating) with VM-hosted services (durable, long-lived). Pod-to-VM communication is a real attack path. Identity-based firewalling (Part 5) becomes mandatory IP-based rules don’t survive contact with Kubernetes.

GPUs are first-class capacity. A single GPU is a six-figure asset with a multi-year procurement lead time. Utilisation visibility, sharing strategy, and tenant accountability stop being nice-to-haves and become architectural inputs. Private AI Model and GPU Metrics in VCF Operations (new in 9.1) is the answer to “who’s using what.”

Data sovereignty becomes architectural. Public-cloud inference exposes data, prompts, and (sometimes) model weights to the provider. Production AI in regulated industries needs explicit data-residency, model-residency, and audit boundaries. Sovereign Cloud + Private AI is now a real HLD requirement, not a marketing slide.

Model weights are first-class assets. Recovery design now extends to model weights, embeddings, training datasets, and inference pipelines that didn’t exist as protected assets five years ago. Cyber recovery design (Part 1 ACC + IRE, with QLC ReadyNodes in Part 6) has to account for them.

Get those five shifts wrong in the HLD and the design fails at production scale, regardless of how many GPUs you bought.

The 9.x foundation for AI workloads

VCF 9.0 put in place the structural prerequisites for production AI:

• vSphere Supervisor as the unified control plane for VMs and Kubernetes (mixed-mode infrastructure)

• VKS (vSphere Kubernetes Service) for containerised AI workloads with NSX VPC integration

• VMware Private AI Foundation with NVIDIA a curated platform with Private AI Services for deploying AI workloads

• Deep Learning VM (DL VM) templates for accelerated AI workload provisioning

• NSX VPC-aware lateral security with vDefend microsegmentation

• VKS deployment in NSX VPCs for self-service AI infrastructure

• vSAN ESA as the storage substrate with cluster-wide deduplication

9.1 doesn’t replace any of that. It extends it with the specific capabilities that make AI workloads viable at production scale.

What 9.1 added for AI specifically

The 9.1 AI-relevant features I’d call out in any HLD review.

Enhanced DirectPath I/O for NVIDIA ConnectX-7 and BlueField-3. 9.1 adds support for NVIDIA ConnectX-7 NICs and BlueField-3 DPUs in NIC mode via Enhanced DirectPath I/O. This unlocks GPUDirect RDMA (direct GPU-to-GPU communication across hosts) and GPUDirect Storage (direct GPU-to-storage paths). The traffic that historically had to traverse CPU and host memory now bypasses both. For multi-host training and large-model inference, this is the difference between viable and non-viable on virtual infrastructure.

AMD Instinct MI350 GPU support. 9.1 adds support for the AMD Instinct MI350 series GPUs, which deliver a 4× generational AI compute increase over previous AMD GPUs. Combined with native PyTorch and vLLM support, this means AI workloads can move between accelerators (NVIDIA, AMD) with minimal code changes reducing the vendor lock-in risk that has been baked into AI infrastructure design for years.

DirectPath Enablement for GPUs. 9.1 adds DirectPath Enablement to VCF Private AI Services, giving high-performance, exclusive GPU access to a single VM. The VM can fully utilise GPU capabilities without sharing overhead. For workloads where GPU consolidation is not the priority (large-model training, latency-sensitive inference), this is the right consumption pattern.

Mixed compute infrastructure (AMD + Intel + NVIDIA). 9.1 supports CPU choice across AMD and Intel, and GPU choice across AMD and NVIDIA, with a consistent operations and security model across all of them. The HLD implication: vendor mix becomes a procurement and supply-chain decision, not a platform decision. Build a workload domain around what’s available.

Private AI Model and GPU Metrics in VCF Operations. New in 9.1. VCF Operations now exposes AI-specific metrics natively: time-to-first-token, token throughput, GPU utilisation, GPU memory pressure, model-level visibility on the same console as the rest of the infrastructure. MLOps teams get production AI telemetry without standing up a separate observability stack.

AI Metrics Observability Dashboard (Grafana-based). 9.1 ships a Grafana-deployable AI Metrics Observability Dashboard with detailed GPU metrics: utilisation, temperature, power usage, memory temperature, memory clock. Requires Grafana to be deployed alongside, but gives data scientists and SREs an out-of-the-box production AI dashboard.

Live Application Stack Blueprints. Multi-VM AI application topologies (e.g. a RAG pipeline with embedding service + vector DB + inference VM + RAG orchestrator) can be captured as reusable templates. Version them, redeploy them, eliminate the configuration drift that breaks AI pipelines between dev/test/prod. For AI workloads where the stack is rarely a single VM, this is structural.

Open ecosystem: PyTorch, vLLM, OPEA, Hugging Face. 9.1 provides native support for PyTorch and vLLM, integration with OPEA (Open Platform for Enterprise AI) for vetted RAG and GenAI blueprints, and a Hugging Face partnership delivering 1.8 million open-source models ready to deploy on AMD GPUs (NVIDIA path is via Private AI Foundation with NVIDIA). Customers choose the framework and the model; VCF runs them.

vMotion stun time on GPU VMs under 1 second. From the Private AI Foundation with NVIDIA 9.0.x notes, carried forward into 9.1: vMotion now stuns GPU-attached VMs for less than 1 second. This is the difference between maintenance windows that disrupt training jobs and maintenance windows that don’t. The HLD implication: GPU host lifecycle (covered in Part 3) no longer needs special handling for AI VMs.

Encrypted vMotion with Intel QuickAssist offload. 9.1 offloads encryption processing for encrypted vMotion to Intel QuickAssist Technology hardware, reducing CPU overhead by ~70%. For AI hosts where every CPU cycle matters, this restores secure-mobility-without-tax.

Centralised policy injection and data sovereignty controls. 9.1’s AI governance posture lets enterprises enforce centralised policy injection (e.g. content safety rules, prompt-injection mitigation, audit logging) and data sovereignty controls (e.g. model-access boundaries, residency enforcement) at the platform layer. For regulated AI deployments, this is the architectural answer to compliance.

40% lower AI server cost claim. Broadcom’s headline cost claim for VCF 9.1 AI infrastructure, driven largely by NVMe Memory Tiering reducing DRAM requirements per host (Part 1), Global Deduplication reducing storage capacity (Part 6), and Encrypted vMotion offload (above). Independent validation will come over time, but the architectural levers behind the claim are real.

The AI infrastructure topology (LLD)

A production AI deployment on VCF 9.1 splits cleanly into six planes.

Compute plane. GPU-equipped hosts in dedicated or shared workload domains. NVIDIA GPUs (H100, H200, Blackwell as supported) for the NVIDIA AI path. AMD Instinct MI300X/MI350 for the AMD AI path. Mixed-vendor estates supported. DirectPath Enablement for exclusive single-VM GPU access. vGPU for shared/divided GPU access. AMD vIOMMU for AMD GPU PCI passthrough.

Network plane. Enhanced DirectPath I/O for ConnectX-7 / BlueField-3 (NIC mode). RoCE (RDMA over Converged Ethernet) on the underlay for GPUDirect RDMA. EVPN-VXLAN interoperability with the physical fabric (Part 4) for predictable overlay/underlay behaviour. NSX VPC-based tenant isolation (Part 4) for multi-tenant AI workloads. MTU and jumbo frame considerations baked in.

Storage plane. vSAN ESA as the capacity tier for AI data pipelines (Part 6). NVMe Memory Tiering (Part 1) as the cost lever for memory-heavy AI workloads (large vector databases, in-memory embeddings). Native S3 Object Storage (tech preview, Part 6) as the model archive and dataset tier. GPUDirect Storage for direct GPU-to-storage I/O paths. Stretched namespaces for metro-scale AI training.

Security plane. vDefend with VKS CNI integration (Part 5) for pod-level lateral security across AI containers. IDPS Turbo Mode at 9 Gbps per host (Part 5) to inspect AI east-west traffic without becoming the bottleneck. Identity-based firewalling for ephemeral pod identity. Centralised policy injection for content safety and prompt-injection mitigation. ACC + IRE for ransomware recovery of model weights (Part 1).

Observability plane. Private AI Model and GPU Metrics in VCF Operations time-to-first-token, token throughput, GPU utilisation, memory pressure, model-level visibility. AI Metrics Observability Dashboard (Grafana) for detailed GPU telemetry. Standard VCF Operations for infrastructure-level visibility. PromQL support for custom AI metric queries.

Tenancy and governance plane. NSX Projects + VPCs (Part 4) for tenant isolation. VPC-aware lateral security (Part 5) for per-tenant policy. VCF Automation self-service surface for AI workload provisioning. Live Application Stack Blueprints for repeatable multi-VM deployments. Data sovereignty controls for regulated tenants.

HLD design patterns for production AI

Pattern 1: Single-tenant AI on dedicated infrastructure. Single project, single VPC, dedicated GPU workload domain. Direct GPU passthrough (DirectPath Enablement) for max performance. Suitable for centralised AI/ML teams with a single customer (the enterprise itself). Simplest to operate, but limited to one workload group at a time.

Pattern 2: Multi-tenant AI with shared GPU infrastructure. Multiple projects (one per team/business unit), each with its own VPC. Shared GPU pool consumed via vGPU profiles for fair sharing. vDefend VPC-aware lateral security for tenant isolation. Private AI Model and GPU Metrics for per-tenant chargeback. Common pattern in large enterprises with multiple AI teams and in service-provider deployments where multiple customers consume the same GPU estate.

Pattern 3: Sovereign AI for regulated workloads. Dedicated NSX Project with strict isolation. Dedicated SSP (Part 5) for security policy. Dedicated storage substrate with Data-at-Rest + Data-in-Transit Encryption (Part 6). Centralised policy injection for prompt-injection mitigation and content safety. Audit logging via VCF Operations. Suitable for financial services, government, defence, and healthcare AI workloads.

Pattern 4: Mixed-accelerator estate. AMD MI350 hosts and NVIDIA H100/H200 hosts coexisting in the same VCF instance, in distinct workload domains or distinct clusters within a domain. PyTorch / vLLM workloads portable across both. OPEA blueprints standardise the RAG and GenAI stack. Reduces vendor concentration risk and improves supply-chain resilience.

Pattern 5: Edge AI inference + central training. Central training cluster (dense GPU, full DirectPath, large vSAN ESA + Memory Tiering) feeds inference clusters distributed to VCF Edge sites (covered in Broadcom’s Edge 9.1 announcements). Models trained centrally, replicated outward, served at the edge. Fleet-level VCF Operations gives unified visibility across training and inference planes.

The AI security overlay (cross-reference Part 5)

Don’t re-architect security for AI use what VCF 9.1 already provides:

• vDefend DFW on every vNIC, enforced at the hypervisor (Part 5)

• VKS CNI integration extends DFW into Kubernetes pod-level for containerised AI workloads

• IDPS Turbo Mode at 9 Gbps per host inspects AI east-west traffic without bottlenecking

• VPC-aware lateral security isolates AI tenants from each other and from general workloads

• Identity-based firewalling solves the ephemeral-pod identity problem

• NDR maps detection across VM, container, and AI-specific assets

• ACC + IRE (Part 1) for ransomware recovery of model weights and training datasets

• Centralised policy injection for content-safety and prompt-injection mitigation

The AI security posture isn’t a separate design it’s the platform security posture (Part 5) applied to a higher-throughput, ephemeral-identity, model-weight-aware workload class.

AI storage tier strategy (cross-reference Part 6)

AI workloads use the same storage architecture as everything else in VCF 9.1, but the tiering decisions are more deliberate:

• DRAM active model weights and activations for the inference path

• NVMe Memory Tier (Part 1) large embeddings, vector indexes, cold model layers. The cost lever that makes 1– 2 TB AI hosts viable at scale

• vSAN ESA capacity active datasets, checkpoints, working sets. Global Dedup particularly effective on training data with repeated patterns

• Native S3 Object Storage (tech preview, Part 6) model archive, training dataset archive, inference logs. Included in VCF license, multi-tenant

• GPUDirect Storage direct GPU-to-storage path for training data, bypasses CPU and host memory

The tiering decision is per-pipeline: ingest → training → checkpointing → inference → archive each has its own tier preference. The HLD should make these explicit.

Open ecosystem and model lifecycle

VCF 9.1’s open ecosystem story matters in HLD terms because it changes lock-in risk:

• PyTorch and vLLM native support workloads move between accelerators with minimal code changes

• OPEA (Open Platform for Enterprise AI) vetted, industry-standard blueprints for RAG and generative AI. Reduces design risk on new AI patterns

• Hugging Face partnership 1.8M open-source models out-of-the-box on AMD GPUs. Combined with NVIDIA Private AI Foundation, gives enterprises a genuine model catalogue

• Live Application Stack Blueprints capture the multi-VM AI topology once, deploy repeatedly. Prevents the configuration drift that breaks AI pipelines between environments

• VCF Automation self-service application teams consume AI infrastructure as a service, with platform-defined guardrails

Architects who’ve seen what proprietary AI orchestration coupling looks like (and what unwinding it costs) will recognise this as a material design input.

Common HLD/LLD anti-patterns for AI workloads

Treating AI as a special-case workload domain. 9.1’s posture is that AI is a workload class on the same platform, with the same lifecycle, security, and observability. Don’t architect a separate stack for AI the duplication will hurt at Day-2.

Sizing GPUs without observability. A six-figure GPU asset with no utilisation visibility is a budget problem waiting to happen. Private AI Model and GPU Metrics + the AI Metrics Observability Dashboard need to be in the HLD from Day-0.

Ignoring east-west security throughput. If your AI traffic generates 7 Gbps of east-west flow per host and your IDS/IPS caps at 3 Gbps, you’re either dropping inspection or hitting performance ceilings. IDPS Turbo Mode (9 Gbps) is the answer; spec it.

Skipping GPUDirect. Enhanced DirectPath I/O with ConnectX-7 / BlueField-3 unlocks GPUDirect RDMA and GPUDirect Storage. Without it, multi-host training traffic goes through CPU and host memory. For training-class workloads, this is a design-critical decision, not a performance tune.

Vendor-locking on a single accelerator. Supply-chain pressure on GPUs means single-vendor designs carry procurement risk. PyTorch + vLLM + OPEA support across NVIDIA and AMD makes mixed-accelerator designs viable. At least evaluate it.

Forgetting model weights in the recovery design. Ransomware recovery (Part 1) historically focused on VM data. Model weights, training datasets, and embeddings are first-class assets now. ACC + IRE protection scopes need to explicitly cover them.

Self-hosting open-source models without governance. 1.8M Hugging Face models is a feature; 1.8M Hugging Face models with no vetting is a supply-chain risk. Centralised policy injection, model-access boundaries, and audit logging need to be enabled.

Treating Live Application Stack Blueprints as nice-to-have. AI pipelines drift between dev/test/prod faster than typical app stacks. Blueprints make environments reproducible. They’re structural for production AI, not optional.

Decision framework: where does this AI requirement live?

When an AI requirement arrives, work through the hierarchy:

• New AI workload from an existing team add to the team’s existing project/VPC, consume from the shared GPU pool

• New AI team add a custom Project with VPC, vDefend security profile, and Private AI Model + GPU Metrics scope

• New large-model training requirement ConnectX-7 / BlueField-3 hosts with Enhanced DirectPath I/O, GPUDirect RDMA on the underlay, vSAN ESA + GPUDirect Storage tier

• New inference workload vGPU-shared profile in the existing pool, NSX VPC, vDefend policy, Live App Stack Blueprint if multi-VM

• New regulated/sovereign AI workload dedicated Project + dedicated SSP + Data-at-Rest + Data-in-Transit encryption + centralised policy injection + audit logging

• New accelerator vendor add hosts to existing or new workload domain, port workloads via PyTorch/vLLM, validate via OPEA blueprint

• New AI archive requirement evaluate Native S3 Object Storage (tech preview) first, fall back to external if production-critical

• New AI recovery requirement extend ACC + IRE scope to cover model weights, training datasets, embeddings; QLC ReadyNodes (Part 6) for the IRE BoM

This sequence keeps the AI design grounded in the platform architecture rather than fragmenting into per-feature reasoning.

Closing

Production AI on VCF 9.1 is not a separate stack. It’s the platform applied to a workload class with five structural twists wider east-west traffic, ephemeral identity, GPU as first-class capacity, sovereignty as architecture, and model weights as protected assets. 9.1 responds with Enhanced DirectPath I/O for GPUDirect RDMA, AMD Instinct MI350 support, mixed-accelerator portability via PyTorch and vLLM, Private AI Model and GPU Metrics in VCF Operations, Live Application Stack Blueprints, vMotion stun under 1 second on GPU VMs, and the platform-level security and recovery overlay from Parts 5 and 1.

For my own design reviews, the AI questions I work through every time:

• Which workload classes training, inference, RAG, agentic and what’s the topology shape for each?

• Which accelerators NVIDIA, AMD, mixed and what’s the portability strategy?

• GPU consumption model DirectPath Enablement (exclusive) or vGPU (shared)?

• GPUDirect RDMA and GPUDirect Storage required for the workload, and if yes, ConnectX-7 / BlueField-3 specced?

• Tenancy model single-tenant, multi-tenant projects, or sovereign isolation?

• Storage tiering DRAM, NVMe Memory Tier, vSAN ESA, Native S3 mapped to ingest, training, checkpoint, inference, archive?

• Security overlay vDefend VKS CNI integration, IDPS Turbo, identity-based firewalling, centralised policy injection?

• Observability Private AI Model and GPU Metrics scope, Grafana AI Metrics Dashboard deployment, custom PromQL queries?

• Recovery ACC + IRE scope covering model weights, training datasets, embeddings, with QLC ReadyNodes for the IRE BoM?

• Lifecycle vMotion stun under 1 second confirms GPU VMs are first-class for maintenance windows?

Get those answered and the AI section of the HLD writes itself.

That closes Part 7 and with it, the architect’s read of VCF 9.1. Together with the Architect’s Field Guide (Part 1), the Fleet and Instance HLD/LLD (Part 2), the Lifecycle Deep Dive (Part 3), the Networking Deep Dive (Part 4), the Security Architecture Deep Dive (Part 5), and the Storage Architecture Deep Dive (Part 6), this is the seven-part read of VCF 9.1 I’d hand to a senior architect coming to the platform fresh for production AI.