VCF 9.1: Designing Fleet and Instance Topology From HLD to LLD

When customers ask me about VCF 9.1, the first question is usually about a feature Memory Tiering, ESX Live Patching, the new VKS scale ceiling. But the first design decision I actually make in a VCF 9.x engagement isn’t about any feature. It’s about how to draw the boundaries.

How many fleets? How many instances inside each fleet? Which domains live inside which instance? Which clusters inside which domains?

VCF 9.x rewrote this hierarchy from what it was in 5.x. In 5.x and earlier, the conversation went straight from “Cloud Foundation” to “Management Domain and Workload Domains,” and SDDC Manager held the whole thing together. In 9.x, that’s no longer how the model is shaped. The new hierarchy is:

VCF Private Cloud → VCF Fleet → VCF Instance → VCF Domain → vSphere Cluster

Each layer has its own design decisions. Each has a different blast radius. Each is documented in a different part of the HLD. If you skip the layer thinking, you end up with VCF deployed but operating like a collection of disconnected vCenters which is the failure mode the new model is designed to avoid.

This is a walkthrough of how I work through Fleet and Instance design in a VCF 9.1 engagement, from HLD-level decisions down to LLD-level details. I close with what’s actually new in 9.1 for fleet/instance design it’s more than you might think. The Management Services Cluster, Fleet Update Service, and the lifecycle architecture all changed.

The hierarchy you’re designing against

The official hierarchy in 9.x is five layers deep:

VCF Private Cloud the highest-level management and consumption boundary. Strategic, not operational. Can contain one or more fleets.

VCF Fleet managed by one set of fleet-level management components (notably VCF Operations and VCF Automation). Provides shared services across instances: SSO, certificates, licensing, lifecycle orchestration, observability. Contains one or more instances.

VCF Instance a discrete VCF deployment. Has its own SDDC Manager, its own management vCenter, its own management NSX. Contains one management domain and optionally one or more VI workload domains.

VCF Domain lifecycle and isolation boundary inside an instance. Two types: the management domain (mandatory, hosts infrastructure components) and VI workload domains (optional, host actual workloads).

vSphere Cluster where ESXi capacity lives. Clusters exist inside domains.

The mental model that works in design reviews:

• Fleet is the umbrella for shared services and governance

• Instance is a discrete VCF footprint with its own core management stack

• Domain is where lifecycle and workload isolation live

• Cluster is where you scale capacity

If you can answer “what’s the blast radius if X fails?” at each layer without hand-waving, your topology is in good shape. If you can’t, the HLD has gaps.

HLD: Fleet decisions

The first question is: how many fleets?

A VCF Fleet is the boundary of shared management services. Inside a fleet, you get:

• A common VCF Operations control plane (inventory, observability, lifecycle orchestration, capacity)

• A common VCF Automation control plane (self-service catalog, IaC pipelines)

• Single Sign-On across all instances and vCenters in the fleet backed by identity providers like Azure AD, Okta, Ping, ADFS, or OAuth 2.0

• Unified certificate management

• Unified licensing (one license file per VCF Operations instance covers the whole stack cores, vSAN TiB, advanced services)

• Unified password management

• Unified tag management across vCenter instances in the fleet

• Fleet-wide lifecycle, configuration drift detection, and health views

That’s a lot of consolidation. For most organisations, one fleet is the right answer.

Two-fleet (or more) designs only make sense when:

• You have hard regulatory or contractual boundaries that prevent shared identity, certificate, or licensing planes different legal entities, different customers, or a regulated/non-regulated split where shared SSO is impermissible

• You have geographic latency constraints that exceed fleet latency budgets (Broadcom publishes the VCF Fleet Latency Diagram on ports.broadcom.com keep that handy)

• You have administrative separation requirements that justify duplicating the fleet management stack

The default position should be one fleet, multiple instances rather than multiple fleets. Multiple fleets duplicates VCF Operations, VCF Automation, the Management Services Cluster, the Identity Broker, certificate management, and licensing. That’s real overhead, both in cost and in operational complexity.

A fleet can span a single region or a dual-region (active-active or active-passive) deployment. The fleet latency budget across regions is the constraint to verify before committing to the design.

HLD: Instance decisions

Inside the fleet, you draw instances. An instance is a discrete VCF deployment with its own SDDC Manager, its own management vCenter, and its own management NSX. Each instance gets exactly one management domain and zero or more VI workload domains.

The HLD question: when do you add another instance versus expanding an existing one?

Add an instance when one or more of these is true:

• You need a new region or site with its own management plane (the most common driver)

• You need a separate failure or lifecycle domain patch and upgrade can be staggered across instances

• You need administrative isolation between environments (production vs non-production, business unit A vs B) that’s stronger than what domain-level isolation gives you

• You need a different version posture (one instance running 9.1, another instance running 9.1.x patches more aggressively)

• You’re approaching the scale limits of the instance’s management components

Don’t add an instance just to:

• Get a new vCenter use a new workload domain instead

• Separate two app teams use a new workload domain, or separate clusters within a domain

• Run a slightly different cluster configuration vSAN ESA and OSA can coexist in different workload domains within one instance

In 9.1 specifically, there’s a new mandatory component to plan into every instance: the VCF Management Services Cluster. This is a containerized Kubernetes cluster that replaces the standalone Identity Broker appliance pattern from 9.0. It hosts vIDB (VMware Identity Broker) and is the foundation for further consolidation of management services onto containers rather than appliance VMs. From an HLD perspective:

• Each instance bring-up (greenfield) deploys a Management Services Cluster

• Upgrade to 9.1 from 9.0 deploys the cluster as part of the upgrade process

• Networking requirement: minimum 12 contiguous IP addresses for cluster worker nodes, plus 4 FQDNs for cluster services

• Operationally invisible day-to-day managed via VCF Operations and Lifecycle, not directly

This is a footprint change to factor into instance sizing.

HLD: Domain decisions

Within an instance, you have one mandatory management domain plus zero or more VI workload domains.

The management domain is dedicated to infrastructure components: SDDC Manager, fleet-level components (if this instance hosts them), the management vCenter, management NSX Manager nodes, NSX Edge clusters (optional), and the VCF Management Services Cluster. It does not host user workloads.

This is the first cluster you deploy when standing up an instance. It needs to be sized for the management components, not for tenant capacity. With the High Availability model, plan for:

• 3× NSX Manager nodes

• 3× VCF Operations nodes

• 3× VCF Automation nodes

• 3× VCF Operations for Logs nodes

• 3× VKS appliances (if VKS in scope)

• The new Management Services Cluster (9.1 onwards)

That’s a real footprint on the management domain cluster typically 4+ hosts to accommodate HA, anti-affinity, and capacity headroom.

VI workload domains are where user workloads run. Each has its own vCenter. NSX Manager can be dedicated per workload domain or shared between workload domains within an instance that’s a design decision driven by isolation requirements, scale, and your fault domain model.

Workload domain design patterns I commonly use:

Per-environment domains production, pre-production, development each as a workload domain. Clean lifecycle separation, clean blast radius.

Per-zone domains when network or security zoning is the dominant axis (DMZ vs internal vs PCI scope), domain boundaries align with zones.

Per-workload-class domains when traditional VMs, containerized workloads (VKS), and AI/HPC each warrant their own design pattern, separate workload domains keep the design clean. AI workloads in particular benefit from a dedicated workload domain given the host BoM is different (GPU passthrough, RoCE-capable networking).

In a typical large enterprise design, I’d expect to see 2–5 workload domains per instance. Beyond that, the operational overhead starts to outweigh the isolation benefits and another instance becomes the better answer.

LLD: Fleet-level component design

At the fleet level, the components you size and place are:

• VCF Operations sized for the total fleet inventory (objects, hosts, VMs). Plan for an HA deployment of 3 nodes minimum

• VCF Operations for Logs sized for log ingest volume. 3 nodes for HA

• VCF Automation sized for catalog complexity and concurrency. 3 nodes for HA

• Identity Broker (vIDB) now part of the Management Services Cluster in 9.1

• Optional VKS appliances 3 nodes for HA when VKS is in scope

Placement: fleet-level components live in the management domain of the first (primary) instance in the fleet. If the fleet spans multiple regions, fleet latency budgets become the design constraint for cross-region communication between fleet-level components and instance management planes. The VCF Fleet Latency Diagram from Broadcom is your reference.

Identity. Federate the Identity Broker against your enterprise IdP (typical: Azure AD, Okta, Ping, ADFS, or OAuth 2.0). Plan the role assignments at VCF level 9.1 brings VCF-level role assignments with integrated SSO, which means you can centralise authorization design instead of replicating it per vCenter.

Certificates. VCF Operations is the certificate management plane for the fleet. Integrate it with your enterprise CA (or Let’s Encrypt if appropriate). Automate certificate rotation. 9.1’s bulk certificate operations let you push certificate changes across all VCF components simultaneously the LLD should capture which CA, which trust chains, which rotation cadence.

Licensing. One license file per VCF Operations instance covers the entire VCF stack cores, vSAN TiB, Advanced Services. 9.1 adds connected-mode automation: license files refresh every 24 hours automatically. For disconnected environments, the offline registration process is now a 4-step flow register fleet, download verification, upload usage, download confirmation.

Backup. Fleet management components are critical. The LLD must include backup architecture for VCF Operations, VCF Automation, the Management Services Cluster, and SDDC Manager.

LLD: Instance-level designs

For each instance:

• SDDC Manager one per instance, in the management domain. Lifecycle is shifting in 9.x to Fleet Manager (inside VCF Operations), but SDDC Manager retains its role at the instance level for workload domain operations

• Management vCenter one per instance, in the management domain. Sized for the management workload, not tenant workload

• Management NSX 3 manager nodes for HA, in the management domain. Edges optional depending on north-south traffic patterns

• Management Services Cluster new in 9.1, mandatory, in the management domain. 12 contiguous IPs and 4 FQDNs needed

• Storage vSAN is the typical choice for the management domain. ReadyNodes simplify the BoM

• Network VLAN/IP planning for management traffic, vMotion, vSAN, NSX overlay, and the Management Services Cluster

LLD: Workload Domain designs

For each VI workload domain:

• vCenter one per workload domain. Sized for the workload domain’s host and VM counts. vCenter Quick Patch in 9.1 dramatically reduces patch downtime, so HA design assumptions can relax

• NSX Manager per-domain or shared with other workload domains in the same instance. Per-domain gives cleanest isolation; shared reduces footprint. Decide based on your isolation policy

• Storage vSAN ESA, NFS, FC, or iSCSI. 9.1 lifts NFS to 256 hosts per datastore, opening larger consolidated designs. vSAN ESA dedup now spans more cluster types and works on encrypted volumes re-baseline raw-to-logical capacity

• Cluster topology number of clusters per domain, fault domain design within clusters, stretched cluster patterns where applicable

• vSphere Lifecycle Manager image management is per-domain. 9.1’s Fleet Update Service adds parallel cluster upgrades, reducing maintenance windows on multi-cluster domains

What’s actually new in 9.1 for fleet and instance design

VCF 9.1 isn’t just a feature release. Several changes hit fleet and instance architecture directly.

VCF Management Services Cluster. New mandatory containerized Kubernetes cluster per instance. Hosts vIDB and (over time) more management services. Replaces standalone appliance VMs. Reduces sprawl long-term, but adds a new networking and FQDN footprint in 9.1.

Fleet Update Service. New lifecycle architecture inside the VCF Services Platform. Parallel cluster upgrades reduce maintenance windows. Improves scale and performance for upgrades and patching. Lifecycle is no longer optional tooling it’s the backbone.

HTTP Offline Depot Support. For environments without internet access to the standard Broadcom depot, 9.1 adds HTTP offline depot support for the VCF Installer and Fleet Depot Service. Cleaner LLD for disconnected and air-gapped deployments.

Connected-mode licensing automation. License files auto-refresh every 24 hours. No more 180-day manual acknowledgments. Worth mentioning in the operational runbook.

Bulk certificate operations. Push certificate operations across all components simultaneously. Hours-long certificate refresh exercises collapse to minutes.

VCF Operations new console structure. Aligned to the functional pillars of Build, Manage, Operate, and Protect. The LLD should reference the new IA when documenting where operators perform which tasks.

Real-time metrics and PromQL support. VCF Operations adds real-time metrics, improved dashboards, and PromQL support. Observability LLDs gain a more direct mapping to Prometheus-style queries.

VxRail capabilities in VCF Operations. For customers on the Dell VxRail path, key Day 0/1/2 capabilities previously handled by VxRail Manager can now be run via VCF Operations on vSAN ReadyNodes. Worth flagging in transition designs.

Updated VCF Fleet Design Blueprints and Fleet Latency Diagrams. Broadcom has refreshed both for 9.1. The blueprints span Edge, Fleet Management, Monitoring, Alerting, Troubleshooting, Workload Consumption, Lateral Security, Data Protection, DR, and Cyber Recovery. The latency diagrams are at ports.broadcom.com under Network Diagrams. Reference these in the HLD/LLD they’re the canonical Broadcom designs to work from.

Common HLD and LLD anti-patterns

A few patterns I see repeatedly in design reviews and have learned to flag early.

“Private cloud = one vCenter.” Leadership says “private cloud,” and someone interprets that as “one vCenter to manage it all.” In VCF 9.x, this is the wrong mental model. Translate “private cloud” into the platform objects: fleet, instances, domains, clusters. Pick the boundary that matches blast radius and lifecycle requirements.

Too many fleets. Splitting into multiple fleets without a hard regulatory or latency reason duplicates VCF Operations, VCF Automation, the Management Services Cluster, identity, certificates, licensing. That’s significant operational overhead. Default to one fleet, multiple instances.

Too many instances. Adding an instance for every new use case fragments the fleet and grows the management surface area. Use workload domains for use-case separation within an instance.

Skipping the latency check. The Fleet Latency Diagram from Broadcom is non-negotiable for multi-region designs. Verify the round-trip latency between fleet management components and instance management planes before committing to topology.

Treating SDDC Manager as the centre. In 9.x, the centre is VCF Operations and Fleet Manager. SDDC Manager has narrowed in scope. The HLD should reflect that.

Decision framework: where does this thing belong?

Ten times out of ten, when someone asks “where does X live in my VCF design,” it resolves to one of:

• New cluster when X is more capacity (more hosts, more VMs, same operational model)

• New workload domain when X needs a separate vCenter, lifecycle boundary, or NSX scope inside an existing instance

• New instance when X needs a separate SDDC Manager, separate management plane, or sits in a new region or hard-isolated security zone

• New fleet when X needs separate fleet-level services (identity, automation, observability) rare, usually regulatory or geographic

• New private cloud when X is a strategic or governance separation, not an operational one

If you can place every requirement against this hierarchy, your HLD is structurally sound.

Closing

VCF 9.1 reduces operational sprawl through containerized management services, fleet-wide lifecycle, parallel cluster upgrades, bulk certificate operations, and tighter observability. But none of that helps if the fleet, instance, and domain boundaries are drawn wrong.

For my HLD reviews, the first three questions are always:

• How many fleets, and why?

• How many instances per fleet, and what’s the boundary criterion?

• How many domains per instance, and which lifecycle and isolation requirements drive them?

Get those three right and the rest of the LLD has a stable scaffold to hang on. Get them wrong and you’ll fight the platform for years.

The Broadcom design blueprints and fleet latency diagrams are the right starting point. Use them. Reference them. Don’t reinvent the topology adapt the canonical patterns to the customer’s constraints.

That’s how I read VCF 9.1 for fleet and instance design.

Sources

• VCF 9.1 Updated VCF Design Blueprints & VCF Fleet Latency Diagrams (William Lam)

• VCF 9.1 What’s New: Management Services and Licensing (Gibson Virtualization)

• VCF 9.1 What’s New (vCollin)

• Scale, Simplify, and Secure Your Private Cloud Operations with VCF 9.1 (Broadcom)

• Planning a Successful VMware Cloud Foundation 9.0 Deployment (Broadcom)

• Operations in VMware Cloud Foundation 9.0 (Broadcom)

• Why VCF 9.0 Improves IT Operations and Management (Broadcom)

• VCF Fleet Management TechDocs (Broadcom)

• VCF 9.0 Mental Model Part 1: Fleets, Instances, Domains (Digital Thought Disruption)

• VCF 9.0 Mental Model Part 2: Fleet Services vs Instance Management Planes (Digital Thought Disruption)

• VMware Cloud Foundation 9.0.2 VCF 9.x Upgrade, Lifecycle & Fleet Manager Explained