> ## Content Index > Fetch the complete content index at: https://www.mohammadsiddiqui.com/llms.txt > Use this file to discover other available public pages before exploring further. # The Engineer's Companion: Topology, Network, Storage and Integrations at Each Stage - URL: https://www.mohammadsiddiqui.com/the-engineers-companion-topology-network-storage-and-integrations-at-each-stage/ - Published: 2026-09-19T04:03:47.000Z - Updated: 2026-09-19T04:12:02.000Z - Description: The data plane barely changes across all four stages. The management plane triples. That is the real cost curve of adoption, and it belongs in front of an operations lead early. - Author: Mohammad Siddiqui - Tags: architecture, operations, lifecycle The staged path from vSphere 8 to VCF 9.1 is a sequencing decision. This is the engineering detail underneath it: what changes in the topology at each stage, what the network and storage have to provide, what the management plane looks like, and where third party integrations and APIs need attention. It is written to be read alongside a design document rather than instead of one. Every version and sizing figure needs verifying against current documentation before it goes in a plan. START HERE IF YOU HAVE NOT This assumes the staged approach is already agreed. If you are still deciding whether to stage the move at all, or need the argument for a sponsor or a change board, [The Staged Path From vSphere 8 to VCF 9.1](https://www.mohammadsiddiqui.com/the-staged-path-from-vsphere-8-to-vcf-9-1/) covers the paths, the stages and what it costs to pause between them. Same five stages. That one is the plan, this is the build. ## Logical topology, stage by stage LOGICAL TOPOLOGY, AND WHAT EACH STAGE ADDS STAGE 1 AND 2 MANAGEMENT PLANE vCenter 9.1 VCF Operations (Aria Ops 8.x upgrades here) VCF License Server (headless) DATA PLANE ESX 9.1 clusters, image-managed vSAN 9.1 or external storage Native vSphere lifecycle. Per-cluster vLCM. STAGE 3 ADDS VCF Management Services runtime Fleet Lifecycle SDDC Lifecycle Software depot Log management 9.1.x Identity Broker Fleet-wide patching replaces per-cluster vLCM. STAGE 4 ADDS NSX Manager cluster, Edge nodes VCF Automation (orgs, projects) Supervisor and VKS Avi Load Balancer Data Services Manager Workload domains Tenancy model becomes a design artefact. What each stage adds to the management plane. The data plane changes least. The thing worth noticing is that the data plane barely changes across all four stages. Hosts, clusters and storage look much the same at Stage 4 as they did at Stage 2\. What grows is the management plane, and with it the number of appliances, IP addresses, certificates and integration points your team has to operate. That is the real cost curve of adoption, and it is worth putting in front of an operations lead early. ## Network requirements NETWORK REQUIREMENTS, CUMULATIVE BY STAGE STAGE 1 and 2Management VLAN reachable from every host and applianceDNS forward and reverse for all appliances. License Server needs A and PTRFQDNs strictly lowercase. A single capital breaks Operations integrationNTP from a common source. Certificate and token validation depend on itSyslog moves to TLS on 1514\. Port 514 unencrypted is blocked in 9.xvMotion network sized for cluster-by-cluster host evacuationSTAGE 3 addsAdditional management IPs for the Management Services runtime appliancesSoftware depot reachability from every component being patchedLog ingest path sized for fleet-wide volume, not per-clusterIdentity Broker to directory reachability, plus SSO endpoint publicationSTAGE 4 addsOverlay MTU end to end. 1600 minimum for GENEVE, 1700 recommended with headroomTEP VLAN and IP pool sized for every host plus Edge nodesEdge uplink VLANs and BGP peering to the physical fabricAvi service engine data and management networksVPC address planning, and EVPN L3 VNI mapping if integrating a physical EVPN fabric Cumulative. Each stage keeps everything from the stages before it. ## Engineering detail by stage *Expand any stage for the topology, network, storage, management and integration detail.* Stage 0: Discover Physical topology to capture - Host count per cluster, model, CPU sockets and cores per socket. Core counts drive licensing, which requires a minimum of 16 cores per CPU. - Boot device type and capacity per host. Anything under 128 GB, and anything on SD or USB, is a remediation item. - NIC count, speed and current uplink assignment. Stage 4 will want more. - Memory per host against the 8 GB minimum and 12 GB production guidance. - Server model and firmware against the Broadcom Compatibility Guide. Status can move over a hardware lifetime, so check current rather than trusting the procurement record. Logical topology to capture - vCenter inventory, including whether Enhanced Linked Mode is in use and how many vCenters are in the SSO domain. - Cluster layout, DRS and HA configuration, EVC baseline per cluster. - Whether clusters are baseline-managed or image-managed. This determines Live Patch eligibility later. - Datastore inventory, type and version. VMFS version, vSAN OSA or ESA, external array presentation. - Current Aria Operations deployment: node count, sizing, adapter inventory, custom content. Network to verify - DNS forward and reverse resolution for every planned appliance FQDN. Reverse is the one people miss. - Confirm all FQDNs are lowercase. Mixed case breaks VCF Operations integration. - NTP reachability and consistency across hosts and appliances. - Current syslog configuration. If anything targets port 514 unencrypted, it needs moving to TLS 1514. - Available management IP addresses for new appliances. Count them before you need them. Integrations and APIs to inventory - Backup product version and its support statement for vSphere 9.1 and vSAN 9.1\. Check the vendor matrix, not the Broadcom one. - Monitoring agents and any tooling calling the vCenter API directly. Note the API version in use. - ITSM integration, CMDB discovery, and anything doing automated inventory collection. - Automation and IaC: Terraform provider versions, PowerCLI scripts, Ansible collections, custom REST clients. - Certificate authority integration and how certificates are currently issued and renewed. Stage 1: Comply Physical - No physical change required beyond remediating boot devices found in Stage 0. - Confirm hosts remain on the BCG for the target ESX version before upgrading. Logical topology added - VCF Operations appliance or cluster, replacing the Aria Operations deployment in place. - VCF License Server, a headless appliance with no UI of its own. - vCenter moves to 9.1 with its existing inventory intact. Enhanced Linked Mode topology is preserved but should be validated after upgrade. Network - License Server requires both an A record and a PTR record. Forward-only resolution causes licence assignment failures that present as vCenter instances not connected to a license server. - Management network reachability from VCF Operations to every vCenter and host it will collect from. - Outbound reachability to the Broadcom Business Services console for licence file download, or a documented offline process. - vMotion network capacity. Host evacuation during cluster upgrades is the heaviest sustained load this network will see. - Syslog re-pointed to TLS 1514 before the upgrade, not after. This fails silently rather than loudly. Storage - Datastore capacity and IOPS headroom for the new appliances. Size against the deployment guide for your estate scale rather than the minimum. - vSAN upgrades in place with the ESX upgrade. On-disk format upgrade is a separate consideration with its own rolling operation. - If on vSAN OSA, decide whether an ESA migration is in scope now or later. It is a rebuild, not an upgrade. - External arrays: confirm the array firmware and multipathing configuration against the vSphere 9.1 HCL. Management - Licence files are created on the Business Services console, uploaded into VCF Operations, and assigned to vCenter from there. - Certificate lifecycle: confirm which CA issues certificates and whether auto-renewal is configured. Auto-renewal fires 60 days before expiry and cannot be enabled for a certificate already inside that window. - Identity: vCenter SSO carries forward. Identity Broker is a Stage 3 consideration unless you have a specific reason to bring it forward. Integrations and APIs - Backup: re-validate VADP integration after the vCenter upgrade. Test a restore, not just a backup. - Anything calling the vCenter REST or SOAP API should be tested against 9.1\. Deprecations exist between major versions. - PowerCLI, Terraform provider and Ansible collections all need version alignment with 9.1. - VCF Operations exposes a REST API for metrics, alerts and inventory. If you are exporting to a data lake or an external dashboard, this is the integration point. - SIEM: confirm log flow resumed after the syslog change. A silent logging gap is the failure mode. Stage 2: Stabilise Logical - Move any remaining baseline-managed clusters to image-managed. This is what unlocks Live Patch, and it costs little at this point. - VM hardware version and VMware Tools upgrades, scheduled with application owners. Management - Establish patch cadence for the 9.1 stream and write it into change policy, with layer-specific change categories rather than one blanket approval weight. - Assign an owner for BOM and release note review. These are living documents in 9.x. - Confirm continuous compliance evidence can be produced rather than assembled at audit time. Integrations - Re-baseline monitoring thresholds. Metrics shift after an upgrade and stale thresholds generate noise that trains people to ignore alerts. - Confirm CMDB discovery is picking up the new versions correctly. - Validate that any automation still functions against the upgraded API surface, in a non-production context first. Stage 3: Operate Logical topology added - VCF Management Services runtime, hosting Fleet Lifecycle, SDDC Lifecycle and supporting services. - Software depot, which holds the binaries every other component patches from. - Log management 9.1.x, which depends on Management Services being present. - Identity Broker, if you are consolidating SSO across the fleet. Network - Additional management IP addresses for the runtime appliances. Plan the range rather than allocating ad hoc. - Software depot must be reachable from every component that will patch from it. During a depot patch, other component patches are blocked because the binaries are unavailable. - Log ingest path sized for fleet-wide volume. Per-cluster estimates understate this considerably. - Identity Broker to directory reachability, plus wherever the SSO endpoint is published to users. Storage - Software depot capacity for the binary repository. This grows with each release you retain. - Log retention sizing. Decide the retention period before deployment, because it drives capacity and is awkward to change later. - Additional appliance datastore capacity. Management - Patch ordering becomes a real constraint. Fleet lifecycle patches first, before any other component. Identity Broker and Salt RaaS each need their host runtime patched before them. - Per-cluster vLCM operations are replaced by fleet-wide lifecycle. The operational runbooks change, and so does who runs them. - Management packs: adapter instances need credentials, collection intervals and target definitions. Application and database packs require VCF or VCF Edge rather than vSphere Foundation. Integrations and APIs - Convert legacy Log Insight content packs. Management packs now carry both metric and log content in one artefact. - ServiceNow integration via the management pack, if ITSM is in scope. This is what turns an alert into an assigned incident. - Salt for configuration management and drift remediation, if adopting continuous compliance. - Prometheus Management Pack Builder and the PromQL API, for teams with existing Prometheus tooling. VCF or VCF Edge only. - VCF Operations API for exporting to external analytics or AI pipelines. Stage 4: Adopt Physical topology - Edge node placement. Dedicated Edge cluster or collapsed with management, decided on scale and north-south throughput requirements. - Additional NIC capacity for overlay traffic on every host participating in NSX. - If GPU workloads are in scope, accelerator placement, power and cooling, and the fabric to support them. Logical topology added - NSX Manager cluster, Edge nodes, Tier-0 and Tier-1 gateway design. - Supervisor enablement per cluster, and VKS cluster classes. - VCF Automation with its organisation and project hierarchy. This is the tenancy model and it is expensive to change once tenants depend on it. - Avi Load Balancer controllers and service engine groups. - Workload domain layout, if separating management from workload domains. Network - Overlay MTU end to end. 1600 is the minimum for GENEVE, 1700 is the sensible target because it leaves headroom for future encapsulation changes. Every switch, router and firewall in the path needs it. - TEP VLAN and IP pool sized for every host plus every Edge node. Undersizing this is a common and annoying mistake. - Edge uplink VLANs and BGP peering to the physical fabric, including AS numbers and prefix filtering agreed with the network team. - Avi service engine data and management network placement. - VPC address planning. If integrating with a physical EVPN fabric, L3 VNI mapping per VRF, and note that L2 MAC advertisement is not supported in the current NSX integration. Storage - Storage policies per tenant class, since policy-driven storage is how self-service allocates capacity. - vSAN capacity planning against quota commitments rather than current consumption. - Check storage efficiency feature support against your topology. Global deduplication carries exclusions including stretched clusters and two-node configurations. - If Data Services Manager is in scope, capacity and performance for database workloads specifically. Management - The provider and consumer split becomes an operating model decision, not just a technical one. Who curates the catalogue, who owns quotas, who responds when a self-service workload fails. - Quota tiers and infrastructure policy per organisation and project. - Content catalogue: VM templates, VKS cluster classes, database blueprints. - Agent identity if agentic workloads are in scope, which is a new identity class your IAM has no model for. Integrations and APIs - VCF Automation API for infrastructure as code, and the Git repository structure that feeds it. - GitOps via the Argo CD based service, noting it is Tech Preview and scoped to one instance per hosting namespace within a single region. - Kubernetes API surface for application teams, and how they authenticate to it. - Third party CNI, CSI and ingress choices for VKS, validated against the supported matrix. - Backup for Kubernetes workloads, which is a different product conversation from VM backup. - Avi API for programmatic load balancer configuration, commonly consumed by application pipelines. ## Third party integrations: the general rule Every integration falls into one of three categories, and knowing which one you are dealing with saves a lot of time. - **Version-gated.** Backup, replication, storage array integration. The vendor publishes a support matrix and you check it. Do this in Stage 0, because a backup product that does not support 9.1 yet is a schedule constraint you cannot engineer around. - **API-gated.** Automation, monitoring agents, CMDB discovery, custom tooling. These usually keep working but should be tested rather than assumed. Deprecations between major versions are real. - **Configuration-gated.** Syslog, SNMP, certificate issuance, directory integration. These break because a setting changed underneath them. The syslog port change is the clearest example, and it fails silently. The configuration-gated ones cause the most trouble, because nothing alerts you. A logging pipeline that stopped three weeks ago looks exactly like a quiet estate until somebody needs the logs. ## An API note worth making early If the customer has built automation against vCenter over several years, that automation is an asset and a constraint at the same time. It encodes real operational knowledge, and it also encodes assumptions about API behaviour that may not survive a major version change. Inventory it in Stage 0, test it against 9.1 in a non-production context during Stage 1, and treat any rework as scoped work rather than something the team absorbs quietly. Automation that silently misbehaves after an upgrade is worse than automation that fails outright. ## Sources - [VCF 9.1 Design documentation](https://techdocs.broadcom.com/us/en/vmware-cis/vcf/vcf-9-0-and-later/9-1/design.html?ref=mohammadsiddiqui.com) - [Upgrading your vSphere Foundation to 9.1](https://techdocs.broadcom.com/us/en/vmware-cis/vcf/vcf-9-0-and-later/9-1/deployment/upgrading-your-vsphere-foundation-to-9-1.html?ref=mohammadsiddiqui.com) - [KB 440630: Upgrade sequence and related issues](https://knowledge.broadcom.com/external/article/440630/upgrade-sequence-and-related-issues-for.html?ref=mohammadsiddiqui.com) - [ESX hardware requirements](https://techdocs.broadcom.com/us/en/vmware-cis/vsphere/vsphere/9-0/esx-installation-and-setup/installing-and-setting-up-esxi/esxi-requirements/esxi-hardware-requirements.html?ref=mohammadsiddiqui.com) - [Broadcom Compatibility Guide](https://compatibilityguide.broadcom.com/?ref=mohammadsiddiqui.com) - [Product Interoperability Matrix](https://interopmatrix.broadcom.com/Interoperability?ref=mohammadsiddiqui.com) - [Broadcom API documentation](https://developer.broadcom.com/xapis?ref=mohammadsiddiqui.com) - [VCF Upgrade Planning Tool](https://vmware.github.io/vcf-upgrade-planner/index.html?ref=mohammadsiddiqui.com) *Sizing figures, port numbers and version requirements move between releases. Verify each against current documentation before committing to a design.* *Views expressed here are my own.*