VCF 9.1 Storage Architecture Deep Dive: vSAN ESA, Global Deduplication, Object Storage, and the Disaggregated Model

This is Part 6 of an architect’s read of VCF 9.1. Parts 1–5 covered the field guide, the fleet and instance HLD/LLD, the lifecycle deep dive, the networking deep dive, and the security architecture. This one is storage.

Storage in VCF used to mean one thing: vSAN HCI on the management domain, vSAN HCI on workload domains, plus the occasional NFS or FC mount. The conversation was about disk groups, cache tiers, and FTT. Storage design was about which servers had which disks.

VCF 9.x rebuilt that. The platform now offers a layered storage architecture: vSAN ESA as the design baseline, three deployment models (HCI, Storage Cluster, Compute Cluster), three storage types (block, file, and now object), three tiers (DRAM, NVMe Memory Tier, ESA capacity), and cross-vCenter datastore sharing that makes vSAN look more like a traditional array than a hyperconverged thing.

9.1 finishes several pieces of that architecture. Global Deduplication moves from limited availability to GA with up to 8× capacity reduction and Data-at-Rest Encryption compatibility. ZStandard replaces LZ4 as the compression algorithm. Auto-RAID arrives. ESA and OSA can finally co-exist in a single estate with concurrent mounting. Native S3 Object Storage drops as a tech preview the first time VCF has had its own object storage rather than partner integrations. Storage Clusters can be shared cross-vCenter, like external arrays.

This is the architect’s deep dive on what the storage architecture actually looks like in 9.1 from the HLD design baselines down to the LLD-level component choices.

The storage problem VCF used to have

Pre-9.x storage in VCF was a collection of capabilities rather than a coherent architecture:

• vSAN OSA (Original Storage Architecture) as the dominant model, with disk groups, dedicated cache tiers, and per-disk-group deduplication

• vSAN ESA available but considered new deployed for performance use cases, not as the default

• vVols still in use for storage-array-managed VMs, but increasingly without strategic investment

• NFS, FC, and iSCSI mounted alongside vSAN with limited integration into VCF lifecycle and consumption

• File services existed but limited in scale (≤ 250 shares per cluster)

• Object storage required a third-party ISV integration

• Encryption posture forced trade-offs dedup didn’t work with data-at-rest encryption

• Stretched clusters and storage clusters were architecturally distinct from regular vSAN, with separate design considerations

Day-2 storage design meant orchestrating across multiple storage feature sets, separate compatibility matrices, and ISV add-ons for object storage. Storage architecture decisions were per-feature, not platform-coherent.

VCF 9.x rebuilt this. 9.1 finishes the architectural shift.

The 9.x storage architecture (foundation)

The new storage architecture rests on three pillars.

vSAN ESA as the design baseline. For new VCF 9.x deployments, ESA is the recommended architecture. ESA removes the disk group construct, takes full advantage of NVMe-only configurations, and enables features that simply weren’t possible in OSA: scalable snapshots (100× faster than OSA), global deduplication, multi-source replication for vSAN Protection, NVMe Memory Tiering, NVMe Dispersed Namespaces for stretch clusters, and now native S3 object storage. OSA is still supported, but ESA is the architectural direction.

Three deployment models. vSAN now supports three cluster models that can coexist in a single VCF instance:

• vSAN HCI traditional hyperconverged. Compute and storage co-located. Best for most general workloads

• vSAN Storage Cluster disaggregated model. Dedicated storage cluster providing capacity to one or more vSphere clusters. Like an external array, but software-defined

• vSAN Compute Cluster compute-only vSphere cluster that mounts remote vSAN datastores from storage clusters. Lets compute scale independently of storage

Three storage types on one platform. With Native S3 Object Storage arriving in 9.1 (tech preview), vSAN now delivers block, file, and object storage all on the same cluster. This was previously the domain of multi-product solutions or external storage. The implication for HLD: object storage requirements no longer drive separate platform decisions they’re first-class on VCF.

In addition to vSAN, traditional storage protocols are first-class in VCF: NFS (up to 256 hosts per datastore in 9.1), FC, and iSCSI (with iSNS auto-discovery added in 9.1). The architectural shift is that all of these now sit under the unified VCF Operations / Fleet Manager lifecycle plane storage design no longer means orchestrating across separate tooling for each protocol.

What 9.1 actually changed in storage

VCF 9.1 layered substantial new storage capabilities on the 9.0 foundation. The ones I’d flag in any HLD review:

Global Deduplication generally available. In 9.0, Global Dedup in vSAN ESA was limited availability behind a Technical Qualification Request, restricted to single-site clusters of 3–16 hosts on 25 GbE or faster, and didn’t work with Data-at-Rest Encryption. In 9.1, it goes GA. Supported on vSAN HCI clusters from 3 to 64 hosts and on vSAN Storage Clusters. Compatible with Data-at-Rest Encryption dedup post-processing temporarily decrypts blocks in memory to find matches, preserving the security posture without compromising data reduction. Customers can achieve up to 8× capacity reduction. Cluster-wide deduplication domain rather than per-disk-group. Post-process, no write-path performance impact.

ZStandard (ZSTD) compression replaces LZ4. vSAN ESA in 9.1 moves from LZ4 to ZSTD compression. Better ratios, especially for structured data (databases like SQL Server and Oracle benefit most due to repeatable patterns in integers, dates, headers, keys). Compression happens high in the storage stack before encryption and before network transmission so the benefits compound through the rest of the stack. Always-on, cluster-level feature.

Auto-RAID. New storage policy option in 9.1 that automatically selects the optimal RAID configuration for each object based on cluster size, features enabled, and stretched cluster topology. If the cluster grows, shrinks, or transitions to/from stretched, Auto-RAID re-configures associated VMs automatically. Eliminates the manual RAID-design overhead and the policy drift that comes with cluster changes. For HLD, this changes the storage policy conversation from “which FTT/RAID for which workload” to “Auto-RAID or hand-tuned, and why.”

vSAN Storage Clusters can be shared cross-vCenter. Previously, a vSAN Storage Cluster’s datastore was scoped to one vCenter. In 9.1, vSAN Storage Clusters can be mounted across vCenter boundaries just like a traditional external array. This means a single storage cluster can serve multiple workload domains, multiple instances, and multiple vCenters. Storage and compute scale independently in a way that was previously the domain of external SAN.

ESA and OSA concurrent mounting. In 9.0 and earlier, OSA clusters and ESA clusters were mutually exclusive for cross-mounting. In 9.1, that restriction is gone. OSA clusters can mount ESA datastores, ESA clusters can mount OSA datastores, and vSAN Compute Clusters can mount both OSA and ESA datastores concurrently. The migration implication: customers can stand up new ESA storage capacity alongside existing OSA clusters and migrate workloads incrementally via Storage vMotion. This is the cleanest OSA → ESA transition path the platform has offered.

Data-in-Transit Encryption for vSAN Storage Clusters. In disaggregated topologies, traffic between client clusters and the vSAN Storage Cluster traverses the network. 9.1 adds Data-in-Transit Encryption for this client-to-storage traffic, completing the end-to-end encryption story across disaggregated vSAN deployments.

Native S3 Object Storage (Tech Preview). This is the headline storage feature in 9.1. For the first time, vSAN delivers native S3-compatible object storage not a partner integration, not a bolt-on appliance. Built directly into vSAN, sharing infrastructure with vSAN File Services (distributed file system, containerised protocol services). Provisioned and managed through VCF Automation and the vSphere Supervisor. Multi-tenant by design buckets-as-a-service, with project and quota management. Included in the VCF license (no separate purchase). Works with Global Deduplication. Delivers approximately 34% lower TCO compared to standalone on-premises object storage products. Tech preview in 9.1.x means it’s available for evaluation but not yet recommended for production-critical workloads. For HLDs that have historically called out a separate object storage tier, this changes the conversation.

Cyber Recovery ReadyNodes with QLC. 9.1 introduces ReadyNodes optimised specifically for cyber recovery workloads, using QLC (Quad-Level Cell) NAND. The balance is intentional high density, adequate performance for recovery workflows, durable enough for the recovery-write pattern, all-flash density without all-flash cost. Combined with the IRE (Isolated Recovery Environment) clean room from Part 5, this makes the cyber recovery design more affordable at the BoM level.

vSAN for Recovery any-source replication into ESA. 9.1 enables replication from any vSAN storage platform efficiently into vSAN ESA. Combined with the deep snapshot chains in ESA, this makes vSAN ESA a viable consolidation target for protection across heterogeneous source estates vSAN, VMFS, NFS, even FC-backed VMware datastores. The vSAN Protection multi-source replication topology (covered in Part 1) sits on top of this.

Persistent volume scale increases. For Kubernetes workloads, RWO persistent volume limits jump from 7,500 to 25,000 per Supervisor (233% increase), and from 30,000 to 50,000 per vCenter (66% increase). Eliminates scaling constraints for large Kubernetes deployments and multi-tenant K8s environments.

Enhanced storage capacity reporting. Clearer insights into storage usage trends, projection, and capacity recommendations. Especially relevant for NVMe Memory Tiering capacity planning (mentioned in Part 1).

Resizable shared VMDKs. VMDKs shared between multiple VMs (e.g. clustered applications) can now be dynamically resized without downtime. Smaller feature, but it removes a long-standing operational pain point.

The new storage topology (LLD)

A complete VCF 9.1 storage architecture in topology terms:

Memory tier: DRAM. The fastest tier. Where active VM memory pages live.

Memory tier: NVMe Memory Tier (per host). Local NVMe acting as a secondary memory tier (Part 1). Cold pages migrate here. Native software mirroring in 9.1 no hardware-mirrored DIMMs required for tier resilience.

Block storage tier: vSAN ESA. The capacity tier for VMs. ESA-based, with Global Deduplication (now GA), ZStandard compression, Data-at-Rest and Data-in-Transit Encryption, Auto-RAID, scalable snapshots, multi-source replication.

File storage tier: vSAN File Services. NFS v3/v4.1 and SMB with Kerberos. 500 file shares per cluster, 100 SMB shares. Containerised stateless protocol services on each host. Works in HCI, disaggregated, 2-node, and stretched topologies.

Object storage tier: vSAN Native S3 (Tech Preview). S3-compatible object storage on the same vSAN substrate. Multi-tenant. Provisioned through VCF Automation. Works with Global Dedup.

External storage protocols. NFS (256 hosts/datastore in 9.1), FC, iSCSI (with iSNS auto-discovery in 9.1). All under unified Fleet Manager lifecycle.

Stretched/metro tier: NVMe Dispersed Namespaces. For metro DR designs where DRAM-class latency budgets matter. Partner-validated NVMe-based stretch cluster solutions.

Across all of these: VCF Operations provides unified observability and lifecycle. Fleet Manager (Part 3) handles upgrade and patch. Encryption is end-to-end where required.

HLD design patterns

Five practical storage architecture patterns I work from.

Pattern 1: vSAN HCI for most workloads. Compute and storage co-located. ESA as the architecture. Global Dedup on for capacity-heavy clusters. Suitable for the majority of enterprise deployments where scaling compute and storage together makes operational sense.

Pattern 2: Disaggregated vSAN Storage Cluster. Dedicated storage cluster providing capacity to multiple vSphere clusters across vCenter boundaries (new in 9.1). Compute and storage scale independently. Storage cluster sized for capacity efficiency (Global Dedup + ZSTD compression aggressive); compute clusters sized for workload. Useful when compute growth and storage growth follow different curves typical in mature enterprise estates and many service-provider designs.

Pattern 3: Hybrid HCI + Compute clusters mounting shared storage. Some clusters run HCI (general-purpose workloads), others run compute-only (mount the shared Storage Cluster). Storage architecture is heterogeneous within a single VCF instance but operationally unified.

Pattern 4: Mixed-architecture transition (ESA + OSA). Existing OSA estate. New ESA capacity added. 9.1’s ESA+OSA concurrent mounting lets you mount the new ESA datastore from existing OSA clusters and migrate via Storage vMotion. Cleanest OSA → ESA transition path. Particularly important for customers approaching end-of-life on OSA hardware but not ready for a full forklift refresh.

Pattern 5: Cyber Recovery topology. Cyber Recovery ReadyNodes (QLC-based, introduced in 9.1) running vSAN ESA in the IRE clean room (Part 5). Multi-source replication from production vSAN/VMFS/NFS sources into the recovery target. Tag-based VM selection for protection groups. Manual seeding for initial sync. This is the architectural completion of the on-prem cyber recovery story.

Storage tier strategy

Tier design in VCF 9.1 has a clean layered structure:

• DRAM (tier 0) active memory pages

• NVMe Memory Tier (tier 1) cold memory pages, per-host local NVMe

• vSAN ESA block capacity tier VM data, with Global Dedup + ZSTD compression

• vSAN File Services tier NFS/SMB shares for unstructured data

• vSAN Native S3 Object Storage tier (tech preview) S3-compatible buckets for app data, backups, archive

• External storage tier NFS, FC, iSCSI where existing investment or specific workload demands it

The HLD-level decision: where does each workload class sit, and what’s the tiering strategy? Memory-intensive workloads benefit from Memory Tiering (up to 40% TCO reduction). General workloads sit on vSAN ESA capacity. File-protocol workloads land on File Services. Backup, archive, and S3-native apps can now land on the native object tier instead of external storage.

Common HLD/LLD anti-patterns

Defaulting to OSA. For new 9.1 deployments, ESA is the design baseline. OSA still works, but it doesn’t get the new capabilities Global Dedup, ZSTD compression, Auto-RAID, scalable snapshots, multi-source replication, NVMe Memory Tiering, native object storage. If you’re writing an HLD that defaults to OSA without a specific justification, revisit it.

Treating vSAN as the only storage option. 9.1 storage architecture is layered. External NFS (256 hosts/datastore), FC, and iSCSI (with iSNS) all sit cleanly in the architecture. For customers with significant existing investment in NetApp, Pure, Dell PowerStore, or HPE Alletra, the design should integrate not displace.

Sizing without Global Deduplication. In 9.1, Global Dedup is GA and works with encryption. For capacity sizing, the raw-to-effective ratio shifts substantially up to 8× in best cases. If you’re still sizing for raw capacity without modelling dedup, the design will over-provision. Re-baseline.

Skipping the ESA + OSA concurrent mounting capability for migrations. 9.1’s ability to mount ESA datastores from OSA clusters (and vice versa) is the cleanest OSA → ESA migration path. If the HLD treats OSA → ESA as a forklift refresh, you’re missing the easier option stand up new ESA capacity, mount it from OSA clusters, Storage vMotion incrementally, decommission OSA at end of life.

Designing object storage as a separate platform. Native S3 Object Storage is in tech preview in 9.1.x, included in the VCF license, multi-tenant via VCF Automation. For backup, archive, and S3-native apps, this changes the architectural conversation. Don’t default to a separate object storage platform without evaluating the native option.

Ignoring Auto-RAID. Auto-RAID handles the policy decision and re-configures on cluster changes automatically. Hand-tuned RAID policies are still valid for specific workload requirements, but the default should be Auto-RAID. The HLD should explicitly call out where Auto-RAID is used and where it isn’t.

Forgetting Data-in-Transit Encryption for disaggregated topologies. If you’re using vSAN Storage Clusters with cross-vCenter sharing (new in 9.1), client-to-storage traffic crosses the network. DiT Encryption closes that gap. Spec it into the LLD for any disaggregated design where the threat model includes network-side observation.

Underestimating the impact of vVols deprecation. vVols are deprecated as of 9.0 and will be fully disabled in a future 9.x release. If the existing estate depends on vVols, the HLD needs a migration plan to vSAN ESA, NFS, or VMFS not a Day-2 problem.

Decision framework: where does this storage requirement live?

When a storage requirement arrives, work through the hierarchy:

• New VM capacity add to existing vSAN ESA cluster (HCI or Storage Cluster) with Auto-RAID policy

• New file share vSAN File Services on existing cluster (up to 500 shares/cluster)

• New object storage requirement if tech preview is acceptable, native S3 Object Storage on vSAN; otherwise external

• New capacity-efficiency demand enable Global Deduplication on the cluster if not already on

• New stretched cluster ESA with NVMe Dispersed Namespaces (partner-validated)

• New disaggregated workload domain vSAN Storage Cluster + vSAN Compute Clusters, with cross-vCenter sharing (9.1)

• Cyber recovery target ReadyNodes with QLC running vSAN ESA, multi-source replication from production sources

• OSA estate hitting end-of-life add ESA capacity, mount cross-architecture, Storage vMotion incrementally

• Persistent volume scale for K8s vSAN with the new 9.1 limits (25K RWO/Supervisor)

• Backup/archive target evaluate native S3 first; fall back to external if tech preview not acceptable

This sequence keeps the design grounded in the new architecture rather than reverting to per-feature reasoning.

Closing

Storage in VCF 9.1 stops being a feature list and becomes a coherent architecture. vSAN ESA as the design baseline. Global Deduplication generally available with Data-at-Rest Encryption support. ZStandard compression replacing LZ4. Auto-RAID. vSAN Storage Clusters going cross-vCenter. ESA and OSA finally interoperable. Data-in-Transit Encryption for disaggregated topologies. Native S3 Object Storage as a tech preview. Cyber Recovery ReadyNodes with QLC for affordable IRE BoMs. Persistent volume scale lifting to support real Kubernetes density.

For my own design reviews, the storage questions I work through every time:

• vSAN ESA, OSA, or both (with concurrent mounting for migration)?

• HCI, Storage Cluster, Compute Cluster or a mix?

• Global Deduplication enabled, and how does it affect capacity sizing?

• Auto-RAID for the cluster, or hand-tuned policies for specific workloads?

• Encryption posture Data-at-Rest, Data-in-Transit, or both?

• Native S3 Object Storage (tech preview) or external object?

• Stretched topology with NVMe Dispersed Namespaces?

• Cyber Recovery ReadyNodes for the IRE target?

• vVols migration plan if existing estate depends on them?

Get those answered and the storage layer of the HLD writes itself.

That closes Part 6. Together with the Architect’s Field Guide (Part 1), the Fleet and Instance HLD/LLD (Part 2), the Lifecycle Deep Dive (Part 3), the Networking Deep Dive (Part 4), and the Security Architecture Deep Dive (Part 5), this is the six-part read of VCF 9.1 I’d hand to a senior architect coming to the platform fresh.