What are the requirements for running Virtual SAN?

Virtual SAN has specific requirements for hardware (hosts, CPU, memory, number & type of storage devices), software (version of vCenter, vSphere configuration, ESXi host version & configuration), and networking.

In traditional configurations, Virtual SAN requires 3 hosts, with at least one disk group each, configured in a vSphere Cluster with Virtual SAN enabled, with a VMkernel port configured for Virtual SAN traffic that has connectivity to other hosts in the cluster. Note* Multicast traffic is required.

More detailed information for the different versions of Virtual SAN can be found in the relative KB article for each version of Virtual SAN.

Virtual SAN 6.0: http://kb.vmware.com/kb/2106708
Virtual SAN 5.5: http://kb.vmware.com/kb/2058424

What type of hardware is required for Virtual SAN?

Hosts that are certified to run VMware vSphere along with components that are certified to run Virtual SAN. Components certified for Virtual SAN are tested by the VMware Storage and Availability Business Unit.

Virtual SAN can be installed on hardware in a bring your own component, Ready Node, or Engineered Appliance offering.

Bring Your Own
The VMware vSphere Hardware Compatibility List details which hosts and CPU configurations are supported with ESXi.
The vSphere HCL can be found here: http://vmwa.re/hcl

When combined with the vSphere HCL, the Virtual SAN Compatibility Guide provides a list of hardware certified to run Virtual SAN.
The Virtual SAN Component HCL can be found here: http://vmwa.re/vsanhclc/

Ready Nodes
Ready Nodes are a pre-validated hardware options from various OEMs.
Ready Nodes can be found here: http://vmwa.re/vsanhcl/ 
The Ready Node selector can be found here: http://vsanreadynode.vmware.com/

Warning: Using uncertified hardware may lead to performance issues and/or data loss. The reason for this is that the behavior of uncertified hardware cannot be predicted. VMware cannot provide support for environments running on uncertified hardware.

What are the cluster requirements of Virtual SAN?

Cluster requirements include

• A minimum of 3 nodes must contribute storage, unless using a 2 Node configuration along with an external witness.
  • 2 Node configurations were introduced in Virtual SAN 6.1.
• All ESXi hosts must be managed by vCenter Server 6.0 and configured as a Virtual SAN cluster member.
• vCenter Server must be at a release level equal to or higher than the ESXi hosts it is managing.
• ESXi hosts in a VSAN cluster may not participate in any other cluster.

What are the memory requirements of Virtual SAN 6?

Memory requirements are determined by the number of disk groups and devices that are managed by ESXi.

Hosts should contain at least 32GB of RAM to accommodate the maximum number of disk groups and devices.

To calculate Virtual SAN 6.0 memory consumption , use this equation:

BaseConsumption + (NumDiskGroups x ( DiskGroupBaseConsumption + (SSDMemOverheadPerGB x SSDSize)))

Where:

• BaseConsumption: This  is the fixed amount of memory consumed by Virtual SAN per ESXi host. This is currently 3 GB.
  • This memory is mostly used to house the VSAN directory, per host metadata, and memory caches.
• NumDiskGroups: This is the number of disk groups in the host, should range from 1 to 5.
• DiskGroupBaseConsumption: This is the fixed amount of memory consumed by each individual disk group in the host. This is currently 500 MB. This is mainly used to allocate resources used to support inflight operations on a per disk group level.
• SSDMemOverheadPerGB: This is the fixed amount of memory we allocate for each GB of SSD capacity.
  • This is currently 2 MB in hybrid systems and is 7 MB for all flash systems.
  • Most of this memory is used for keeping track of blocks in the SSD used for write buffer and read cache.
• SSDSize: The size of the SSD in GB.

KB Article 2113954 provides scenarios to better illustrate Virtual SAN memory requirements.

What are the processor requirements?

Processors that are approved on the vSphere Compatibility Guide are approved for use with Virtual SAN.

• Virtual SAN typically consumes no more than 10% of CPU overhead per host for versions up to 6.1.
• When using advanced Space Efficiency features in All-Flash architectures of Virtual SAN 6.2, may consume an additional 5% of CPU.

What are the software requirements?

The software requirements for Virtual SAN 6 are:

• VMware vCenter Server must be at the same version or higher than the ESXi hosts it is managing.
• ESXi hosts that participate in Virtual SAN Clusters must be version 6.0.
• ESXi hosts participating in a VSAN 6.0 cluster must be running the same ESXi version.
• Hosts versions may be mismatched during the duration of an upgrade.
• When upgrading from Virtual SAN 5.5 to 6.0, the on-disk format must be upgraded to use all available features for the specific edition of Virtual SAN.
• The VSAN-FS format for Virtual SAN 6.0 and 6.1 must be 2.0 to use all features of each edition.
• The VSAN-FS format for Virtual SAN 6.2 must be 3.0 to use all features of 6.2.

What are the network requirements?

The networking requirements for Virtual SAN 6 are:

• For hybrid configurations, each host must have a minimum of a single physical 1 GB Ethernet NIC available solely for Virtual SAN use.
• For all flash configurations, each host must have a minimum of a single physical 10 GB Ethernet NIC available for Virtual SAN use. This NIC can be shared with other traffic.
• Layer 2 multicast must be enabled on the physical switch connecting all hosts in the VSAN cluster.  Layer 3 multicast is supported.
• Each ESXi host in the cluster must have a vmkernel port, regardless of whether it contributes to storage. For more information, see the Set Up a VMkernel Network for Virtual SAN section in the VMware Virtual SAN 6.0 Documentation.

Note: When running multiple Virtual SAN clusters, it is required to have isolated Virtual SAN networks.
This isolation can be achieved through the use of:

• Each Virtual SAN network residing on a non-routed VLAN
• Each Virtual SAN network with its own distinct Multicast addresses
  KB Article 2075451 covers setting Multicast addressing in Virtual SAN.

For more information, see Networking Requirements for Virtual SAN section in the VMware Virtual SAN 6.0 Documentation.

What are the storage requirements for Virtual SAN 6?

The storage requirements are:

• For Caching:
  • At least 1 SAS/SATA Solid State Drive (SSD), PCIe flash disk, or other supported caching device.
  • Hybrid configurations will use 70% of the capacity of each caching device will be used for read caching.
  • Hybrid configurations will use 30%* of the capacity of each caching device will be used as a write buffer.
  • All-Flash configurations will use 100%* of the capacity of each caching device will be used as a write buffer.

*Up to 600GB

• Capacity for virtual machine storage:
  • Hosts running in a hybrid cluster configuration must have at least 1 SAS, NL-SAS or SATA magnetic Hard Disk (HDD).
  • Hosts running in an all-flash disk group cluster configuration must have at least 1 SAS/SATA Solid State Drive (SSD), or a PCIe flash disk
• Storage Controllers:
  • A SAS or SATA Host Bus Adapter (HBA), or RAID controller that is set up in non-RAID (pass through) or RAID 0 mode.
  • The HBA/RAID Controller must meet a minimum queue depth of 256. The Virtual SAN Compatibility Guide details recommendations for each certified controller.
  • Controller cache should be disabled or configured for 100% read.
• Flash Boot Devices:
  • When booting a Virtual SAN 6.0 enabled ESXi host from a USB or SD card, the size of the disk must be at least 4 GB.
  • When booting a Virtual SAN host from a SATADOM device, you must use a Single-Level Cell (SLC) device and the size of the boot device must be at least 16 GB.

Notes:

• Virtual SAN requires exclusive access to the local disks in the ESXi host.
• Virtual SAN disks cannot be shared with another file system, such as Virtual Flash File System (VFFS), VMFS partitions, or an ESXi boot partition.
• Do not format storage devices with VMFS or any other file system.
• Ensure flash storage is not claimed by vSphere Flash Read Cache.
• If you install ESXi on a USB or SD device and allocate all local storage to Virtual SAN, you do not have any local disk or datastore available for persistent logging. Configure a Dump Collector and a Syslog Collector to direct ESXi memory dumps and system logs to a server on the network, rather than to a local disk. For information, see the Configure ESXi Dump Collector with ESXCLIsection in the http://pubs.vmware.com/vsphere-60/topic/com.vmware.vsphere.install.doc/GUID-85D78165-E590-42CF-80AC-E78CBA307232.htmlvSphere Installation and Setup guide.

What are the licensing requirements for Virtual SAN 6?

Virtual SAN licensing varies by consumption type, version and edition

Virtual SAN may be used with any edition of vSphere. It is licensed by method and edition.

• Licensing Method
  • For Virtual Desktop Infrastructure environments
    • Virtual SAN is licensed per desktop
    • Virtual SAN Advanced Edition licensing is included in VMware Horizon Advanced & Enterprise Suites.
  • For all other environments
    • Virtual SAN is licensed per CPU for each host consuming Virtual SAN resources
    • The capacity of the license must cover the total number of CPUs in the cluster.
  • For Remote Office Branch Office Deployments
    • Introduced in Virtual SAN 6.1, this edition allows for up to 25 virtual machines to be licensed at a remote or branch site.
    • ROBO licensing does not limit the number of hosts in the cluster, but rather the number of virtual machines to 25 per site.
    • A single Virtual SAN ROBO license can be spread across multiple sites, up to the 25 virtual machine limit.
    • More than total 25 virtual machines across multiple sites require additional ROBO licenses for each multiple of 25, with no more than 25 in a single site.
• By Edition
  • Virtual SAN Standard licensing does not allow for All-Flash configurations
    • Storage Policy Based Management
    • Read/Write SSD Caching
    • Distributed RAID
    • Fault Domains
    • Snapshots/Clones
    • 6.1 added support for 5 minute RPO when replicating from one Virtual SAN datastore to another Virtual SAN datastore
    • 6.1 added support for health monitoring and vRealize Operations support with the Management Pack for Storage Devices
    • 6.1 added support for 2 Node configurations
    • 6.2 added support for software checksums
  • Virtual SAN Advanced licensing adds on Standard licensing features and supports for All-Flash configurations
    • 6.1 added for Stretched Cluster configurations
    • 6.2 removed support for Stretched Cluster configurations
    • 6.2 added support for cluster Deduplication & Compression for All-Flash configurations
    • 6.2 added support for Erasure Coding as a new Failure Tolerance Method for All-Flash configurations
  • Virtual SAN Enterprise licensing adds support for Quality of Service, and Stretched Cluster configurations
    • 6.2 added support for Stretched Cluster configurations
    • 6.2 added support for Quality of Service through IOPS limits