Hardware and network requirements

vStack HCP runs on the same servers simultaneously as a hypervisor, software-defined storage (SDS), and a software-defined network (SDN). There is no separate storage array and no separate management servers. That is why a node is sized for three workloads at once: cores and memory go to virtual machines, disks to the storage layer, and a pair of aggregated ports to inter-node SDS and SDN traffic.

A cluster is built from identical nodes. The storage layer is spread across all nodes, and redundancy is set once, during installation, as a number from 1 to 3. This number and the number of nodes determine how many simultaneous failures the cluster survives and how much capacity goes to redundancy.

The storage layer is all-flash only: system disks and pool disks must be SSD or NVMe. The platform does not provide for hybrid configurations with HDDs.

For the cluster, the network works as an internal bus. A node connects with two LACP aggregates through two switches. One aggregate carries SDN traffic and public access points, the other carries storage traffic and cluster communication. The switching requirements are therefore as strict as the server requirements.

The management VM (vStack Center) lives on the same cluster. It does not need a separate server, but its resources are part of the overall cluster balance, and in the cluster networks an address with the last octet 250 is reserved for it.

Node requirements

Component	Minimum requirement
Processor	Intel Xeon E5-2600 v4 or newer. AMD processors are not supported in the 3.x line
Virtualization	Intel VT-x with EPT. VT-d is required if PCI devices are passed through to VMs
RAM	384 GB or more. Up to 64 GB of it is taken by the ZFS cache (ARC)
Boot mode	UEFI. Booting in BIOS (Legacy) mode is not supported
System disks	2 SSDs of at least 96 GB each
Storage layer disks	X SSD or NVMe disks of at least 512 GB each, where X is the number of nodes in the cluster
Network adapters	At least one adapter with two 10 GbE ports. Two adapters are recommended, one in a PCI slot of each processor socket
HBA	LSI 9400-16i or another controller from the list below
Switching	According to the cluster switching scheme
Nodes in the cluster	4 or more. 4 nodes is the minimum recommended configuration

Supported adapters and controllers

A node boots with the GENERIC_PROD kernel. An adapter works only if its driver is in this kernel. No loadable modules for other families are shipped. Having a driver means the adapter will be visible to the system, not that it has been tested as part of a cluster. Tested models are listed in the recommended configurations.

Class	Driver	Family
HBA and RAID	mpr	LSI/Broadcom SAS3: 9300, 9400 (including 9400-16i)
	mps	LSI/Broadcom SAS2: 9200
	mrsas, mfi	LSI/Broadcom MegaRAID
	smartpqi	Microchip (Adaptec) SmartPQI
	aac, ahd	Adaptec
	ciss	HP Smart Array
	hptiop	HighPoint RocketRAID
	isp	QLogic Fibre Channel
	ahci	Onboard SATA controllers
NVMe	nvme, nda	Directly attached NVMe drives
Network	ix	Intel 10 GbE: 82599, X520, X540, X550
	ixl	Intel 700 series: X710, XL710, XXV710
	ice	Intel E810
	iavf	Intel virtual functions
	igc, em	Intel 1 GbE, including the igb family
	mlx5en	Mellanox/NVIDIA ConnectX-4 and newer, mce* interfaces
	bce, bge	Broadcom NetXtreme and NetXtreme II (1 GbE)
	vmx	VMware VMXNET3

Not supported, because their drivers are not in the kernel:

• LSI/Broadcom 9500 and 9600 HBAs (mpi3mr driver);
• Broadcom NetXtreme-C/E network adapters (bnxt driver);
• Chelsio network adapters (cxgbe driver).

Storage disks and pools

The storage layer is built in three levels.

Disks. SSD and NVMe only. There is no split into "cache" and "capacity" as in hybrid systems, and no storage tiers either.

Each node needs as many storage disks as there are nodes in the cluster. The installer asks for the first pool disk, for example da1, and takes X consecutive disks starting from it: with da1 on a 5-node cluster, these are da1–da5. System disks must not fall into this range.

Pool. A group of disks under ZFS. A pool is not built on a single node but from one disk on each node of the cluster. A node contributes its own disks to the pool directly and sees its neighbors' disks over iSCSI. Therefore a pool has as many disks as there are nodes, and the cluster has as many pools as there are nodes as well. A pool is built as RAID-Z with a level equal to the cluster redundancy (the "N plus" parameter): 1 — RAID-Z1, 2 — RAID-Z2, 3 — RAID-Z3. All disks in it are equal.

Pool redundancy matches cluster redundancy. A 7-node cluster with redundancy 2 keeps a pool running as long as at least 5 disks in the pool are alive. A disk failure affects the whole pool, not an individual virtual machine.

Redundancy also determines the share of capacity that goes to parity: N disks out of X. On 4 nodes with redundancy 2, this is half of the raw capacity.

Recommended cluster configurations

The platform has no strict requirements for the model range. Below are configurations that are already running in production. Use them as a reference.

Node parameter	Minimum: 4 nodes (3+1)	Recommended: 9 nodes (7+2)	Maximum: 24 nodes (22+2)
Platform	2U Intel server platform	2U Intel server platform	2U Intel server platform, High Performance Fans, NVMe Direct Connection
Processors	2 × Intel Xeon E5-2620 v4 (8 cores) or higher	2 × Intel Xeon Gold 6348 (28 cores)	2 × Intel Xeon Platinum 8558P (48 cores)
Memory	12 × 32 GB DDR4 RDIMM 2400 MHz	16 × 64 GB DDR4 RDIMM 3200 MHz	32 × 64 GB DDR5 RDIMM 5600 MHz
HBA	1 × LSI 9400-16i	1 × LSI 9400-16i	Direct NVMe connection
Storage layer disks	4 × 960 GB SSD SATA (DWPD=1, Intel DC S4510 or an enterprise equivalent)	9 × 3840 GB SSD SAS (DWPD ≥ 1, Samsung PM1643a or an enterprise equivalent)	24 × 3840 GB SSD NVMe (DWPD ≥ 1, Dapustor R5101 or an enterprise equivalent)
System disks	2 × 240 GB SSD SATA (Samsung PM883 or an enterprise equivalent)	2 × 240 GB SSD SATA (Samsung PM883 or an enterprise equivalent)	2 × 960 GB SSD NVMe (Intel P5530 or an enterprise equivalent)
Network adapters	2 × dual-port 10/25 Gbps Eth SFP28 (Mellanox CX-5, Intel XXV710, E810)	2 × dual-port 10/25 Gbps Eth SFP28 (Mellanox CX-5, Intel XXV710, E810)	2 × dual-port 25 Gbps Eth SFP28 (Mellanox CX-5, Intel XXV710, E810)
Power	2 × power supply (Hot Plug)	2 × power supply (Hot Plug)	2 × power supply (Hot Plug)

Redundancy

The installer allows redundancy 2 on 4 nodes as well, but then half of the pool capacity goes to parity. For a production installation with N+2, we recommend 5 nodes or more.

Switching requirements

• At least two switches with Tagged VLAN and LACP / vPC support.
• Each switch has at least as many ports as the number of nodes multiplied by 2.
• LACP on node ports in active mode. Static aggregation (mode on) is not suitable: the node expects LACPDUs from the switch.
• IGMP snooping is enabled, and the VLAN of isolated networks has an IGMP querier. Isolated virtual networks run as VXLAN over the multicast groups 239.248.x.x. Without a querier, the switch stops forwarding this traffic when the group entries expire.
• MTU 9216 on switch ports. MTU 9000 on node aggregates: the installer accepts a larger value, but writes no more than 9000 to the node configuration.

Any modern enterprise-class switch will do. Tested models:

• Cisco Nexus 3524X;
• Extreme Networks X590-24x-1q-2c;
• Huawei CloudEngine S6730-H24X6C;
• Juniper EX4600-40F-AFI;
• Eltex MES5448.

Network segments

A node brings up two LACP aggregates, with two ports in each:

Aggregate	What it carries
lagg10	The network encapsulating isolated virtual networks (VXLAN), and the public access point network
lagg20	The cluster network and the storage network (iSCSI, native)

If there are only enough ports for one aggregate, you can skip the second one. Then all segments go through lagg10. How to choose the scheme is described in the installation guide.

Traffic is separated by VLAN. The numbers below are taken from the sample installation checklist and match the installer defaults.

Segment	Sample VLAN ID	Routing	MTU
Cluster network	1001	—	same as the aggregate
Network encapsulating isolated virtual networks	1002	non-routed	same as the aggregate
Public access point network	1003	non-routed	1500
Storage network (native)	1004	routed	same as the aggregate
VLAN ID range for routed virtual networks	1010–1060	—	—

The address space of the public access point network is set separately. In the examples in this section, it is 192.0.2.0/24. The node default gateway is taken from the same network.

Addresses in the cluster networks are allocated as follows:

• a node gets an address with the last octet 100 + node ID: node 1 — .101, node 2 — .102, and so on;
• the address with the last octet 250 in the public access point network and in the cluster network is taken by the management VM (vStack Center). Its web interface opens at the address in the public access point network;
• .249 is reserved as well.

The node ID is in the range 1–63: Cluster Manager will not accept a larger number.

How to verify the configuration after installation through vStack Center is described in the vStack Center documentation.

Limitations and workarounds

Limitation	Why	What to do
AMD processors	The 3.x line supports Intel only. AMD EPYC support is being prepared in the 4.0 line	Use Intel Xeon servers
Booting in BIOS mode	The installer partitions the system disk for UEFI with an ESP partition	Enable UEFI in the server settings before installation
LSI 9500/9600 HBAs, Broadcom bnxt and Chelsio cxgbe adapters	The mpi3mr, bnxt, and cxgbe drivers are not in the node kernel	Use controllers and adapters from the supported list
Fewer storage layer disks than nodes	Each pool takes one disk from each node, and there are as many pools as nodes	Add disks to the nodes up to X before you start the installation
HDDs in the storage layer	The platform is designed for all-flash; all tested configurations are built on SSD and NVMe	Put SSD or NVMe disks in the pools
No storage tiers	The "cache on SSD plus capacity on HDD" scheme is not supported. A separate "slow" pool is not a solution either: a pool inherits the characteristics of its disks	Plan all capacity on SSD and NVMe
Redundancy 2 on 4 nodes consumes half of the capacity	2 disks out of 4 in each pool go to parity	For N+2, plan 5 nodes or more
One switch instead of two	An LACP / vPC aggregate is built across a pair of switches. With a single switch, switching fault tolerance is lost	Install two enterprise-class switches
Fewer switch ports than nodes × 2	Each node connects with two ports to each switch	Increase the number of ports or add a switch
LACP in on mode	Without an LACPDU exchange, the aggregate on the node side does not come up	Set the ports to mode active
IGMP snooping is off, or the VLAN of isolated networks has no querier	Isolated network traffic goes through multicast groups that the switch tracks via IGMP	Enable IGMP snooping and assign a querier in the VLAN of isolated networks
Aggregate MTU above 9000	The installer writes no more than 9000 to the configuration. On Mellanox adapters, an MTU above 9000 also makes the interface fall back to 1500	Set 9000 on the aggregates, and leave 9216 to the switch

See also

• Node installation and cluster assembly — the installation procedure and the parameters the installer asks for.
• Initial cluster setup — acceptance of the cluster composition and basic settings after installation, in the vStack Center documentation.