Node installation and cluster assembly

All nodes are installed from the same boot image, and the steps are the same on every node. The cluster parameters (ID, number of nodes, redundancy) are entered on each node again and must match everywhere. Only the node ID is unique.

The installer asks for four groups of parameters:

cluster composition: cluster ID, number of nodes, redundancy, node ID;
service addresses: the WSS URL for VM consoles and the KMS server name for activating Windows images;
network and disks: the first storage layer disk, aggregates, VLANs, default gateway, hostname;
system disk partitioning.

All values are taken from the installation checklist. Fill it in before the first server boots from the image.

The image is self-contained: it includes the base system, packages, and the management VM template (vStack Center). No internet access is needed during installation.

The cluster assembles itself on first boot. Each node waits for iSCSI sessions with all the others, then creates pools, deploys templates, and brings up the management VM. You do not need to reboot the nodes strictly at the same time, but all of them must come up within 30 minutes.

When to use

This guide takes you from bare servers to an assembled cluster. It is not suitable for adding a node to a running cluster: the number of nodes and redundancy are set during installation and cannot be changed afterwards.

Prerequisites

The nodes and switches meet the hardware and network requirements.
The installation checklist is filled in (table below).
The switches are configured: vPC or LACP in active mode, the required VLANs, MTU 9216 on the ports, IGMP snooping and a querier in the VLAN of isolated networks.
An A record is created for the domain name of the management web interface. It points to the address with the last octet 250 in the public access point network: for the 192.0.2.0/24 network, this is 192.0.2.250.
UEFI boot and VT-x are enabled in the server settings. Enable VT-d if you plan to pass PCI devices through to VMs.
The server clock is set to UTC, and the nodes will have access to an NTP server. Without time synchronization, the cluster will not start.
You have a vStack OS boot image. Images are served from the vStack repository only to allowed IP addresses. To add your address, contact support.
You have access to the server console: physical or through the management interface (Virtual media).

Installation checklist

No.	Parameter	Sample value	Note
1	First storage layer disk	`da1`	The installer takes as many consecutive disks as there are nodes in the cluster
2	LACP aggregate configuration: names, interfaces, MTU, address	`lagg10: ix0,ix1, 9000` and `lagg20: mce0,mce1, 9000`	Option with two port pairs
3	Number of nodes in the cluster	5	4 by default
4	"N plus" parameter	2	Cluster and storage layer redundancy, from 1 to 3
5	Cluster ID	100	From 1 to 999
6	Cluster network VLAN ID	1001
7	VLAN ID of the network encapsulating isolated virtual networks	1002	non-routed
8	Public access point network VLAN ID	1003	non-routed
9	Storage network (native) VLAN ID	1004	routed
10	Address space of the public access point network	`192.0.2.0/24`	non-routed
11	Default gateway	`192.0.2.1`	An address from the public access point network
12	VLAN ID range for routed virtual networks	`1010-1060`
13	Domain name of the management web interface	`mgmtvm.q100.example.com`	A record pointing to the `.250` address
14	WSS URL	`wss://mgmtvm.q100.example.com/.wss`	Connection endpoint for VM consoles
15	KMS domain name (for Windows images)	`kms02.example.com`

Choose a port scheme

The installer supports two port layouts. Choose one for the whole cluster; it determines whether you need a second aggregate.

	Dedicated SDN and SDS port pairs	Combined port pair
Aggregates	`lagg10` (VXLAN and public access points) and `lagg20` (cluster network and storage)	`lagg10` only
Physical ports	2 ports per aggregate	2 ports per node
Step for entering the second aggregate name	Enter `lagg20` and its parameters	Press Ctrl-C
When to choose	Storage traffic must be separated from virtual network traffic	The node or switch has enough ports for only one aggregate

Install a node

Repeat these steps on each cluster node.

Write the boot image to media or attach it through Virtual media in the server management interface. The image takes about 17–19 GB.
Check that UEFI is selected in the boot settings and VT-x is enabled.
Boot from the media. On the welcome screen, click Install.
Select Continue with default keymap and click Select.
Specify the first storage layer disk (item 1 of the checklist).
Specify the cluster ID, a number from 1 to 999.
Specify the number of nodes in the cluster. The default is 4, the minimum recommended configuration.
Specify the cluster and storage layer redundancy, a number from 1 to 3.
Specify the ID of the current node. It must be unique in the cluster.
Enter the WSS URL, the connection endpoint for VM consoles. Specify the URL with the scheme, for example wss://mgmtvm.q100.example.com/.wss. You can change the value later.
Specify the VLAN range, for example 1010-1060. This is a reserve for the networks from whose address space VMs get IP addresses.
Specify the KMS server name for Windows images. You can change the value later.
Specify the name of the first aggregate. Leave lagg10.
Fill in the form for the first aggregate:
- two physical interfaces;
- Aggregation Protocol: leave lacp;
- MTU: 9000. The form suggests 9216, but no more than 9000 goes into the node configuration;
- IP address and network prefix. The last octet equals 100 + node ID: for the first node 192.0.2.101/24, for the second 192.0.2.102/24, and so on.
Proceed to the second aggregate:
- if you need it, specify the name lagg20 and fill in the same form. On Mellanox adapters, the interfaces are named mce0 and mce1; do not set an MTU above 9000 on them;
- if you do not need it, press Ctrl-C at the name entry step.
Fill in the VLAN form. For each segment (cluster network, VXLAN, public access points, storage), specify the VLAN ID, the network address, and the aggregate on which this VLAN will be brought up. Take the values from items 6–10 of the checklist.
Specify the default gateway, an address from the public access point network (item 11).
Specify the hostname strictly in the q<cluster ID>u<node ID> format: the cluster ID has one to three digits, the node ID exactly three digits. For cluster 100 and node 1, this is q100u001. The configuration generator does not accept a name in any other format.
On the system components selection screen, click Ok.
Select Manual to partition the system disk manually.
In the partition editor, click Create and select GPT.
Create the first partition, efi.
Create the root file system. The recommended size is 100 GB: such a partition can later be expanded without node downtime.
Create a swap partition (SWAP).
Click Finish, then Commit. Two progress bars will run.
In the boot configuration window, click No and enter the name of the new entry.
Enter the superuser (root) password twice. The characters are not displayed as you type.
In the network configuration window, click Cancel. The network has already been described in the previous steps.
Select the time zone mode: Yes if the server firmware clock is set to UTC, No if this is unknown.
If you selected No, specify the UTC time zone. All cluster nodes must run in UTC.
Check the date and time. If they are correct, click Skip.
On the screen of services started at node boot, leave the default configuration. ntpd is already enabled in it.
On the security options screen, leave the default configuration.
Skip the step for creating additional users: click No.
In the final configuration window, select Exit and click Ok. The preliminary placement of cloud images and repositories starts. For several minutes there is no progress bar; the waiting window stays open.
Wait for the packages to be installed on the node.
Click No to finish the installation.

Assemble the cluster

Make sure vStack OS is installed on all nodes.
Reboot each node with the Reboot button. Strict synchronization is not required, but stay within 30 minutes: that is how long each node waits for iSCSI sessions with the others.
Wait for the nodes to create pools and place cloud images on them. A node that has completed this stage waits for the others again.
Wait for the management VM (vStack Center) to come up and the cluster to start operating.

Verify the result

On any node:

hostname returns a name in the q100u001 format.
ifconfig lagg10 (and ifconfig lagg20, if there is a second aggregate) shows laggproto lacp, MTU 9000, and both ports have the ACTIVE, COLLECTING, and DISTRIBUTING flags. A port without these flags means that the link to the switch did not come up in LACP.
zpool list shows the cluster pools in the ONLINE state.
The address with the last octet 250 in the public access point network responds: the management VM runs there.

How to check the cluster composition and pools in the web interface is described in the vStack Center documentation.

Troubleshooting

The aggregate names were changed. The name lagg10 is hardcoded in the configuration of the cluster service, and other settings also expect the standard names. Leave lagg10 and lagg20 as they are.
MTU above 9000. The installer writes 9000 anyway, and on Mellanox adapters a larger MTU makes the interface fall back to 1500. Set 9000, and leave 9216 to the switch.
The aggregate does not come up. Check that the switch ports are in LACP active mode, not on, and that both node ports go into the same vPC / LACP group.
A node ID is duplicated. A cluster with identical node IDs will not assemble. Check the ID against the checklist on each node.
The hostname is in a different format. The configuration generator accepts only q<1–3 digits>u<3 digits>. Reinstall the node with the correct name.
The time zone is not UTC, or there is no NTP. A different time zone breaks guest operating systems, and without time synchronization the cluster does not start. Select UTC and give the nodes access to NTP.
The final configuration window "hangs". There is no progress bar during the cloud image placement stage. Wait a few minutes; the time depends on the speed of the media and disks.
A node did not wait for the others. If iSCSI sessions with all nodes do not come up within 30 minutes, the node aborts the assembly and retries it on the next boot. Bring up the lagging nodes, then reboot the node that stopped waiting.

When to use​

Prerequisites​

Installation checklist​

Choose a port scheme​

Install a node​

Assemble the cluster​

Verify the result​

Troubleshooting​

See also​