Skip to main content

Storage pools

The problem and why it matters​

A hyperconverged cluster has no external storage system. Disks are installed in the same servers that run virtual machines, and the software-defined storage layer (SDS) combines these disks into pools. A pool provides redundancy, compression, and deduplication and stores all cluster data.

This raises a practical issue for administrators: a pool is not just a "disk with a usage percentage", but an object with two different capacities, four state flags, and disks on all cluster nodes. Without understanding this model, you can misinterpret the numbers in the UI: raw capacity is mistaken for available capacity, and the total size of virtual machine disks is mistaken for used space.

This page explains the model. For viewing pools, see How to view cluster pools; for working with dataset properties, see How to view and change dataset properties.

Model and entities​

The cluster storage layer consists of three levels.

Disks and pool. A pool is a group of disks combined by ZFS. It consists of disks from all cluster nodes, not a single node, and its redundancy matches the cluster's redundancy. Losing a disk is therefore a pool-level event rather than an event affecting a single virtual machine. Disk distribution across pools, the resulting RAID-Z level, and compatible disks are described in the vStack OS documentation: Storage disks and pools.

A pool has a name (name), a persistent identifier (guid), and the number of the node currently serving it (node_id) — the node through which the cluster currently accesses the pool. In the UI, this is the Current node column. The pool's disks do not belong to this node: they are distributed across all nodes, and each pool disk has its own node number.

Dataset. A unit within a pool that holds a virtual machine disk. It is addressed by a full name such as pool-a/vm-100-disk-0 and its own GUID. Dataset properties — compression, deduplication, volume size, checksums — are stored separately from pool properties and changed through separate methods.

EntityAddressed byOwned by
Poolguid, or the pair nodeId (serving node) + namecluster
Datasetdisk guidpool

How it works​

The vStack Center storage layer has two independent paths to data, and the difference between them explains most unexpected responses.

The first path is the pool map from the vStack Center database. The cluster dispatcher keeps a snapshot of all pool states and responds to requests immediately, synchronously. The map is available even when a node does not respond, but it shows the last captured state rather than the current state.

The second path is a live query to the node. The request is relayed to the Storage agent on the required node. First comes the intermediate response {"message": "relayed"}, then the final response via a callback from the agent. Live methods provide the full set of ZFS properties, but an unresponsive node means an error instead of data.

What you needMethodModeData source
All pools grouped by nodepool-mapsynchronousvStack Center database
All pools with capacity and statepool-map-by-guidsynchronousvStack Center database
Pools served by a nodepools-listasynchronousNode Storage agent
Full properties of one poolpools-getasynchronousNode Storage agent
Dataset propertiesfilesystems-getasynchronousNode Storage agent
Changing dataset propertiesfilesystems-setasynchronousNode Storage agent

Two pool capacities​

Seven fields describe pool capacity, and confusing them can be costly: the difference between raw and usable capacity is the portion allocated to redundancy.

FieldWhat it shows
size_rawRaw capacity — the sum of pool disk capacities
sizeUsable capacity — size_raw minus the portion allocated to redundancy
usedSpace used by data
freeRemaining usable capacity: size minus used
capacity_pctPool usage as a percentage
provisionedThe sum of the requested sizes of all datasets in the pool
provisioned_pctThe ratio of provisioned to usable capacity, as a percentage

Datasets use thin provisioning: the requested volume size is not reserved immediately. Thus, provisioned is calculated from requested sizes, while used reflects actual usage, and the first number can readily exceed the second. A provisioned_pct value above 100 indicates overcommitment: the pool's datasets have requested more space than the pool's usable capacity. This is not an emergency in itself, but from this point on, monitor pool usage using capacity_pct rather than remaining quotas.

Four state flags​

FieldMeaning
healthPool state; ONLINE for a healthy pool
consumingThe cluster has accepted the pool for active data storage (1 / 0)
scrubbingA data integrity check is in progress (1 / 0)
resilveringRedundancy is being restored after disk replacement (1 / 0)

Alongside these fields, the map returns lif — the network address at which the pool is accessible.

The consuming flag controls writes to the pool. The cluster does not write data to a pool with a value of 0, and operations that require writes are rejected: creating and deleting virtual machine snapshots, rolling back to a snapshot, sending a backup, creating a machine on an explicitly selected pool, cloning a machine using the source pool, deleting a machine disk, and operations on backup datasets. If such an operation fails while the virtual machine is running, check its pool's consuming value before looking for a cause in the machine itself. Adding and expanding a disk do not check this flag.

Who can see what​

The two levels have different visibility scopes. This is a property of the methods, not a setting.

A pool is a cluster-level object. Live pool methods (pools-list, pools-get) are available only to the global Cluster master role. Two global roles can read the pool map (pool-map): Cluster master and Resource Viewer. A VDC (virtual data center) administrator cannot see a pool: a pool has no owning VDC, and a single pool holds datasets from different VDCs. The exception is pool-map-by-guid: the canonical API documentation does not specify access to this method.

A dataset is a VDC-level object. Reading properties (filesystems-get) and changing them (filesystems-set) are available to the VDC administrator role and above, and only within their own VDC. Dataset ownership is checked before contacting the node, and a dataset belonging to another VDC is rejected with the same error code as a nonexistent one: the existence of a dataset in another VDC is not disclosed. A global role can work with datasets in any VDC.

Limitations​

At the pool level, the cluster API is read-only. The storage group has no methods for creating, deleting, or expanding a pool — it provides a map, a list, and properties. A request cannot bypass this: the only write operation in the storage layer at this level is changing dataset properties through filesystems-set.

Live pool methods go through a single node. At any given time, a pool has one serving node, and the request is relayed to it. An unavailable node therefore produces a relay error instead of pool properties. Workaround: view the pool table in the UI — it responds from the vStack Center database and remains available when the node is unavailable, but shows the last captured state.

Dataset properties are controlled by a feature flag. Until the filesystem_options feature is enabled in the cluster, the dataset properties form in the UI does not open, and filesystems-set returns an error.

There are no storage tiers. The platform does not support an "SSD cache plus hard disk capacity" layout, and a separate "slow" pool cannot be created either — see disk requirements in the vStack OS documentation.