How does SIOC work ?

[#1]

What's the need of SIOC?

1. Share of datastore is provisioned on the basis of the number of ESXI hosts. if Host A has more number of VM's than another Host B. VM's on the Host B will have more shares of datastore than VM's on Host A.

2. It creates a situation when one VM monopolising the datastore by levelling out all requests for I/O that the datastore receives and leads to depriving others of that resource which will cause             performance degradation. This behaviour is called the noisy neighbour.

3. Storage I/O Control was introduced in Vsphere 4.1 and works only on block-based Virtual Machine File Systems (VMFS) data stores (iSCSI and Fibre Channel);. In vSphere 5, however, SIOC has been enhanced with support for NFS data stores.

SIOC stats collection mechanism

1. SIOC keep tracks of a performance metric called Normalized Latency, compares it with the congestion latency threshold every four seconds, and only kicks in if the Normalized Latency is above the threshold then SIOC gets kicked in.

SIOC calculates this Normalized Latency by factoring in latency values for all hosts and all VMs accessing the datastore, taking into account different I/O sizes (larger I/O sizes means higher observed latency than smaller I/O sizes).

2. From vSphere 5.1 , "stats only" mode is available to gather performance statistics. because SDRS uses the last 24 hours' average to make recommendations.Stats only mode is disabled due to the (significant) increase of log data into the vCenter database. but from ESXi 6.0, VMware has enabled IO metric collection for every datastore by default. This can cause some weird latency spikes on your storage system that can be very hard to isolate if you don’t know what you’re looking for or what might be causing it.

When does SIOC get trigger and what does it do?

1. When latency (between host and datastore as well VM and datastore)  and equal or higher than 30MS, and the statistics around this are computed every 4 seconds, the “datastore-wide disk scheduler” will determine which action to take to reduce the overall/average latency and increase fairness. ( we can set this latency threshold manually from 5ms to 100ms before 5.1 ). This feature was removed from 5.1 because when user set too low threshold then SIOC will start to throttle IOPS too early and decrease the throughput of the storage.

2. This is a  major enhancement in vSphere 5.1 is replacing the manual setting of the congestions threshold with a percentage of peak throughput that is automatically determined for each device by injecting IOs and observing both latency and throughput. When the throughput of the LUN starts to suffer this means that it reached its maximum and the latency corresponding to 90% of the maximum throughput is used as a threshold.

3. Disk shares, like most fairness mechanisms, do not kick in unless there is a congestion on the resource. As long as there is enough for everybody, there is nothing preventing any VM from consuming what it wants unless you specify a hard limit on the maximum number of IOPS.  

SIOC control throttles VMs IOPS by modifying the device queue depth on the host, forcing the VM to adjust its IOPS to match its configured disk shares in comparison to other VMs that share the same LUN from all hosts.

How to configure SIOC ?

Configuring Storage I/O Control is a two-step process:

1	Enable Storage I/O Control for the datastore.
2	Set the number of storage I/O shares and the upper limit of I/O operations per second (IOPS) allowed for each virtual machine.

By default, all virtual machine shares are set to Normal (1000) with unlimited IOPS.

1. If a virtual machine has more than one virtual disk, you must set the limit on all of its virtual disks. Otherwise, the limit will not be enforced for the virtual machine. In this case, the limit on the virtual machine is the aggregation of the limits for all virtual disks.

For example: - if you want to restrict some snapshot or backup application with 128Kb IOs to 5Mb per second, set a limit of 40 IOPS.

SIOC in ESXI 6.5

Vmware has come up with SIOC v2 and it's supported from 6.5

1. SIOC v2 in this release is based on vSphere API for I/O filters.SIOC v1 implementation is still supported in vSphere 6.5, so the current settings that you have configured for your VMs will work just fine if you choose not to use SIOC v2

2. The user will specify a policy for SIOC and the policy determines how many shares the VM is configured with. SIOC v2 is only supported with VMs that run on VMFS and NFS datastores.

Since SIOC v2 is now an I/O filter it gives us the ability to leverage Storage Policy Based Management (SPBM) to apply storage I/O control in this release.

In vSphere 6.5 Storage IO Control has been reimplemented by leveraging the VAIO framework.

What is VAIO framework?

VAIO stands for vSphere APIs for IO Filtering.

VMware provides certain categories of I/O filters that are installed on your ESXi hosts and vCenter Server

The supported types of filters include the following:

• Replication: Replicates all write I/O operations to an external target location, such as another host or cluster.
• Encryption: Offered by VMware. Provides encryption mechanisms for virtual machines. For more information, see the vSphere 6.5 Security new features.
• Caching: Implements a cache for virtual disk data. The filter can use a local flash storage device to cache the data and increase the IOPS and hardware utilization rates for the virtual disk. If you use the caching filter, you might need to configure a Virtual Flash Resource.
• Storage I/O control

VAIO framework enabled policy based management. This goes for caching, replication and indeed also QoS. Instead of configuring disks or VMs individually, you will now have the ability to specify configuration details in a VM Storage Policy and assign that policy to a VM or VMDK.

Should end user enable or disable the SIOC?

it depends on the environment where you are using this feature.

1.We need to enable where host VMs for multiple tenants (like public clouds) as it helps reduce the effect of a noisy neighbour trying to hog storage resources on the rest of the well-behaving VMs.

2. We need to disable the SIOC in auto tier storage configurations. Especially if it’s auto tiering at the block level. It’s also recommended to have IO metric collection disabled because it will get different results based on where the blocks on the datastore physically reside.

Points to be Noted about SIOC

• SIOC is disabled by default
• SIOC needs to be enabled on a per Datastore level
• SIOC only engages when a specific level of latency has been reached
• SIOC has a default latency threshold of 30MS
• SIOC uses an average latency across hosts
• SIOC uses disk shares to assign I/O queue slots
• SIOC does not use vCenter, except for enabling the feature
• SIOC is included only in the Enterprise Plus edition of vSphere

Although SIOC is enabled using a vCenter server, latency values are actually stored on the datasite itself. This allows SIOC to function, even when vCenter is down or becomes unavailable after SIOC is enabled.