This is a follow up to the article I posted about a year ago that describes new IOPS management functionality in VMware Cloud Director (VCD) 10.2.
Storage IOPS is next to compute, networking and storage capacity a limited resource service providers want to manage in order to fairly share underlying physical resources in a multitenant environment.
As was described in the original article VCD supported storage IOPS management however the feature was quite hidden and available only via API. The recent release of VMware Cloud Director not only fully exposes the functionality in the UI but also adds some new functionality. Let’s dive into it.
There are two main mechanisms now how you can manage IOPS.
vCenter Server managed IOPS
This mechanism relies on setting IOPS limits at storage policy level directly in vCenter Server. That is possible with host based and with vSAN based storage policies. This mechanism is quite simple – when a VM disk is provisioned to such IOPS limited storage policy it will inherit the IOPS limit – a constant number per policy. You will not be able to set proportional IOPS based on disk capacity.
I would recommend using such mechanism only if you want to avoid noisy neighbors. The concept is not new, VCD could use such vSAN policies for some time and host based policies were already supported in VCD 10.1. The only difference is that now in 10.2 the tenant will see the
limit reservation set at VM disk level but will not be able to change it.
VCD Managed IOPS
This is much more sophisticated mechanism where you can really manage IOPS as pool of available capacity that you slice and allocate to tenant Org VDCs. This is the mechanism that was until now only available via API.
You will start by tagging your datastores with their IOPS capacity – that has not changed and still must be done from within VC via custom properties.
At Provider VDC level you can then create IOPS managed storage policies and define their service level in terms of disk IOPS defaults, maximums or IOPS allocation based on disk size (0 means unlimited).
This storage policy configuration can be inherited or overridden at Org VDC level. This is big improvement compared to the old approach where you had to create such storage policies always at Org VDC level.
Another new thing is that you can disable IOPS placement mechanism for such storage policy. This is useful in case you want to use Datastore Clusters. VCD will no longer try to place each virtual disk based on a particular datastore available IOPS. The placement decision is instead done by vCenter Server – you should therefore enable Storage DRS with I/O balancing automation. There is no need in such case to tag individual datastores in VC with their IOPS capacity.
Some of the old caveats still apply:
- Disk IOPS can be assigned only to regular VMs or named (independent) disks, not to VM templates.
- The disk IOPS will be always allocated against the Org VDC storage profile even if the VM is powered-off. This means the cloud provider can oversubscribe IOPS at the provider VDC storage profile level.
- System administrator can override IOPS limits when deploying/editing tenant VMs in the system context.
9 thoughts on “VMware Cloud Director – Storage IOPS Management – Part II”
Thank you for posting, this is really interesting.
These are great enhancements at VCD layer which definitely gives better control to service providers. However I see that these values will be visible to Organization administrators. Do you know how we can hide this values to customer.
At the moment if customer goes to details of the storage policies assigned to him he can see all these IOPs related fields but can be hide them?
Unfortunatelly that is not possible.
I have followed your guide but something seems not to work.
We have activated datastore clusters with IO balancing and put on fully automatic so vCenter does the placement.
We have disabled “impact placement ” in the storage policy on the tenant.
When changing the storage policy of a vm, it says
out of 1 candidate hubs: 1 hubs eliminated because: Unable to find valid Datastore Cluster for the following Items when placing VirtualMachine ”
It only seems to work when we disable Storage DRS and tag the datastores with iopsCapacity and some value
I ‘ve a storage policy vSAN with a IOPS Limit.
vSAN doesn’t support SIOC;
So when I encrypt a VM it’s seems that the iops limit is by passed..
Is there a way with vCD to limit the iops for that kind of VM (Cryted and stored on vSAN) ?
Yes, you would use VC managed IOPS as mentioned in the begining of the article.
Sorry but with following storage policy rules, IOPS are not limited for my encrypted VM on vsan..
Encryption > Default encryption properties
Name Default encryption properties
Description Storage policy component for VM and virtual disk encryption
Provider VMware VM Encryption
Allow I/O filters before encryption False
#Rule-set 1: VSAN
Storage Type VSAN
Site disaster tolerance Dual site mirroring (stretched cluster)
Failures to tolerate No data redundancy
Number of disk stripes per object 1
IOPS limit for object 3500
Object space reservation Thin provisioning
Flash read cache reservation 0%
Disable object checksum No
Force provisioning No
Faced a big problem when using the Disk IOPS Per GB Max setting when using a VM with 8 to 20 GB disk that generate 600-800 IOPS. They start to slow down and run into IOPS.Can the following approach be implemented?
SSD Tier-Gold – performance 20 IOPS per 1 GB (MGP – 1000 IOPS)
The minimum guaranteed performance (MGP) of a virtual disk is independent of its size.
10 GB virtual disk with Tier-Gold SSD policy
P = 10 (GB) * 20 = 200 (IOPS)
200 < MGP for SSD Tier-Gold 1000 IOPS
The virtual disk will be guaranteed 1000 IOPS
Edit Storage Policy Settings\Disk IOPS Per GB Max – 20 IOPS
Do you know why the VCD IOPS Limiting are based on I/O reservation and not just I/O Limit. With this setting enabled all VCD disks get 100% reservation. From working with VMware on this is not possible to set the IOPS Limit to a higher value than the IOPS Reservation. I wonder wht the reason behind this is as setting the IOPs Limit using vCenter does not set a I/O reservation.