Calculating inufficient resources for HA

[AllBlack]

Hi,

I was wondering what the best way is to calculate to resources needed for HA?

My HA seems to work pretty well but now I ran into a problem of starting VMs because

it would violate HA configuration. I have set not to power on VMs if it violates constraints.

Obviously this is what I want as HA is more important

than starting up VMs in this scenario.I would like to get a better idea to what amount of resources is required to guarantee HA?

Out of 46Ghz I had 28Ghz unreserved and out of 32Gb of Ram I had 11Gb unused. Yet my VM did

not fire up.

I can see that it possibly could be an issue of one server not being able to carry the load of the second server but am wondering how much more there is to it?

Cheers

[ac57846]

To calculate the HA capacity of your servers:

First the "slot" size for CPU and RAM: Take the largest reservation on any VM in the cluster, if no reservation is specified assume 256MB / 256 MHz. Remember to add the virtualisation overhead to the RAM reservation.

Next slots per host: Take the ESX server with the least resource available, how many "slots" capacity does that host have? Minimum host reservable Capacity / Slot size = Slots per host

Next slots per cluster: Subtract the Host Failure Capacity from the number of hosts in the cluster, multiply by the slots per host.

You now have the number of VMs you are allowed to run in your HA cluster under strict admition control.

As you may have noticed this is an extremely conservative equation & you can be very sure that the reservations will be well satisfied under the designed failure conditions. On the other hand the usable capacity of your cluster will be low, especially if you have a host with much less resource than other hosts in the cluster.

Al.

[AllBlack]

That would significantly reduce the amount of VMs I can use if I had one VM with a high reservation is it not?

Still somewhat confused, my slot size is 2Ghz/2Gb Ram and my lowest spec host is 22644Mhz/16Gb of Ram

To obtain slots per host, do I make a calculation each for CPU and RAM?

- 22644 GHz / 2Ghz for CPU , being 11 slots

- 16Gb/2Gb for RAM, being 8 slots

Obviously using each of these values would give me a different value. Not sure which one to pick.

I have two hosts, One for failover level. That would give me 11 or 8 VMs in this situation

Or do I have it completly wrong?

I have a 46Ghz/32 GB cluster, I would like to run more than that on it

[kjb007]

This is a good reason why reservations aren't a magic bullet. Are you sure you need a reservation on a particular vm? Is the vm running into memory contention that you need to guarantee it a certain amount of RAM? If not, lower or remove the reservation.

-KjB

[AllBlack]

I agree and I don't tend to specify any. Seems like one creeped in by converting a physical box.

[ac57846]

8 Slots per cluster! Very conservative and gives an unexpected result, especially in small clusters and where the VMs or hosts vary widely.

I would change HA to allow constraints violation, this will allow you to run more VMs.

[AllBlack]

Hey guys,

Something still not right and I am confused. Is HA actually looking at a reservation or the configured size when calculating?

I have no reservations. All machines use what they have been given upon creation, the configured size. I have one machine with a configured size of 2048Mb (running virtual center, used min. requirement) and that is the highest configured size of all.

The entire memory allocation is confusing me. What is best practice here? For example, I have a windows 2003 VM and it has a recommended size of 256Mb but the size as been set to 1024Mb. Is there any point in this and is this affecting HA? I am not the only one creating VMs on this cluster and some of my peers like to assign memory to the VMs as if it was a physical box

Using the formula handed to me above I would be able to use about 90VMs and still provide for HA.

I have 34 on host 1 and 15 on host 2 and I am unable to power on VMs. I removed constraint but that is not a good idea as HA is priority for us.

I am not sure were the problem lies at this stage. Current usage on cluster is:

CPU reservation 46491 Mhz / 16896 mhz used / 29595 mhz free

MEM reservation 29435 Mb / 17607 mb used / 11828 mb free (host 1 using 7.6 Gb, Host 2 using 11.3 Gb)

[kjb007]

It is configured for maximum reservation needed for powered on vm, it is in the resource management guide:[http://www.vmware.com/pdf/vi3_35/esx_3/r35/vi3_35_25_resource_mgmt.pdf] . Based on 49 vm's, there's probably a VM that has around 512 MB "reserved" for it, which would include guest memory + overhead. Remember also that a 64 bit vm uses slightly more overhead. You can also try and set the slot size a little smaller, to say 150 instead of the 256. It may increase the number of slots, again assuming that something doesn't have 512 reservation.

You can add these values to the HA advanced config: das.vmMemoryMinMB and das.vmCpuMinMhz. There are others that have used this and were able to get around strict admission control, to make it more like 3.0.x, as the admission control has is more strict now.

-KjB

[AllBlack]

No reservation to be found anywhere.

My memory allocations differ somewhat between VMS but most of them have 1024Mb set.

If I understand you correctly, there would be value in decreasing this to 512 or 256?

I am still confused what the best way is of determening this amount? Would 256 be enough?

Most applications need a lot more so to what extend is this parameter limiting what an app requires?

I am under the impression ESX gives and releases resources as needed So if my allocation is 256Mb

but a 1024mb is required would ESX grant that to this VM (assuming resources are available)?

Sorry for being somewhat ignorant, I am sure I will never forget once I grasp it

Cheers

[kjb007 · Accepted solution]

I would say don't touch any of your vm's. Add those two entries into your ha advanced config, and set that value to 150. The issue here is the HA config, how it handles the slot size by default, and not the way you are managing your vm's. Just add those two entries into ha, and you should be able to get over this error.

-KjB

[AllBlack]

Thanks, I will give that a go. I have some more questions regarding memory usage but will do so in a new thread.

cheers

[AllBlack]

That solved it indeed. Thanks

[DB_Cong]

Hello, Our HA has got HA issue after updating to "ESX 3.5 Update 1".

We have 2 ESX connecting with 1 iSCSI SAN, with HA & DRS enabled.

Following is the hardware:

ESX01: 1.6Ghz4core2CPU = 12.8Ghz;

2GB*8=16GB

ESX02: 2.3Ghz4core2CPU = 17.6Ghz

2GB*8=16GB

This error only occurs on ESX01.

We have verified the DNS and confirmed both HOST and PRT records are healthy.

It could not be resolved by reconfiguring HA, disabling DRS or disabling and enabling HA.

No CPU/Memory reservation is configured for any virtual machines.

The following 2 errors and 1 information generates very 1 hour:

1. HA agent on ESX01 in cluster ESX has an error. 2. Insufficient resources to satisfy HA failover level on cluster ESX. 3. Sufficent resource are available to satisfy HA failover level in cluster ESX. (30s later)

Refering to the links:

http://communities.vmware.com/message/939731#939731

http://communities.vmware.com/thread/145149

The error remains evenif I set the das.vmMemoryMinMB and das.vmCPUMinGhz to 128 and reconfigured HA on each host.

Your suggestion will be most appreciated.

Thanks so much...

[kjb007]

Looks like your ha agent has a problem. I would remove the host from the cluster, and then add it back in. That should uninstall and reinstall the ha agent, and you should be good to go.

-KjB

[DB_Cong]

Thanks kjb007, but those errors remain evenif I remove the hosts, reboot the hosts, and recreate the cluster.

Without configuring the das.vmMemoryMinMB and das.vmCpuMinMHz, the CPU and Memory reservation could been in the tab"Resource Allocation" as follows:

CPU Reservation: 30395 MHz

CPU Reservation Used: 0 MHz

CPU Unreserved: 30395 MHz

Memory Reservation: 29452 MB

Memory Reservation Uesed: 910 MB

Memory Unreserved: 28542 MB

After configuring the das.vmMemoryMinMB and das.vmCpuMinMHz as 128, same errors keep generating every hours.

After configuring the das.vmMemoryMinMB and das.vmCpuMinMHz as 50, the frequency of the error falls down to every 4 hours.

May I have your advice?

Thank you very much.

[kjb007]

How many vm's do you have and what do their resources look like (cpu/ram) and what OS are you running?

Look in your vpxd log to see what your calculated slot size is. I'll double check, but this should be in your vpxd log, or your vmware.log in your vm folder.

-KjB

[DB_Cong]

Thanks, I have 12 VMs(19GB) totally, and 9 VMs(13GB memory) are powered on.

They are all Windows Server 2003 and 1 of them is the x64(power on).

[java_cat33]

Hi - Did you manage to find the log that contains the slot size info? I can't find it in my vpxd logs.

[DB_Cong]

Neither do I.

[kjb007]

Check your vmware.log as well to see if slot size is generated at boot time. I know I've seen it, but I'm not sure which log.

-KjB