VMware Networking Community
Yassir_bout
Enthusiast
Enthusiast
Jump to solution

Problem to deploy Guest Introspection when HA - Admission Control is active

Hi all,

I have a problem when I deploy GuestIntrospection with Admission Control actived

I have this message "Cannot deploy agent on host due to insufficient resources (CPU or Memory)"

Our  hosts has 400Go of memory and 32 logicals CPU and I have just 3 VMs (NSXMANAGER, VC and PSC). That's it Smiley Happy

In my vCenter I have three hots, this is the chronology of my problem :

1- The system deploy OVF template of the first GuestIntrospection(1)

2- Make clone of the first one, twice

3- Delete the VM GuestIntrospection(1)

4- Reconfigure the VM of the GuestIntrospection(2)

5- Reconfigure the VM of the GuestIntrospection(3)

6- Power On the GuestIntrospection(2)

7- Power On the GuestIntrospection(2)

In the end I founf just 2 GuestIntrospection deployed, not the fisrt. I have to make resolve to deploy the first one.

I don't know why the system delete the first one.

I havn't this problem when I desabled the Control Admission

Can some one help please !!!!!!

Reply
0 Kudos
1 Solution

Accepted Solutions
bayupw
Leadership
Leadership
Jump to solution

Hi, the error message says that you have insufficient CPU/memory resources and you have HA Admission Control set using slot size policy.

Please note when using slot policy, vSphere will use the largest powered on VM's CPU & Memory reservation as a slot size.

This vSphere HA Slot Policy Admission Control doc​ has a good explanation on how the Slot Policy Admission Control works with examples on the video.

You mentioned that you have 32 logical CPU, let's assume one logical CPU is 1000GHz so total is 32GHz.

Imagine if your largest powered on VM has 16 vCPU reservation = 16GHz that means each host you only have 2 slots (can only powered on 2 VM)

You can check this slot size under Cluster > Monitor > vSphere HA > Summary

pastedImage_2.png

You may want to check the cluster resource reservation or VM level CPU/memory resource reservation

pastedImage_3.png

The slot size policy does not normally suitable if your virtual machine size and reservation are varies, it is more suitable if you have standard size of virtual machine and reservation.

If you would like to still use vSphere HA for availability, I would suggest you to change the admission control policy to be based on percentage, see this doc Cluster Resources Percentage Admission Control

Bayu Wibowo | VCIX6-DCV/NV
Author of VMware NSX Cookbook http://bit.ly/NSXCookbook
https://github.com/bayupw/PowerNSX-Scripts
https://nz.linkedin.com/in/bayupw | twitter @bayupw

View solution in original post

Reply
0 Kudos
13 Replies
Sreec
VMware Employee
VMware Employee
Jump to solution

Is the issue specific to Guest Introspection VM ? What version of VC and ESXI are we running. I have seen an issue with 6.5 VC and 6.0 host , no matter how much amount of resources are available Admission control never work unless we switch to Slot/Host failure tolerate policy in HA .

Cheers,
Sree | VCIX-5X| VCAP-5X| VExpert 6x|Cisco Certified Specialist
Please KUDO helpful posts and mark the thread as solved if answered
Reply
0 Kudos
Yassir_bout
Enthusiast
Enthusiast
Jump to solution

VC 6.0

ESXi 6.0

And I don't know why I'v to disable Admission Control to success deployement of my Guest Introspection. I'v the same probleme when I want to deploy Trend Micro from NSX

Screenshot - 2017-11-29 , 13_44_44.png

2017-11-30 001.png

Reply
0 Kudos
Sreec
VMware Employee
VMware Employee
Jump to solution

Can you please post the HA admission control setting when you enable it ?  Also do check with other policies(Define host failover option) and let me know the results .

pastedImage_0.png

Cheers,
Sree | VCIX-5X| VCAP-5X| VExpert 6x|Cisco Certified Specialist
Please KUDO helpful posts and mark the thread as solved if answered
Reply
0 Kudos
Yassir_bout
Enthusiast
Enthusiast
Jump to solution

This is what I'v by default when I active the Control Admission.

I also tested The option Reserved failover capacity with 2 Hosts, and I had the same problem.

Screenshot - 2017-12-01 , 09_59_05.png

Reply
0 Kudos
Sreec
VMware Employee
VMware Employee
Jump to solution

So if i understand correctly when Admission Control is enabled you not able to deploy and power on any agent VM's and it doesn't make any difference with other HA policy? Did you tried testing with some dummy vm's ? Are they experiencing same problem. Also do post cluster resource summary when Admission control is enabled .I hope you have tried re configuring HA as a break fix method ?

Cheers,
Sree | VCIX-5X| VCAP-5X| VExpert 6x|Cisco Certified Specialist
Please KUDO helpful posts and mark the thread as solved if answered
Reply
0 Kudos
bayupw
Leadership
Leadership
Jump to solution

Hi, the error message says that you have insufficient CPU/memory resources and you have HA Admission Control set using slot size policy.

Please note when using slot policy, vSphere will use the largest powered on VM's CPU & Memory reservation as a slot size.

This vSphere HA Slot Policy Admission Control doc​ has a good explanation on how the Slot Policy Admission Control works with examples on the video.

You mentioned that you have 32 logical CPU, let's assume one logical CPU is 1000GHz so total is 32GHz.

Imagine if your largest powered on VM has 16 vCPU reservation = 16GHz that means each host you only have 2 slots (can only powered on 2 VM)

You can check this slot size under Cluster > Monitor > vSphere HA > Summary

pastedImage_2.png

You may want to check the cluster resource reservation or VM level CPU/memory resource reservation

pastedImage_3.png

The slot size policy does not normally suitable if your virtual machine size and reservation are varies, it is more suitable if you have standard size of virtual machine and reservation.

If you would like to still use vSphere HA for availability, I would suggest you to change the admission control policy to be based on percentage, see this doc Cluster Resources Percentage Admission Control

Bayu Wibowo | VCIX6-DCV/NV
Author of VMware NSX Cookbook http://bit.ly/NSXCookbook
https://github.com/bayupw/PowerNSX-Scripts
https://nz.linkedin.com/in/bayupw | twitter @bayupw
Reply
0 Kudos
Yassir_bout
Enthusiast
Enthusiast
Jump to solution

Hi Bayu,

Thanks for your replay.

I read the two links that you sent me, the video too.

I tried to apply the calculation on my cluster, here are the results obtained:

Host 1 : CPU : 36Ghz ; Mem : 128Gb

Host 2 : CPU : 36Ghz ; Mem : 96Gb

Host 3 : CPU : 68Ghz; Mem : 415Gb

The highest VM uses : CPU : 3040Mhz ; Mem : 16384

If I apply the same calculation as in the video to obtain the slot size,

Slot Size : CPU : 3040Mhz ; Mem : 16384

Host 1 : give me 128Gb / 16Gb = 7 slot

Host 2 : 96Gb / 16Gb  = 5 slot

Host 3 : 415 Gb /16Gb = 25 slot

The total for the cluster is : 37slot

I don't understand why it's give me this calculate for the slot : CPU = 32Mhz ; Memory : 16511MB

I was expecting to have :

Slot Size : CPU : 3040Mhz ; Mem : 16384

The total slot size is correct : 36 slots in the cluster vs 37 calculated

Screenshot - 2017-12-05 , 16_51_13.png

All this brings me to my question :

Why the the GuestIntrospection is deleted when i deployed It with NSX manager ????

I have 9 available slot

And i deploy just three VMs,

The system delete the first one (GuestIntrospection(1)) and success to deploy the 2 others VMs (2 & 3).

Screenshot - 2017-11-28 , 16_42_50.png

Reply
0 Kudos
bayupw
Leadership
Leadership
Jump to solution

Hi John,

The 32 Mhz is the default if you don't have any VM with CPU reservation as per the document:

If you have not specified a CPU reservation for a virtual machine, it is assigned a default value of 32MHz.

I can see your slot size for memory is 16GB, the NSX Manager probably have this memory reservation.

The additional MB on the memory slot/reservation is probably some additional memory overhead, see this doc: Overhead Memory on Virtual Machines 

Based on the screenshot, this means 1 slot size is 16GB and any VMs that have memory less than 16GB requires 1 slot, anything more than that requires 2 slots or more.

3 slots are used which I think you have 3 VMs.

The available slots are 9 slots which means if you are planning to deploy VMs with less than 16GB memory, you can only deploy additional 9 VMs.

As I mentioned previously, if you don't have standard/identical VM size but you have identical hosts, normally the percentage admission control is better.

If you would like to set the failover capacity manually, for three hosts it would be 33% if you would like to have N+1 availability

pastedImage_0.png

Bayu Wibowo | VCIX6-DCV/NV
Author of VMware NSX Cookbook http://bit.ly/NSXCookbook
https://github.com/bayupw/PowerNSX-Scripts
https://nz.linkedin.com/in/bayupw | twitter @bayupw
Reply
0 Kudos
bayupw
Leadership
Leadership
Jump to solution

Any error messages?

Could you check how many slot size available after deployment?

Could you check which host that have GI VM deployed and not deployed?
If the GI VM always gets deleted on a same host, I think that specific host probably doesn't have enough slot size.

You can try to free up that host

Bayu Wibowo | VCIX6-DCV/NV
Author of VMware NSX Cookbook http://bit.ly/NSXCookbook
https://github.com/bayupw/PowerNSX-Scripts
https://nz.linkedin.com/in/bayupw | twitter @bayupw
Reply
0 Kudos
Yassir_bout
Enthusiast
Enthusiast
Jump to solution

Hi Bayu,

I have just to mention that the six VMs (3 Guest Introspection & 3 Trend Micro Deep Security) are not calculated into the used slots, the powered off machines too.

Used slot : 4

1- AEXPPSCATELIER

2- AEXPVCATELIER

3- NSXMANAGER

4- JBF_VM1_2007

I think because they are configured to not be moved to another host, since they protect the host himself.

I begin the test and will back with results

Thanks Smiley Happy Screenshot - 2017-12-06 , 09_21_13.png

Reply
0 Kudos
Yassir_bout
Enthusiast
Enthusiast
Jump to solution

Hi

I tested with fixed slot size

When I configure Fixed slot size CPU at 2000Mhz or 3000Mhz with 16531MB

I have the same problem, The system deploy just 2 GuestIntrospection and delete the first one.

When I configure Fixed slot size CPU at 4000Mhz and 16531MB for memory

I'm able to deploy the three VMs (GI) without any problem.

Other things, when I choose 25% CPU and 25% Memory. I'm able to deploy the three GI without any problem

Screenshot - 2017-12-06 , 11_02_56.png

Screenshot - 2017-12-06 , 11_19_11.png

Screenshot - 2017-12-06 , 11_34_14.png

Q : Could you check how many slot size available after deployment?

R : The same because the Introspection VMs are not considered in the Admin Control

Q :If the GI VM always gets deleted on a same host, I think that specific host probably doesn't have enough slot size.

R : Not in the same Host, sometimes the first, sometimes the second

Any suggestion ?????

Reply
0 Kudos
bayupw
Leadership
Leadership
Jump to solution

Any error message when the GI VM gets deleted?

Is there any background or reason to stay with slot size admission control and configure specific fixed slot size?

As I mentioned in previous reply, slot size does not normally suitable if you have varying VM sizes as the calculation might be inefficient.

I would suggest to change the HA admission control policy to be based on percentage

If the automatic calculation of percentage is not accurate, you can override and use 33% for a 3 hosts cluster

pastedImage_0.png

If you need more information about some design/architectural decision, these 2 blog posts are good references

High Availability Admission Control Setting and Policy | CloudXC

Example Architectural Decision – VMware HA – Percentage of Cluster resources reserved for HA | Cloud...

Bayu Wibowo | VCIX6-DCV/NV
Author of VMware NSX Cookbook http://bit.ly/NSXCookbook
https://github.com/bayupw/PowerNSX-Scripts
https://nz.linkedin.com/in/bayupw | twitter @bayupw
Reply
0 Kudos
Yassir_bout
Enthusiast
Enthusiast
Jump to solution

Any error message when the GI VM gets deleted?

Yes, see the screenshot below

Screenshot - 2017-12-07 , 09_37_32.png

Screenshot - 2017-12-07 , 09_30_33.pngScreenshot - 2017-12-07 , 09_35_54.png

Screenshot - 2017-12-07 , 09_35_41.png

Screenshot - 2017-12-07 , 09_31_53.png

Is there any background or reason to stay with slot size admission control and configure specific fixed slot size?

No, Right now i'm abale to deploy the six VMs (GI & TrendMicro Deep Security) when I choose the % option. I want just to inderstand why I have this problem when I fix slot size to 2000Mhz or 3000Mhz with 16531Mb. Knowing that my GI VM has only 2CPU and 1024Mb of memory

thank you very much for your amplication

Reply
0 Kudos