13 Replies Latest reply on Dec 7, 2017 6:54 AM by John Wick

    Problem to deploy Guest Introspection when HA - Admission Control is active

    John Wick Novice

      Hi all,

      I have a problem when I deploy GuestIntrospection with Admission Control actived

      I have this message "Cannot deploy agent on host due to insufficient resources (CPU or Memory)"

      Our  hosts has 400Go of memory and 32 logicals CPU and I have just 3 VMs (NSXMANAGER, VC and PSC). That's it

      In my vCenter I have three hots, this is the chronology of my problem :

      1- The system deploy OVF template of the first GuestIntrospection(1)

      2- Make clone of the first one, twice

      3- Delete the VM GuestIntrospection(1)

      4- Reconfigure the VM of the GuestIntrospection(2)

      5- Reconfigure the VM of the GuestIntrospection(3)

      6- Power On the GuestIntrospection(2)

      7- Power On the GuestIntrospection(2)

      In the end I founf just 2 GuestIntrospection deployed, not the fisrt. I have to make resolve to deploy the first one.

      I don't know why the system delete the first one.

       

      I havn't this problem when I desabled the Control Admission

       

      Can some one help please !!!!!!

        • 1. Re: Problem to deploy Guest Introspection when HA - Admission Control is active
          Sreec Master
          Community WarriorsvExpert

          Is the issue specific to Guest Introspection VM ? What version of VC and ESXI are we running. I have seen an issue with 6.5 VC and 6.0 host , no matter how much amount of resources are available Admission control never work unless we switch to Slot/Host failure tolerate policy in HA .

          • 2. Re: Problem to deploy Guest Introspection when HA - Admission Control is active
            John Wick Novice

            VC 6.0

            ESXi 6.0

            And I don't know why I'v to disable Admission Control to success deployement of my Guest Introspection. I'v the same probleme when I want to deploy Trend Micro from NSX

            Screenshot - 2017-11-29 , 13_44_44.png

            2017-11-30 001.png

            • 3. Re: Problem to deploy Guest Introspection when HA - Admission Control is active
              Sreec Master
              vExpertCommunity Warriors

              Can you please post the HA admission control setting when you enable it ?  Also do check with other policies(Define host failover option) and let me know the results .

               

              • 4. Re: Problem to deploy Guest Introspection when HA - Admission Control is active
                John Wick Novice

                This is what I'v by default when I active the Control Admission.

                I also tested The option Reserved failover capacity with 2 Hosts, and I had the same problem.

                 

                Screenshot - 2017-12-01 , 09_59_05.png

                • 5. Re: Problem to deploy Guest Introspection when HA - Admission Control is active
                  Sreec Master
                  vExpertCommunity Warriors

                  So if i understand correctly when Admission Control is enabled you not able to deploy and power on any agent VM's and it doesn't make any difference with other HA policy? Did you tried testing with some dummy vm's ? Are they experiencing same problem. Also do post cluster resource summary when Admission control is enabled .I hope you have tried re configuring HA as a break fix method ?

                  • 6. Re: Problem to deploy Guest Introspection when HA - Admission Control is active
                    Bayu Wibowo Master
                    Community WarriorsvExpertUser Moderators

                    Hi, the error message says that you have insufficient CPU/memory resources and you have HA Admission Control set using slot size policy.

                    Please note when using slot policy, vSphere will use the largest powered on VM's CPU & Memory reservation as a slot size.

                    This vSphere HA Slot Policy Admission Control doc has a good explanation on how the Slot Policy Admission Control works with examples on the video.

                     

                    You mentioned that you have 32 logical CPU, let's assume one logical CPU is 1000GHz so total is 32GHz.

                    Imagine if your largest powered on VM has 16 vCPU reservation = 16GHz that means each host you only have 2 slots (can only powered on 2 VM)

                    You can check this slot size under Cluster > Monitor > vSphere HA > Summary

                    You may want to check the cluster resource reservation or VM level CPU/memory resource reservation

                     

                    The slot size policy does not normally suitable if your virtual machine size and reservation are varies, it is more suitable if you have standard size of virtual machine and reservation.

                    If you would like to still use vSphere HA for availability, I would suggest you to change the admission control policy to be based on percentage, see this doc Cluster Resources Percentage Admission Control

                    Bayu Wibowo | vExpert NSX, VCIX6-DCV/NV, Cisco Champion, AWS-SAA
                    Author of VMware NSX Cookbook http://bit.ly/NSXCookbook
                    https://nz.linkedin.com/in/bayupw | twitter @bayupw
                    • 7. Re: Problem to deploy Guest Introspection when HA - Admission Control is active
                      John Wick Novice

                      Hi Bayu,

                      Thanks for your replay.

                      I read the two links that you sent me, the video too.

                      I tried to apply the calculation on my cluster, here are the results obtained:

                       

                      Host 1 : CPU : 36Ghz ; Mem : 128Gb

                      Host 2 : CPU : 36Ghz ; Mem : 96Gb

                      Host 3 : CPU : 68Ghz; Mem : 415Gb

                       

                      The highest VM uses : CPU : 3040Mhz ; Mem : 16384

                       

                      If I apply the same calculation as in the video to obtain the slot size,

                      Slot Size : CPU : 3040Mhz ; Mem : 16384

                       

                      Host 1 : give me 128Gb / 16Gb = 7 slot

                      Host 2 : 96Gb / 16Gb  = 5 slot

                      Host 3 : 415 Gb /16Gb = 25 slot

                       

                      The total for the cluster is : 37slot

                       

                      I don't understand why it's give me this calculate for the slot : CPU = 32Mhz ; Memory : 16511MB

                      I was expecting to have :

                      Slot Size : CPU : 3040Mhz ; Mem : 16384

                       

                      The total slot size is correct : 36 slots in the cluster vs 37 calculated

                       

                      Screenshot - 2017-12-05 , 16_51_13.png

                       

                      All this brings me to my question :

                      Why the the GuestIntrospection is deleted when i deployed It with NSX manager ????

                      I have 9 available slot

                      And i deploy just three VMs,

                      The system delete the first one (GuestIntrospection(1)) and success to deploy the 2 others VMs (2 & 3).

                      Screenshot - 2017-11-28 , 16_42_50.png

                      • 8. Re: Problem to deploy Guest Introspection when HA - Admission Control is active
                        Bayu Wibowo Master
                        User ModeratorsCommunity WarriorsvExpert

                        Hi John,

                        The 32 Mhz is the default if you don't have any VM with CPU reservation as per the document:

                        If you have not specified a CPU reservation for a virtual machine, it is assigned a default value of 32MHz.

                         

                        I can see your slot size for memory is 16GB, the NSX Manager probably have this memory reservation.

                        The additional MB on the memory slot/reservation is probably some additional memory overhead, see this doc: Overhead Memory on Virtual Machines 

                        Based on the screenshot, this means 1 slot size is 16GB and any VMs that have memory less than 16GB requires 1 slot, anything more than that requires 2 slots or more.

                        3 slots are used which I think you have 3 VMs.

                        The available slots are 9 slots which means if you are planning to deploy VMs with less than 16GB memory, you can only deploy additional 9 VMs.

                         

                        As I mentioned previously, if you don't have standard/identical VM size but you have identical hosts, normally the percentage admission control is better.

                        If you would like to set the failover capacity manually, for three hosts it would be 33% if you would like to have N+1 availability

                        Bayu Wibowo | vExpert NSX, VCIX6-DCV/NV, Cisco Champion, AWS-SAA
                        Author of VMware NSX Cookbook http://bit.ly/NSXCookbook
                        https://nz.linkedin.com/in/bayupw | twitter @bayupw
                        • 9. Re: Problem to deploy Guest Introspection when HA - Admission Control is active
                          Bayu Wibowo Master
                          User ModeratorsvExpertCommunity Warriors

                          Any error messages?

                          Could you check how many slot size available after deployment?

                          Could you check which host that have GI VM deployed and not deployed?
                          If the GI VM always gets deleted on a same host, I think that specific host probably doesn't have enough slot size.

                          You can try to free up that host

                          Bayu Wibowo | vExpert NSX, VCIX6-DCV/NV, Cisco Champion, AWS-SAA
                          Author of VMware NSX Cookbook http://bit.ly/NSXCookbook
                          https://nz.linkedin.com/in/bayupw | twitter @bayupw
                          • 10. Re: Problem to deploy Guest Introspection when HA - Admission Control is active
                            John Wick Novice

                            Hi Bayu,

                            I have just to mention that the six VMs (3 Guest Introspection & 3 Trend Micro Deep Security) are not calculated into the used slots, the powered off machines too.

                            Used slot : 4

                            1- AEXPPSCATELIER

                            2- AEXPVCATELIER

                            3- NSXMANAGER

                            4- JBF_VM1_2007

                            I think because they are configured to not be moved to another host, since they protect the host himself.

                            I begin the test and will back with results

                            Thanks Screenshot - 2017-12-06 , 09_21_13.png

                            • 11. Re: Problem to deploy Guest Introspection when HA - Admission Control is active
                              John Wick Novice

                              Hi

                              I tested with fixed slot size

                              When I configure Fixed slot size CPU at 2000Mhz or 3000Mhz with 16531MB

                              I have the same problem, The system deploy just 2 GuestIntrospection and delete the first one.

                               

                              When I configure Fixed slot size CPU at 4000Mhz and 16531MB for memory

                              I'm able to deploy the three VMs (GI) without any problem.

                               

                              Other things, when I choose 25% CPU and 25% Memory. I'm able to deploy the three GI without any problem

                               

                              Screenshot - 2017-12-06 , 11_02_56.png

                               

                              Screenshot - 2017-12-06 , 11_19_11.png

                               

                              Screenshot - 2017-12-06 , 11_34_14.png

                               

                               

                               

                               

                               

                              Q : Could you check how many slot size available after deployment?

                              R : The same because the Introspection VMs are not considered in the Admin Control

                               

                              Q :If the GI VM always gets deleted on a same host, I think that specific host probably doesn't have enough slot size.

                              R : Not in the same Host, sometimes the first, sometimes the second

                               

                              Any suggestion ?????

                              • 12. Re: Problem to deploy Guest Introspection when HA - Admission Control is active
                                Bayu Wibowo Master
                                vExpertCommunity WarriorsUser Moderators

                                Any error message when the GI VM gets deleted?

                                Is there any background or reason to stay with slot size admission control and configure specific fixed slot size?

                                 

                                As I mentioned in previous reply, slot size does not normally suitable if you have varying VM sizes as the calculation might be inefficient.

                                I would suggest to change the HA admission control policy to be based on percentage

                                If the automatic calculation of percentage is not accurate, you can override and use 33% for a 3 hosts cluster

                                 

                                If you need more information about some design/architectural decision, these 2 blog posts are good references

                                High Availability Admission Control Setting and Policy | CloudXC

                                Example Architectural Decision – VMware HA – Percentage of Cluster resources reserved for HA | CloudXC

                                Bayu Wibowo | vExpert NSX, VCIX6-DCV/NV, Cisco Champion, AWS-SAA
                                Author of VMware NSX Cookbook http://bit.ly/NSXCookbook
                                https://nz.linkedin.com/in/bayupw | twitter @bayupw
                                • 13. Re: Problem to deploy Guest Introspection when HA - Admission Control is active
                                  John Wick Novice

                                  Any error message when the GI VM gets deleted?

                                  Yes, see the screenshot below

                                  Screenshot - 2017-12-07 , 09_37_32.png

                                  Screenshot - 2017-12-07 , 09_30_33.pngScreenshot - 2017-12-07 , 09_35_54.png

                                  Screenshot - 2017-12-07 , 09_35_41.png

                                  Screenshot - 2017-12-07 , 09_31_53.png

                                   

                                  Is there any background or reason to stay with slot size admission control and configure specific fixed slot size?

                                  No, Right now i'm abale to deploy the six VMs (GI & TrendMicro Deep Security) when I choose the % option. I want just to inderstand why I have this problem when I fix slot size to 2000Mhz or 3000Mhz with 16531Mb. Knowing that my GI VM has only 2CPU and 1024Mb of memory

                                   

                                  thank you very much for your amplication