1 2 Previous Next 19 Replies Latest reply on Aug 13, 2010 9:41 AM by FredPeterson

    Max no of vm per lun

    tom_brechet Novice

       

      Hi guys

       

       

      I seem to remember that VMware stated with ESX 2.x that the maximum Numbers of VMs per Lun is above 100 so I never cared about it. Now there is this "SAN System Design and Deployment Guide" from VMware, released March 2007 which states on page 166 that no more than 16 VM should be placed on a lun? I don't seem to be the only one remembering the old limits as in the HP "VMware Infrastructure 3, HP StorageWorks best practices" there is this statement: While VMware Infrastructure 3 supports up to 80 VMs per LUN for a "light" I/O load... 40 VMs per LUN is a conservative number.

       

       

      So, what is correct? Am I really that mistaken? 16 VM per LUN, is that possible?

       

       

      Would be glad to have an answer on that.

       

       

      Cheers

       

       

      Tom

       

       

       

       

       

       

       

       

       

       

       

          

       

       

       

       

       

        • 1. Re: Max no of vm per lun
          habibalby Expert

          Hi,

          Well, you can have up to 100 VMs per LUN, 32 VMs per LUN may often be acceptable. But that all depends on your needs / VMs I/Os.

           

          It's recommended to keep 8 ~ 14 machines per LUN.

          • 3. Re: Max no of vm per lun
            jhanekom Master

            You must have a different version of the Storageworks best practices document.  Mine specifically state on page 7:

             

            "Carefully determine the number of VMs you can place on a single LUN in your particular environment. Note that, with new VMware Infrastructure 3 capabilities such as iSCSI and NAS, this determination has become even more complex. While VMware Infrastructure 3 supports up to 100 VMs per LUN, 32 VMs per LUN may often be acceptable. However, HP recommends being more conservative and using 8 – 10 VMs per LUN. In a NAS or iSCSI environment, HP recommends using 8 – 16 VMs per LUN. If your VMs are not heavily impacting I/O, you may be able to exceed the conservative recommendations above."

             

            The document I have is dated "6/2006 -2" on the very last page.

            • 4. Re: Max no of vm per lun
              tom_brechet Novice

              Yes, my documentation is definitvely different then. Please see in the attached document page 8 second bullet in Chapter "General SAN connectivity". This seems to be a newer version dated february 2007.

              • 5. Re: Max no of vm per lun
                tom_brechet Novice

                >Hi,

                >Well, you can have up to 100 VMs per LUN, 32 VMs per LUN may often be acceptable. But that all depends on your needs / VMs I/Os.

                >It's recommended to keep 8 ~ 14 machines per LUN.

                 

                 

                Hi,

                 

                 

                Do you have an source for that? Just wondering as I seem to remember the same, never implemented it this way my LUNs where always sized for 10-20 VMs but now I read that i should place 16 VMs on a LUN

                 

                 

                Cheers

                 

                 

                Tom

                 

                 

                • 6. Re: Max no of vm per lun
                  habibalby Expert

                   

                  From HP StorageWorks Best Practice

                   

                   

                  http://h20219.www2.hp.com/ActiveAnswers/downloads/VMware3_StorageWorks_BestPractice.pdf

                   

                   

                  Page 7

                   

                   

                  General SAN connectivity

                  VMware Infrastructure 3 and HP StorageWorks storage combine to provide a robust solution for

                  consolidation, reliability, and disaster recovery. This section provides general guidelines for

                  connecting VMware Infrastructure 3 to an HP StorageWorks SAN.

                  • Ensure each LUN has the appropriate RAID level and storage characteristics for the VMs that will

                  use it.

                  • Carefully determine the number of VMs you can place on a single LUN in your particular

                  environment. Note that, with new VMware Infrastructure 3 capabilities such as iSCSI and NAS, this

                  determination has become even more complex.

                  While VMware Infrastructure 3 supports up to 100 VMs per LUN, 32 VMs per LUN may often be

                  acceptable. However, HP recommends being more conservative and using 8 - 10 VMs per LUN.

                  In a NAS or iSCSI environment, HP recommends using 8 - 16 VMs per LUN.

                  If your VMs are not heavily impacting I/O, you may be able to exceed the conservative

                  recommendations above.

                  • To optimize array performance, spread I/O loads over the available paths to the storage.

                  • HP suggests configuring one Virtual Mirror File System (VMFS) volume per LUN. With VMFS-3 you

                  no longer have to set accessibility.

                  • In an environment with HP BladeSystem servers, a shared diagnostic partition is appropriate.

                  • Local SCSI controllers and Fibre Channel HBAs can be allocated to the VMkernel. However, it may

                  be preferable to use local RAID solutions for the Service Console and Fibre Channel HBAs for

                  external storage such as SANs. 

                   

                   

                  • 7. Re: Max no of vm per lun
                    Texiwill Guru
                    vExpertUser Moderators

                    Hello,

                     

                    How many VMs per LUN depends on the storage device involved and the size of the VMs. A good algorithm for sizing a VM is:

                     

                    Size of VMDK * 2 (possible snapshots) + Size of Allocated Memory + 15MBs (sundry files) + 4GBs (logfile explosion) = Size of VM

                     

                    Assume a 24GB VMDK and 2 GB of Memory for the VM.....

                     

                    24 * 2 + 2 + .15 + 4 = 54.15 GBs per VM

                     

                    I you only have a 512GB LUN... The math states:

                     

                    Size of LUN * 0.80 = Useable Space on LUN / Size of VM = Total # of VMs (rounded down).

                     

                    512 * .8 / 54.15 = 7 VMs

                     

                    Some may argue that 24GBs for a snapshot in this case is unreasonable but I have seen 100GB snapshots because the backup software failed to properly delete the snapshot. Plus, logfiles fill up the disk quite a bit as well.  In an error situation you will burn through disk space with each logfile quite easily. If the VMFS fills up, things stop working.....

                     


                    Best regards,

                    Edward L. Haletky

                    VMware Communities User Moderator

                    ====

                    Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education. CIO Virtualization Blog: http://www.cio.com/blog/index/topic/168354, As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization

                    1 person found this helpful
                    • 8. Re: Max no of vm per lun
                      habibalby Expert

                       

                      Yes, I do agree entierly with Taxiwill. SAN Calculation for VMDK files is the best strategy and planning for your VMs. It's not only depends on the LUN Size and How many VMs per LUN? It depends on the Backup Strategy, Snapshot, logs. if they are stored to the same SAN Storage for Backing up to the Proxy Server and Tape Library. Blah Blah Blah.

                       

                       

                         

                       

                       

                      • 9. Re: Max no of vm per lun
                        tom_brechet Novice

                         

                        Hi Edward

                         

                         

                        Thanks for that formula. But what I need to know is probably exactly the opposite: Not how many VMs per given LUN size but how big a  LUN size for the VMs. I've had LUN sizes varying from 256 to 1GB in my past designs and all worked quite well. Now I have a customer with a 1GB LUN which has some trouble. This is when I try to find the info that existed back then (at least in ESX 2.x) that VMware supports over 100 VMs per LUN. While searching I only find this new to me info that one should never place more that 16 VMs on a LUN.

                         

                         

                        Am I the only one catched cold by this?

                         

                         

                        Cheers

                         

                         

                        Tom

                        P.S.: the consultant with 18 happy and one unhappy VI-Design customer

                         

                         

                        • 10. Re: Max no of vm per lun
                          Texiwill Guru
                          User ModeratorsvExpert

                          Hello,

                           

                          The math really does not change that much when you start to decide on how big a LUN you want..... The last calculation changes.

                           

                          BTW, it was not 100 VMs per LUN in ESX 2 days it was 192 files per LUN.... + 32 Files for every Extent added. And the more Files on the LUN the slower the LUN went. If for example there were 6 files per VM (2 VMDK, 1 vMotion log file, 1 suspend file, 2 REDO files), you could not put more than 32 VMs per non 'extent'ed VMFS. And most people cut that in half even so. So 100s of VMs, never, but 100s of files, perhaps. You could perhaps have 192 VMDKs, but that does not necessarily translate to 100s of VMs.... At least I never once suggested more than 16 VMs per LUN even in v2 days.

                           

                          For ESX v3, to calculate the size of a LUN based on a count of VMs, reverse the algorithm mentioned previously some:

                           

                          Size of VM * # of VMs / 0.8 = Size of LUN required

                           

                          Assume a 54.15GB VM, and 16 VMs.... = 1083 GBs or 1.083TBs of LUN space required.

                           

                          Remember you want to fill a LUN no more than 80% just in case.

                           

                          Also, it is often much better to have several smaller LUNs than one massive LUN. This will help with alleviate SCSI Reservation Conflicts. These conflicts are often what dictates the LUN count more than anything else. It depends on the storage used as well.

                           


                          Best regards,

                          Edward L. Haletky

                          VMware Communities User Moderator

                          ====

                          Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education. CIO Virtualization Blog: http://www.cio.com/blog/index/topic/168354, As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization

                          • 11. Re: Max no of vm per lun
                            williambishop Master

                             

                            I have luns that vary from 1 vm to 150 vm's. They are all scaled according to IO and the array they are connecting to.

                             

                             

                             

                             

                             

                            So the answer is ALWAYS...._.IT VARIES_.

                             

                             

                             

                             

                             

                             

                             

                             

                            • 12. Re: Max no of vm per lun
                              tom_brechet Novice

                               

                              Hi Edward

                               

                               

                              >BTW, it was not 100 VMs per LUN in ESX 2 days it was 192 files per LUN.... + 32 Files for every Extent added. 

                               

                               

                              Am still battling my war and found this info here http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1269

                               

                               

                              >Up to 100 virtual machines may share a VMFS-2 volume if they have less disk-intensive workloads, subject to other system constraints such as CPU and memory resources. 

                               

                               

                              Good to know I'm not completely off my rocker and do at least remember some things correctly

                               

                               

                              BTW: Got a statement from a Field-SE hat all these numbers are no limits but mere recommendations. The only limiting factor for VMware seems to be performance and thats not my question at the moment.  

                               

                               

                              Cheers

                               

                               

                              Tom

                               

                               

                               

                               

                               

                               

                               

                               

                              • 13. Re: Max no of vm per lun
                                Texiwill Guru
                                vExpertUser Moderators

                                Hello,

                                 

                                The max # of files available for VMFS-3 is ~31000, I actually think it is 31715 or something close to that.. Consider that there are at least 1 directory and 9 files per VM, but most likely 15 considering all the logfiles created, there can ~3100 VMs per VMFS-3. I would say this is the maximal count allowed, but you are ideally looking at ~1900 VMs. It is all based on file count and the average # of files/directory for a VM with a single VMDK is ~16.However with a 2TB limit on the size of a VMFS-3, VMDK;s may need to be .5GBs for this to work.

                                 


                                Best regards,

                                Edward L. Haletky

                                VMware Communities User Moderator

                                ====

                                Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education. CIO Virtualization Blog: http://www.cio.com/blog/index/topic/168354, As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization

                                1 person found this helpful
                                • 14. Re: Max no of vm per lun
                                  tom_brechet Novice

                                   

                                  Hi Edward

                                   

                                   

                                  Now we are talking. That statement or any other is exactly what I need from VMware. How many VMs they support on a single LUN. Be it 10 or 100 or 1000 or whatever.

                                   

                                   

                                  I've given you 2 helpful answers now as your post have indeed been the most infomative. I'd like to reserve the "correct answer" for whoever can point me to an official statement of VMware what the exact number is.

                                   

                                   

                                  Many thanks

                                   

                                   

                                  Tom

                                   

                                   

                                   

                                   

                                   

                                  1 2 Previous Next