VMware

This Question is Possibly Answered

1 "correct" answer available (10 pts)
1 2 Previous Next 17 Replies Last post: May 29, 2008 7:55 AM by williambishop  

Max no of vm per lun posted: May 20, 2008 10:59 PM

Click to view tom_brechet's profile Novice 23 posts since
Sep 15, 2005

Hi guys

I seem to remember that VMware stated with ESX 2.x that the maximum Numbers of VMs per Lun is above 100 so I never cared about it. Now there is this "SAN System Design and Deployment Guide" from VMware, released March 2007 which states on page 166 that no more than 16 VM should be placed on a lun? I don't seem to be the only one remembering the old limits as in the HP "VMware Infrastructure 3, HP StorageWorks best practices" there is this statement: While VMware Infrastructure 3 supports up to 80 VMs per LUN for a "light" I/O load... 40 VMs per LUN is a conservative number.

So, what is correct? Am I really that mistaken? 16 VM per LUN, is that possible?

Would be glad to have an answer on that.

Cheers

Tom


Re: Max no of vm per lun

1. May 20, 2008 11:55 PM in response to: tom_brechet
Click to view habibalby's profile Hot Shot 251 posts since
Jan 6, 2006
Hi,
Well, you can have up to 100 VMs per LUN, 32 VMs per LUN may often be acceptable. But that all depends on your needs / VMs I/Os.

It's recommended to keep 8 ~ 14 machines per LUN.

Re: Max no of vm per lun

2. May 20, 2008 11:57 PM in response to: habibalby
Click to view habibalby's profile Hot Shot 251 posts since
Jan 6, 2006

Re: Max no of vm per lun

3. May 21, 2008 12:03 AM in response to: tom_brechet
Click to view jhanekom's profile Master 1,085 posts since
May 13, 2005
You must have a different version of the Storageworks best practices document. Mine specifically state on page 7:

"Carefully determine the number of VMs you can place on a single LUN in your particular environment. Note that, with new VMware Infrastructure 3 capabilities such as iSCSI and NAS, this determination has become even more complex. While VMware Infrastructure 3 supports up to 100 VMs per LUN, 32 VMs per LUN may often be acceptable. However, HP recommends being more conservative and using 8 – 10 VMs per LUN. In a NAS or iSCSI environment, HP recommends using 8 – 16 VMs per LUN. If your VMs are not heavily impacting I/O, you may be able to exceed the conservative recommendations above."

The document I have is dated "6/2006 -2" on the very last page.

Re: Max no of vm per lun

6. May 21, 2008 4:31 AM in response to: tom_brechet
Click to view habibalby's profile Hot Shot 251 posts since
Jan 6, 2006

From HP StorageWorks Best Practice

http://h20219.www2.hp.com/ActiveAnswers/downloads/VMware3_StorageWorks_BestPractice.pdf

Page 7

General SAN connectivity
VMware Infrastructure 3 and HP StorageWorks storage combine to provide a robust solution for
consolidation, reliability, and disaster recovery. This section provides general guidelines for
connecting VMware Infrastructure 3 to an HP StorageWorks SAN.
• Ensure each LUN has the appropriate RAID level and storage characteristics for the VMs that will
use it.
• Carefully determine the number of VMs you can place on a single LUN in your particular
environment. Note that, with new VMware Infrastructure 3 capabilities such as iSCSI and NAS, this
determination has become even more complex.
While VMware Infrastructure 3 supports up to 100 VMs per LUN, 32 VMs per LUN may often be
acceptable. However, HP recommends being more conservative and using 8 - 10 VMs per LUN.
In a NAS or iSCSI environment, HP recommends using 8 - 16 VMs per LUN.
If your VMs are not heavily impacting I/O, you may be able to exceed the conservative
recommendations above.
• To optimize array performance, spread I/O loads over the available paths to the storage.
• HP suggests configuring one Virtual Mirror File System (VMFS) volume per LUN. With VMFS-3 you
no longer have to set accessibility.
• In an environment with HP BladeSystem servers, a shared diagnostic partition is appropriate.
• Local SCSI controllers and Fibre Channel HBAs can be allocated to the VMkernel. However, it may
be preferable to use local RAID solutions for the Service Console and Fibre Channel HBAs for
external storage such as SANs.

Re: Max no of vm per lun

7. May 21, 2008 6:12 AM in response to: habibalby
Click to view Texiwill's profile Guru 10,213 posts since
Jan 13, 2004
Hello,

How many VMs per LUN depends on the storage device involved and the size of the VMs. A good algorithm for sizing a VM is:

Size of VMDK * 2 (possible snapshots) + Size of Allocated Memory + 15MBs (sundry files) + 4GBs (logfile explosion) = Size of VM

Assume a 24GB VMDK and 2 GB of Memory for the VM.....

24 * 2 + 2 + .15 + 4 = 54.15 GBs per VM

I you only have a 512GB LUN... The math states:

Size of LUN * 0.80 = Useable Space on LUN / Size of VM = Total # of VMs (rounded down).

512 * .8 / 54.15 = 7 VMs

Some may argue that 24GBs for a snapshot in this case is unreasonable but I have seen 100GB snapshots because the backup software failed to properly delete the snapshot. Plus, logfiles fill up the disk quite a bit as well. In an error situation you will burn through disk space with each logfile quite easily. If the VMFS fills up, things stop working.....


Best regards,
Edward L. Haletky
VMware Communities User Moderator
====
Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education. CIO Virtualization Blog: http://www.cio.com/blog/index/topic/168354, As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization

Re: Max no of vm per lun

8. May 21, 2008 10:48 PM in response to: Texiwill
Click to view habibalby's profile Hot Shot 251 posts since
Jan 6, 2006

Yes, I do agree entierly with Taxiwill. SAN Calculation for VMDK files is the best strategy and planning for your VMs. It's not only depends on the LUN Size and How many VMs per LUN? It depends on the Backup Strategy, Snapshot, logs. if they are stored to the same SAN Storage for Backing up to the Proxy Server and Tape Library. Blah Blah Blah.


Re: Max no of vm per lun

10. May 22, 2008 7:21 AM in response to: tom_brechet
Click to view Texiwill's profile Guru 10,213 posts since
Jan 13, 2004
Hello,

The math really does not change that much when you start to decide on how big a LUN you want..... The last calculation changes.

BTW, it was not 100 VMs per LUN in ESX 2 days it was 192 files per LUN.... + 32 Files for every Extent added. And the more Files on the LUN the slower the LUN went. If for example there were 6 files per VM (2 VMDK, 1 vMotion log file, 1 suspend file, 2 REDO files), you could not put more than 32 VMs per non 'extent'ed VMFS. And most people cut that in half even so. So 100s of VMs, never, but 100s of files, perhaps. You could perhaps have 192 VMDKs, but that does not necessarily translate to 100s of VMs.... At least I never once suggested more than 16 VMs per LUN even in v2 days.

For ESX v3, to calculate the size of a LUN based on a count of VMs, reverse the algorithm mentioned previously some:

Size of VM * # of VMs / 0.8 = Size of LUN required

Assume a 54.15GB VM, and 16 VMs.... = 1083 GBs or 1.083TBs of LUN space required.

Remember you want to fill a LUN no more than 80% just in case.

Also, it is often much better to have several smaller LUNs than one massive LUN. This will help with alleviate SCSI Reservation Conflicts. These conflicts are often what dictates the LUN count more than anything else. It depends on the storage used as well.


Best regards,
Edward L. Haletky
VMware Communities User Moderator
====
Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education. CIO Virtualization Blog: http://www.cio.com/blog/index/topic/168354, As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization

Re: Max no of vm per lun

11. May 22, 2008 7:45 AM in response to: habibalby
Click to view williambishop's profile Master 1,159 posts since
Mar 9, 2006

I have luns that vary from 1 vm to 150 vm's. They are all scaled according to IO and the array they are connecting to.

So the answer is ALWAYS.....IT VARIES.


Re: Max no of vm per lun

13. May 27, 2008 8:09 AM in response to: tom_brechet
Click to view Texiwill's profile Guru 10,213 posts since
Jan 13, 2004
Hello,

The max # of files available for VMFS-3 is ~31000, I actually think it is 31715 or something close to that.. Consider that there are at least 1 directory and 9 files per VM, but most likely 15 considering all the logfiles created, there can ~3100 VMs per VMFS-3. I would say this is the maximal count allowed, but you are ideally looking at ~1900 VMs. It is all based on file count and the average # of files/directory for a VM with a single VMDK is ~16.However with a 2TB limit on the size of a VMFS-3, VMDK;s may need to be .5GBs for this to work.


Best regards,
Edward L. Haletky
VMware Communities User Moderator
====
Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education. CIO Virtualization Blog: http://www.cio.com/blog/index/topic/168354, As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization

VMware Developer

SDKs, APIs, Videos, Learn and much more in the Developer community.

Learn More

Developer Sample Code

Increase your developer productivity with VMware API sample code.

Learn More

VMworld Sessions & Labs

Online access to the latest VMworld Sessions & Labs and online services.

Learn more

Purchase PSO Credits Online

Purchase credits to redeem training and consulting services online.

Buy Now

Community Hardware Software

View reported configurations or report your own.

Learn More

VMware vSphere

Come witness the next giant leap in virtualization.

Register Today

Communities