VMware Communities: Message List - Max no of vm per lun

Re: Max no of vm per lun

williambishop — Thu, 29 May 2008 14:55:11 GMT

Again, those numbers are not fixed by vmware itself. You'll note that most of the limit numbers were from the storageworks best practice guide....Which is only one array of many. Vmware prefers you not to overload the luns....but it varies according to how strong the array you are using. A symmetrix is easily capable of 150+ vm's per lun.

Re: Max no of vm per lun

eagleh — Thu, 29 May 2008 14:29:30 GMT

"Carefully determine the number of VMs you can place on a single LUN in your particular environment. Note that, with new VMware Infrastructure 3 capabilities such as iSCSI and NAS, this determination has become even more complex. While VMware Infrastructure 3 supports up to 100 VMs per LUN, 32 VMs per LUN may often be acceptable. However, HP recommends being more conservative and using 8 - 10 VMs per LUN. In a NAS or iSCSI environment, HP recommends using 8 - 16 VMs per LUN. If your VMs are not heavily impacting I/O, you may be able to exceed the conservative recommendations above." ---- <VMWare3_StorageWorks_BestPractice.pdf>

Re: Max no of vm per lun

Texiwill — Tue, 27 May 2008 15:11:05 GMT

Hello,

Unforunately there is no exact number. It is a combination of the maximum vCPUs per host, and the size of the VMDKs in use, plus the limit of 2TB VMFS-3.

Best regards,
Edward L. Haletky
VMware Communities User Moderator
====
Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education. CIO Virtualization Blog: http://www.cio.com/blog/index/topic/168354, As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization

Re: Max no of vm per lun

Texiwill — Tue, 27 May 2008 14:56:55 GMT

Hello,

The max # of files available for VMFS-3 is ~31000, I actually think it is 31715 or something close to that.. Consider that there are at least 1 directory and 9 files per VM, but most likely 15 considering all the logfiles created, there can ~3100 VMs per VMFS-3. I would say this is the maximal count allowed, but you are ideally looking at ~1900 VMs. It is all based on file count and the average # of files/directory for a VM with a single VMDK is ~16.However with a 2TB limit on the size of a VMFS-3, VMDK;s may need to be .5GBs for this to work.

Best regards,
Edward L. Haletky
VMware Communities User Moderator
====
Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education. CIO Virtualization Blog: http://www.cio.com/blog/index/topic/168354, As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization

Re: Max no of vm per lun

tom_brechet — Tue, 27 May 2008 15:03:24 GMT

Hi Edward

Now we are talking. That statement or any other is exactly what I need from VMware. How many VMs they support on a single LUN. Be it 10 or 100 or 1000 or whatever.

I've given you 2 helpful answers now as your post have indeed been the most infomative. I'd like to reserve the "correct answer" for whoever can point me to an official statement of VMware what the exact number is.

Many thanks

Tom

Re: Max no of vm per lun

tom_brechet — Mon, 26 May 2008 13:56:57 GMT

Hi Edward

BTW, it was not 100 VMs per LUN in ESX 2 days it was 192 files per LUN.... + 32 Files for every Extent added.

Am still battling my war and found this info here http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1269

Up to 100 virtual machines may share a VMFS-2 volume if they have less disk-intensive workloads, subject to other system constraints such as CPU and memory resources.

Good to know I'm not completely off my rocker and do at least remember some things correctly

BTW: Got a statement from a Field-SE hat all these numbers are no limits but mere recommendations. The only limiting factor for VMware seems to be performance and thats not my question at the moment.

Cheers

Tom

Re: Max no of vm per lun

williambishop — Thu, 22 May 2008 14:45:58 GMT

I have luns that vary from 1 vm to 150 vm's. They are all scaled according to IO and the array they are connecting to.

So the answer is ALWAYS.....IT VARIES.

Re: Max no of vm per lun

Texiwill — Thu, 22 May 2008 14:21:12 GMT

Hello,

The math really does not change that much when you start to decide on how big a LUN you want..... The last calculation changes.

BTW, it was not 100 VMs per LUN in ESX 2 days it was 192 files per LUN.... + 32 Files for every Extent added. And the more Files on the LUN the slower the LUN went. If for example there were 6 files per VM (2 VMDK, 1 vMotion log file, 1 suspend file, 2 REDO files), you could not put more than 32 VMs per non 'extent'ed VMFS. And most people cut that in half even so. So 100s of VMs, never, but 100s of files, perhaps. You could perhaps have 192 VMDKs, but that does not necessarily translate to 100s of VMs.... At least I never once suggested more than 16 VMs per LUN even in v2 days.

For ESX v3, to calculate the size of a LUN based on a count of VMs, reverse the algorithm mentioned previously some:

Size of VM * # of VMs / 0.8 = Size of LUN required

Assume a 54.15GB VM, and 16 VMs.... = 1083 GBs or 1.083TBs of LUN space required.

Remember you want to fill a LUN no more than 80% just in case.

Also, it is often much better to have several smaller LUNs than one massive LUN. This will help with alleviate SCSI Reservation Conflicts. These conflicts are often what dictates the LUN count more than anything else. It depends on the storage used as well.

Best regards,
Edward L. Haletky
VMware Communities User Moderator
====
Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education. CIO Virtualization Blog: http://www.cio.com/blog/index/topic/168354, As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization

Re: Max no of vm per lun

tom_brechet — Thu, 22 May 2008 06:00:35 GMT

Hi Edward

Thanks for that formula. But what I need to know is probably exactly the opposite: Not how many VMs per given LUN size but how big a LUN size for the VMs. I've had LUN sizes varying from 256 to 1GB in my past designs and all worked quite well. Now I have a customer with a 1GB LUN which has some trouble. This is when I try to find the info that existed back then (at least in ESX 2.x) that VMware supports over 100 VMs per LUN. While searching I only find this new to me info that one should never place more that 16 VMs on a LUN.

Am I the only one catched cold by this?

Cheers

Tom
P.S.: the consultant with 18 happy and one unhappy VI-Design customer

Re: Max no of vm per lun

habibalby — Thu, 22 May 2008 05:48:39 GMT

Yes, I do agree entierly with Taxiwill. SAN Calculation for VMDK files is the best strategy and planning for your VMs. It's not only depends on the LUN Size and How many VMs per LUN? It depends on the Backup Strategy, Snapshot, logs. if they are stored to the same SAN Storage for Backing up to the Proxy Server and Tape Library. Blah Blah Blah.

Re: Max no of vm per lun

Texiwill — Wed, 21 May 2008 13:12:12 GMT

Hello,

How many VMs per LUN depends on the storage device involved and the size of the VMs. A good algorithm for sizing a VM is:

Size of VMDK * 2 (possible snapshots) + Size of Allocated Memory + 15MBs (sundry files) + 4GBs (logfile explosion) = Size of VM

Assume a 24GB VMDK and 2 GB of Memory for the VM.....

24 * 2 + 2 + .15 + 4 = 54.15 GBs per VM

I you only have a 512GB LUN... The math states:

Size of LUN * 0.80 = Useable Space on LUN / Size of VM = Total # of VMs (rounded down).

512 * .8 / 54.15 = 7 VMs

Some may argue that 24GBs for a snapshot in this case is unreasonable but I have seen 100GB snapshots because the backup software failed to properly delete the snapshot. Plus, logfiles fill up the disk quite a bit as well. In an error situation you will burn through disk space with each logfile quite easily. If the VMFS fills up, things stop working.....

Best regards,
Edward L. Haletky
VMware Communities User Moderator
====
Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education. CIO Virtualization Blog: http://www.cio.com/blog/index/topic/168354, As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization

Re: Max no of vm per lun

habibalby — Wed, 21 May 2008 11:31:14 GMT

From HP StorageWorks Best Practice

http://h20219.www2.hp.com/ActiveAnswers/downloads/VMware3_StorageWorks_BestPractice.pdf

Page 7

General SAN connectivity
VMware Infrastructure 3 and HP StorageWorks storage combine to provide a robust solution for
consolidation, reliability, and disaster recovery. This section provides general guidelines for
connecting VMware Infrastructure 3 to an HP StorageWorks SAN.
• Ensure each LUN has the appropriate RAID level and storage characteristics for the VMs that will
use it.
• Carefully determine the number of VMs you can place on a single LUN in your particular
environment. Note that, with new VMware Infrastructure 3 capabilities such as iSCSI and NAS, this
determination has become even more complex.
While VMware Infrastructure 3 supports up to 100 VMs per LUN, 32 VMs per LUN may often be
acceptable. However, HP recommends being more conservative and using 8 - 10 VMs per LUN.
In a NAS or iSCSI environment, HP recommends using 8 - 16 VMs per LUN.
If your VMs are not heavily impacting I/O, you may be able to exceed the conservative
recommendations above.
• To optimize array performance, spread I/O loads over the available paths to the storage.
• HP suggests configuring one Virtual Mirror File System (VMFS) volume per LUN. With VMFS-3 you
no longer have to set accessibility.
• In an environment with HP BladeSystem servers, a shared diagnostic partition is appropriate.
• Local SCSI controllers and Fibre Channel HBAs can be allocated to the VMkernel. However, it may
be preferable to use local RAID solutions for the Service Console and Fibre Channel HBAs for
external storage such as SANs.

Re: Max no of vm per lun

tom_brechet — Wed, 21 May 2008 07:21:51 GMT

Hi,
Well, you can have up to 100 VMs per LUN, 32 VMs per LUN may often be acceptable. But that all depends on your needs / VMs I/Os.
It's recommended to keep 8 ~ 14 machines per LUN.

Hi,

Do you have an source for that? Just wondering as I seem to remember the same, never implemented it this way my LUNs where always sized for 10-20 VMs but now I read that i should place 16 VMs on a LUN

Cheers

Tom

Re: Max no of vm per lun

tom_brechet — Wed, 21 May 2008 07:13:33 GMT

Yes, my documentation is definitvely different then. Please see in the attached document page 8 second bullet in Chapter "General SAN connectivity". This seems to be a newer version dated february 2007.

Re: Max no of vm per lun

jhanekom — Wed, 21 May 2008 07:03:12 GMT

You must have a different version of the Storageworks best practices document. Mine specifically state on page 7:

"Carefully determine the number of VMs you can place on a single LUN in your particular environment. Note that, with new VMware Infrastructure 3 capabilities such as iSCSI and NAS, this determination has become even more complex. While VMware Infrastructure 3 supports up to 100 VMs per LUN, 32 VMs per LUN may often be acceptable. However, HP recommends being more conservative and using 8 – 10 VMs per LUN. In a NAS or iSCSI environment, HP recommends using 8 – 16 VMs per LUN. If your VMs are not heavily impacting I/O, you may be able to exceed the conservative recommendations above."

The document I have is dated "6/2006 -2" on the very last page.

Re: Max no of vm per lun

habibalby — Wed, 21 May 2008 06:57:43 GMT

Read here as well

http://www.experts-exchange.com/Storage/Storage_Technology/Q_22410495.html

Re: Max no of vm per lun

habibalby — Wed, 21 May 2008 06:55:39 GMT

Hi,
Well, you can have up to 100 VMs per LUN, 32 VMs per LUN may often be acceptable. But that all depends on your needs / VMs I/Os.

It's recommended to keep 8 ~ 14 machines per LUN.

Max no of vm per lun

tom_brechet — Wed, 21 May 2008 05:59:26 GMT

Hi guys

I seem to remember that VMware stated with ESX 2.x that the maximum Numbers of VMs per Lun is above 100 so I never cared about it. Now there is this "SAN System Design and Deployment Guide" from VMware, released March 2007 which states on page 166 that no more than 16 VM should be placed on a lun? I don't seem to be the only one remembering the old limits as in the HP "VMware Infrastructure 3, HP StorageWorks best practices" there is this statement: While VMware Infrastructure 3 supports up to 80 VMs per LUN for a "light" I/O load... 40 VMs per LUN is a conservative number.

So, what is correct? Am I really that mistaken? 16 VM per LUN, is that possible?

Would be glad to have an answer on that.

Cheers

Tom