VMware Communities: Max no of vm per lun

Up to Discussions in Enterprise Strategy & Planning

This Question is Possibly Answered

1 "correct" answer available (10 pts)

7,013 Views 19 Replies Last post: Aug 13, 2010 9:41 AM by FredPeterson

1 2 Previous Next

tom_brechet

23 posts since
Sep 15, 2005

Currently Being Moderated

May 20, 2008 10:59 PM

Max no of vm per lun

Hi guys

I seem to remember that VMware stated with ESX 2.x that the maximum Numbers of VMs per Lun is above 100 so I never cared about it. Now there is this "SAN System Design and Deployment Guide" from VMware, released March 2007 which states on page 166 that no more than 16 VM should be placed on a lun? I don't seem to be the only one remembering the old limits as in the HP "VMware Infrastructure 3, HP StorageWorks best practices" there is this statement: While VMware Infrastructure 3 supports up to 80 VMs per LUN for a "light" I/O load... 40 VMs per LUN is a conservative number.

So, what is correct? Am I really that mistaken? 16 VM per LUN, is that possible?

Would be glad to have an answer on that.

Cheers

Tom

habibalby

452 posts since
Jan 6, 2006

Currently Being Moderated

1. May 20, 2008 11:55 PM

in response to: tom_brechet

Re: Max no of vm per lun

Hi,

Well, you can have up to 100 VMs per LUN, 32 VMs per LUN may often be acceptable. But that all depends on your needs / VMs I/Os.

It's recommended to keep 8 ~ 14 machines per LUN.

Best Regards, Hussain Al Sayed Consider awarding points for "correct" or "helpful".

Report Abuse

habibalby

452 posts since
Jan 6, 2006

Currently Being Moderated

2. May 20, 2008 11:57 PM

in response to: habibalby

Re: Max no of vm per lun

Read here as well

http://www.experts-exchange.com/Storage/Storage_Technology/Q_22410495.html

Best Regards, Hussain Al Sayed Consider awarding points for "correct" or "helpful".

Report Abuse

jhanekom

1,136 posts since
May 13, 2005

Currently Being Moderated

3. May 21, 2008 12:03 AM

in response to: tom_brechet

Re: Max no of vm per lun

You must have a different version of the Storageworks best practices document. Mine specifically state on page 7:

"Carefully determine the number of VMs you can place on a single LUN in your particular environment. Note that, with new VMware Infrastructure 3 capabilities such as iSCSI and NAS, this determination has become even more complex. While VMware Infrastructure 3 supports up to 100 VMs per LUN, 32 VMs per LUN may often be acceptable. However, HP recommends being more conservative and using 8 – 10 VMs per LUN. In a NAS or iSCSI environment, HP recommends using 8 – 16 VMs per LUN. If your VMs are not heavily impacting I/O, you may be able to exceed the conservative recommendations above."

The document I have is dated "6/2006 -2" on the very last page.

Report Abuse

tom_brechet

23 posts since
Sep 15, 2005

Currently Being Moderated

4. May 21, 2008 12:13 AM

in response to: jhanekom

Re: Max no of vm per lun

Yes, my documentation is definitvely different then. Please see in the attached document page 8 second bullet in Chapter "General SAN connectivity". This seems to be a newer version dated february 2007.

Attachments:

4AA1-0818ENW.pdf (760.3 K)

Report Abuse

tom_brechet

23 posts since
Sep 15, 2005

Currently Being Moderated

5. May 21, 2008 12:21 AM

in response to: habibalby

Re: Max no of vm per lun

>Hi,

>Well, you can have up to 100 VMs per LUN, 32 VMs per LUN may often be acceptable. But that all depends on your needs / VMs I/Os.

>It's recommended to keep 8 ~ 14 machines per LUN.

Hi,

Do you have an source for that? Just wondering as I seem to remember the same, never implemented it this way my LUNs where always sized for 10-20 VMs but now I read that i should place 16 VMs on a LUN

Cheers

Tom

Report Abuse

habibalby

452 posts since
Jan 6, 2006

Currently Being Moderated

6. May 21, 2008 4:31 AM

in response to: tom_brechet

Re: Max no of vm per lun

From HP StorageWorks Best Practice

http://h20219.www2.hp.com/ActiveAnswers/downloads/VMware3_StorageWorks_BestPractice.pdf

Page 7

General SAN connectivity

VMware Infrastructure 3 and HP StorageWorks storage combine to provide a robust solution for

consolidation, reliability, and disaster recovery. This section provides general guidelines for

connecting VMware Infrastructure 3 to an HP StorageWorks SAN.

• Ensure each LUN has the appropriate RAID level and storage characteristics for the VMs that will

use it.

• Carefully determine the number of VMs you can place on a single LUN in your particular

environment. Note that, with new VMware Infrastructure 3 capabilities such as iSCSI and NAS, this

determination has become even more complex.

While VMware Infrastructure 3 supports up to 100 VMs per LUN, 32 VMs per LUN may often be

acceptable. However, HP recommends being more conservative and using 8 - 10 VMs per LUN.

In a NAS or iSCSI environment, HP recommends using 8 - 16 VMs per LUN.

If your VMs are not heavily impacting I/O, you may be able to exceed the conservative

recommendations above.

• To optimize array performance, spread I/O loads over the available paths to the storage.

• HP suggests configuring one Virtual Mirror File System (VMFS) volume per LUN. With VMFS-3 you

no longer have to set accessibility.

• In an environment with HP BladeSystem servers, a shared diagnostic partition is appropriate.

• Local SCSI controllers and Fibre Channel HBAs can be allocated to the VMkernel. However, it may

be preferable to use local RAID solutions for the Service Console and Fibre Channel HBAs for

external storage such as SANs.

Best Regards, Hussain Al Sayed Consider awarding points for "correct" or "helpful".

Report Abuse

Texiwill

11,345 posts since
Jan 13, 2004

Currently Being Moderated

7. May 21, 2008 6:12 AM

in response to: habibalby

Re: Max no of vm per lun

Hello,

How many VMs per LUN depends on the storage device involved and the size of the VMs. A good algorithm for sizing a VM is:

Size of VMDK * 2 (possible snapshots) + Size of Allocated Memory + 15MBs (sundry files) + 4GBs (logfile explosion) = Size of VM

Assume a 24GB VMDK and 2 GB of Memory for the VM.....

24 * 2 + 2 + .15 + 4 = 54.15 GBs per VM

I you only have a 512GB LUN... The math states:

Size of LUN * 0.80 = Useable Space on LUN / Size of VM = Total # of VMs (rounded down).

512 * .8 / 54.15 = 7 VMs

Some may argue that 24GBs for a snapshot in this case is unreasonable but I have seen 100GB snapshots because the backup software failed to properly delete the snapshot. Plus, logfiles fill up the disk quite a bit as well. In an error situation you will burn through disk space with each logfile quite easily. If the VMFS fills up, things stop working.....

Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education. CIO Virtualization Blog: http://www.cio.com/blog/index/topic/168354, As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization

Report Abuse

habibalby

452 posts since
Jan 6, 2006

Currently Being Moderated

8. May 21, 2008 10:48 PM

in response to: Texiwill

Re: Max no of vm per lun

Yes, I do agree entierly with Taxiwill. SAN Calculation for VMDK files is the best strategy and planning for your VMs. It's not only depends on the LUN Size and How many VMs per LUN? It depends on the Backup Strategy, Snapshot, logs. if they are stored to the same SAN Storage for Backing up to the Proxy Server and Tape Library. Blah Blah Blah.

Best Regards, Hussain Al Sayed Consider awarding points for "correct" or "helpful".

Report Abuse

tom_brechet

23 posts since
Sep 15, 2005

Currently Being Moderated

9. May 21, 2008 11:00 PM

in response to: Texiwill

Re: Max no of vm per lun

Hi Edward

Thanks for that formula. But what I need to know is probably exactly the opposite: Not how many VMs per given LUN size but how big a LUN size for the VMs. I've had LUN sizes varying from 256 to 1GB in my past designs and all worked quite well. Now I have a customer with a 1GB LUN which has some trouble. This is when I try to find the info that existed back then (at least in ESX 2.x) that VMware supports over 100 VMs per LUN. While searching I only find this new to me info that one should never place more that 16 VMs on a LUN.

Am I the only one catched cold by this?

Cheers

Tom

P.S.: the consultant with 18 happy and one unhappy VI-Design customer

Report Abuse

Texiwill

11,345 posts since
Jan 13, 2004

Currently Being Moderated

10. May 22, 2008 7:21 AM

in response to: tom_brechet

Re: Max no of vm per lun

Hello,

The math really does not change that much when you start to decide on how big a LUN you want..... The last calculation changes.

BTW, it was not 100 VMs per LUN in ESX 2 days it was 192 files per LUN.... + 32 Files for every Extent added. And the more Files on the LUN the slower the LUN went. If for example there were 6 files per VM (2 VMDK, 1 vMotion log file, 1 suspend file, 2 REDO files), you could not put more than 32 VMs per non 'extent'ed VMFS. And most people cut that in half even so. So 100s of VMs, never, but 100s of files, perhaps. You could perhaps have 192 VMDKs, but that does not necessarily translate to 100s of VMs.... At least I never once suggested more than 16 VMs per LUN even in v2 days.

For ESX v3, to calculate the size of a LUN based on a count of VMs, reverse the algorithm mentioned previously some:

Size of VM * # of VMs / 0.8 = Size of LUN required

Assume a 54.15GB VM, and 16 VMs.... = 1083 GBs or 1.083TBs of LUN space required.

Remember you want to fill a LUN no more than 80% just in case.

Also, it is often much better to have several smaller LUNs than one massive LUN. This will help with alleviate SCSI Reservation Conflicts. These conflicts are often what dictates the LUN count more than anything else. It depends on the storage used as well.

Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Report Abuse

williambishop

1,163 posts since
Mar 9, 2006

Currently Being Moderated

11. May 22, 2008 7:45 AM

in response to: habibalby

Re: Max no of vm per lun

I have luns that vary from 1 vm to 150 vm's. They are all scaled according to IO and the array they are connecting to.

So the answer is ALWAYS...._.IT VARIES_.

Ita feri ut se mori sentiat

Report Abuse

tom_brechet

23 posts since
Sep 15, 2005

Currently Being Moderated

12. May 26, 2008 6:56 AM

in response to: Texiwill

Re: Max no of vm per lun

Hi Edward

>BTW, it was not 100 VMs per LUN in ESX 2 days it was 192 files per LUN.... + 32 Files for every Extent added.

Am still battling my war and found this info here http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1269

>Up to 100 virtual machines may share a VMFS-2 volume if they have less disk-intensive workloads, subject to other system constraints such as CPU and memory resources.

Good to know I'm not completely off my rocker and do at least remember some things correctly

BTW: Got a statement from a Field-SE hat all these numbers are no limits but mere recommendations. The only limiting factor for VMware seems to be performance and thats not my question at the moment.

Cheers

Tom

Report Abuse

Texiwill

11,345 posts since
Jan 13, 2004

Currently Being Moderated

13. May 27, 2008 8:09 AM

in response to: tom_brechet

Re: Max no of vm per lun

Hello,

The max # of files available for VMFS-3 is ~31000, I actually think it is 31715 or something close to that.. Consider that there are at least 1 directory and 9 files per VM, but most likely 15 considering all the logfiles created, there can ~3100 VMs per VMFS-3. I would say this is the maximal count allowed, but you are ideally looking at ~1900 VMs. It is all based on file count and the average # of files/directory for a VM with a single VMDK is ~16.However with a 2TB limit on the size of a VMFS-3, VMDK;s may need to be .5GBs for this to work.

Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Report Abuse

tom_brechet

23 posts since
Sep 15, 2005

Currently Being Moderated

14. May 27, 2008 8:03 AM

in response to: Texiwill

Re: Max no of vm per lun

Hi Edward

Now we are talking. That statement or any other is exactly what I need from VMware. How many VMs they support on a single LUN. Be it 10 or 100 or 1000 or whatever.

I've given you 2 helpful answers now as your post have indeed been the most infomative. I'd like to reserve the "correct answer" for whoever can point me to an official statement of VMware what the exact number is.

Many thanks

Tom

Report Abuse

Go to original post 1 2 Previous Next

Go to original post

This Question is Possibly Answered

May 20, 2008 10:59 PM

Max no of vm per lun

Actions

Bookmarked By (0)

Share This Page

Communities