ESX3, SAN and File and no idea

[lpryor]

Hi.

My current situation:

IBM x3650

2x Quad Xeon 2.8Ghz

16GB RAM

2x 73GB SAS drives

4GB HBA

Connected to a IBM DS3400 SAN with 12x 300GB SAS drives.

My problem is that I have yet to do the ESX course and I have to get this box up and running before my course.

Basically I have 1 VM to create (fileserver) and need to store about 500GB of data on the san.

What is the best way of dividing up the disks to make best use of the san for ESX and file storage?

Thanks to everyone who responds.

Cheers,

Lee

[Dave_Mishchenko]

You'll want to give some consideration to the types of VMs you'll be running and specifically the size of VMDK files you expect and the I/O load of your VM's.

In general a LUN will be sized between 300 and 500 GB and host 10 to 15 VMs. But obviously you'll want a larger one given that one VM will be using 500 GB. You would want to avoid one LUN of 1800 GB (assuming RAID 10 and no hotspares) and having 20 or more VMs on it. The problem with many VMs is that certain operations (powering on VMs, creating new files, expanding files) require an ESX host to lock the entire VMFS partition and this momentarily pauses I/O for all other ESX host and VMs on that partition. You would also want to avoid a situation of creating 2 LUNs say one of 1200 Gb and one of 600 GB and then putting all your high I/O VMs on the 1200 GB LUN. You'll want to spread the I/O over all your disks.

Lastly, you could create a small LUN of 300 GB for VMs and put your file server's C drive on it, and then create a 500 GB LUN and use a raw device mapping to your file server VM which would basically allocate the entire LUN to your VM. That way you could move the LUN easily to a physical server in the future, snaps shots are more easily handled.

Have you considered the RAID level for the DS3400? Also do you plan to share this storage with other ESX hosts down the road?

[esiebert7625]

Well basically you can install ESX to your local drives, you'll probably have enough room left for a 30-40GB local VMFS partition. You can then create a additional VMFS volume from your SAN disk. Sounds like you'll have about 1.4TB usable on your SAN so I would present this as at least 2 LUN's to ESX, preferrably 3. Usually the maximum LUN size you want to create is about 500-600GB. If you create 1 500GB LUN and 2 450GB LUN's you can create 3 VMFS volumes to use with VM's. You could create a VM and create a 500GB virtual disk for it. Just make sure you use a 2MB block size when creating yoru VMFS volume. Optionally you could use a RDM but you're probably better off using a VMDK instead.

Read through these links and they should help you out...

SAN Configuration Guide - http://www.vmware.com/pdf/vi3_esx_san_cfg.pdf

SAN Conceptual and Design Basics - http://www.vmware.com/pdf/esx_san_cfg_technote.pdf

SAN System Design and Deployment Guide - http://www.vmware.com/pdf/vi3_san_design_deploy.pdf

LUNS - http://www.vmware.com/community/thread.jspa?messageID=333672&#333672

LUNS Size - http://www.vmware.com/community/thread.jspa?threadID=36725&start=0&tstart=0

Sizing LUN for enough free space after VM - http://www.vmware.com/community/thread.jspa?messageID=649338

Smaller LUNS or Larger LUNS - http://www.vmware.com/community/thread.jspa?threadID=90203

ESX Server Raw Device Mapping - http://www.vmware.com/pdf/esx25_rawdevicemapping.pdf

VMDK vs. RDM - http://www.vmware.com/community/thread.jspa?messageID=662212&#662212

VM Disk Scenario’s and Performance - http://www.vmware.com/community/thread.jspa?messageID=661499

Choosing and Architecting Storage for your Environment - http://download3.vmware.com/vmworld/2006/adc0135.pdf

VMware Virtual Machine File System: Technical Overview and Best Practices - http://www.vmware.com/pdfs/vmfs-best-practices-wp.pdf

Layers of Virtual Storage in VMware VI3: Configuration without Confusion - http://www.vmware.com/pdf/storage-layers-wp.pdf

VMFS Block Sizes - http://www.vmware.com/community/thread.jspa?threadID=47171

Fyi…if you find this post helpful, please award points using the Helpful/Correct buttons.

-=-=-=-=-=-=-=-=-=-=-==-=-=-=-=-=-=-=-=-=-=-=-

Thanks, Eric

Visit my website: http://vmware-land.com

-=-=-=-=-=-=-=-=-=-=-==-=-=-=-=-=-=-=-=-=-=-=-

[lpryor]

Thanks very much guys. We will probably be growing to 8-10 VM's on the host so after reading the links we'll probably create a 300GB LUN for the VM's but I don't know what to do with the 500GB of files i have on the file server (w2k3).

Is it ok to store that much data in a VMDK or should it be raw on the metal?

Thanks,

Lee.

[Dave_Mishchenko · Accepted solution]

500 GB isn't too large and there are VMDK files out there that are much larger. How are you planning on backing up your VMs - something native in the VM or are you backing up the VM from ESX?

As Eric's post mentioned, if you need a VMDK larger than 256 GB you'll have to format your VMFS partition with a block size of greater than the default of 1 MB. You won't be able to change the block size later without formating the VMFS parition. 2 MB should be fine for you and you choose the block size when you create the data store.

1 MB block size - max VMDK is 256 GB

2 MB -- 512 GB

4 MB -- 1024 GB

8 MB -- 2048 GB.

[lpryor]

Thanks Dave.

We will be backing up to disk on a seperate server using Storagecraft. We are looking at another chassis for the SAN and then do something different.

Thanks for the help on the VMDK... I'm still trying to get my head around this and cannot wait to do the course.