Separate SWAP Drive

[spf62]

We currently are running 26 ESX 3.0.1 servers with 300vm's installed. We have multiple clusters which will be scalled down to ONE

We will be converting the current VM's into a newly setup 3.5 update 3 environment, 2.5 VI. The question came up about setting aside a dedicated SWAP/Page file area for the VM's. Is this someting that is recommended?

thanks in advance for your help

[Ken_Cline]

We currently are running 26 ESX 3.0.1 servers with 300vm's installed. We have multiple clusters which will be scalled down to ONE

How many hosts are you going to have in that one cluster? And what kind of storage do you have backing the cluster?

We will be converting the current VM's into a newly setup 3.5 update 3 environment, 2.5 VI. The question came up about setting aside a dedicated SWAP/Page file area for the VM's. Is this someting that is recommended?

Personally, I don't recommend this setup except in the following situation:

- You are replicating the VMs to another site and you don't want to replicate the swap file. Since most replication takes place at the LUN level, placing the swap files on a different LUN allows you to exclude them from replication.

Lots of people will try to convince you that there is a significant performance difference based on swap location - I say "hogwash". No matter how fast your swap storage is, it is painfully slow when compared to RAM speeds. If performance is your concern, make sure you allocate enough RAM so that you're not swapping. From my perspective, all that splitting out your pagefile does is add administrative overhead...

I always fall back on my mantra - if you don't have a really good reason to change a default, then don't change it!

Ken Cline

Technical Director, Virtualization

Wells Landers

TVAR Solutions, A Wells Landers Group Company

VMware Communities User Moderator

[Lightbulb]

I am not sure of any performance gain that will be attained by doing this. Some folk offload swap to local storage but then Vmotion is impacted. Never seen a reason to do this. Although I assume after posting this I will shortly find out how wrong I am

[spf62]

You make very good points.

I'm sure the powers to be will want to see more replies.

I appreciate your interjection!

Keep'em coming!

[spf62]

We will have approx 26 hosts, we have a HPXP SAN

In addition...I just found out that the reason for the separate SWAP area is for replication.

[ckrisIT]

I'll recommand NOT to set aside a dedicated SWAP/Page file area for the VM's too unless you seriously concern about replication of the datastores. Even you don't need to replicate the SWAP/Page file volume to the remote site, but you still need to create that volume and all the vmdks (the vmdks can be zero size). I'll say the little saving on replication time will never overcome the administrative burden that it creates. We have followed filer vendor's recommandation to seperate them, but I ended up regret doing that.

Just to name a few draw backs that I have experienced:

Extra options have to include in storage vmotion if you want to keep the SWAP/Page file seperated.

Redirection of the SWAP/Page file in Windows for all the VMs that you have.

If you are using iscsi or SAN, LUN lock maybe a catch since you will or might have too many vmdks in one lun. (considering you put all the SWAP/Page file (vmdk) in one or a few LUNs)

Hope this information help.

- ckrisIT

[Ken_Cline]

We will have approx 26 hosts, we have a HPXP SAN

I would strongly encourage you to split this up into multiple clusters. Experience has shown that when your cluster size goes much above 8 hosts that you begin to experience problems with SCSI reservations. I know that VMware says that you can have up to 32 hosts in your cluster -- but that doesn't mean that you should have that many.

In addition...I just found out that the reason for the separate SWAP area is for replication.

Then it's going to come down to cost of replication vs. cost of administrative overhead. The question becomes which is less expensive - people's time or bandwidth. Remember too, that with complexity comes increased risk of mistake - and during a disaster event is when you can LEAST afford a mistake.

Ken Cline

Technical Director, Virtualization

Wells Landers

TVAR Solutions, A Wells Landers Group Company

VMware Communities User Moderator

[Ken_Cline]

I'm sure the powers to be will want to see more replies.

I would encourage you to engage the services of a qualified, experienced consultant to help you with the planned migration. While it may appear to be a simple task, there are lots of complexities that will come up during the process. Having someone there who has "been there, done that" several times can make the whole process much less painful.

Ken Cline

Technical Director, Virtualization

Wells Landers

TVAR Solutions, A Wells Landers Group Company

VMware Communities User Moderator

[spf62]

We are building upon the suggestions of VMWARE and HP. They came in and gave us the best practice for our environment. We have been running VMWARE since version 1.8, so we aren't noobies. But when moving into the 3.5 world we needed some assistance.

It was thier recommendation to have all HOSTS on the same Cluster, this way VMotion, HA and DRS would all work together under one umbrella. They also suggested we keep a separate SWAP LUN to reduce replication time

[Ken_Cline]

We are building upon the suggestions of VMWARE and HP. They came in and gave us the best practice for our environment. We have been running VMWARE since version 1.8, so we aren't noobies. But when moving into the 3.5 world we needed some assistance.

I understand that you've got plenty of experience running / managing a VMware environment - and that's great; however, you're about to undertake a virtual migration project. This is something that you probably don't have a lot of experience with (and that you really don't need to develop expertise in, since you won't be doing a whole bunch of them!). There are a lot of moving parts, and it's helpful to have someone there to manage the process. Also, chances are good that you've already got a full time job that keeps you pretty busy. Bringing in outside help can alleviate a lot of the stress that goes along with one of these projects.

It was thier recommendation to have all HOSTS on the same Cluster, this way VMotion, HA and DRS would all work together under one umbrella. They also suggested we keep a separate SWAP LUN to reduce replication time

Hmm...

How many LUNs are you going to need to support your environment? Remember that each host is limited to 256 LUNs - by extension, since you want all hosts in the cluster to see all LUNs (to enable VMotion/DRS/HA to span all hosts), that means that the entire cluster is limited to 256 LUNs. Since you're doing replication, that implies that you're carving your storage out into relatively small chunks, which translates to lots of LUNs.

Do you plan to use snapshots? Remember that snapshots start out as 16MB files and grow in 16MB increments. That's a lot of metadata updating going on and a lot of locking taking place. If you plan to use VCB, you'll be using snapshots.

Make sure you understand all the implications of such a large cluster. As I said...just because you can doesn't mean you should ...

Ken Cline

Technical Director, Virtualization

Wells Landers

TVAR Solutions, A Wells Landers Group Company

VMware Communities User Moderator

[spf62]

Only have 8 hosts per cluster is way to small for our current plan. That may have worked in the 3.0 version, but it's not needed anymore.

Our LUNS are broken into 250gb sizes. We will probably have about 60 of them.

We snap shot, but they are automatically deleted after 12 hours.

Everything we are doing is coming from suggestions that the VMWARE and HP Consultants gave us.

We are just trying to find out what other users have used and what has worked. From there we will decide on what works for us,

Thank you all for your suggestions and comments