planning a new ESX environment

[jftuga]

We have been running the free VMWare Server 1.0 for almost two years now

and have decided to upgrade and expand to ESX server. Our organization

has just recently licensed the following software:

VMware ESX Server

o VMware ESX Server Virtual SMP

o VMware VMFS (Virtual Machine File System)

o VMware VMotion

o VMware DRS (Distributed Resource Scheduler)

o VMware HA (High Availability)

o VMware Consolidated Backup

o VirtualCenter Management Server

o VirtualCenter Management Agent for ESX Server

o VMware Converter Enterprise

.

Current

Our current VMWare Server environment consists of the following:

Dell PE 2850, two dual core 2.8Ghz Xeon CPUs, 12 GB Ram, three 10K 300

GB drives in RAID-5, one GB ethernet connection

I currently use Ghost to backup each of the VMs every night and then

keep three night's worth of images. We have an identical spare PE2850

that can be used as a cold stand-by.

We run 5 VMs on this server. Two of these VM systems run SQL server

(both low usage), one is a domain controller, one is a print server,

and one is our intranet web server. Two servers run Windows 2000 and the

other three run 2003. Needless to say, disk performance could be better,

since all of these run on a three disk RAID-5 system. When we added a

sixth VM (which ran SQL), we saw a noticeable performance degradation

across all of the VMs, so now this one runs on our cold-spare server,

since it is not mission-critical.

Planned

We have purchased two new Dell 2950s, with two quad core 3.16 GHz CPUs,

32 GB Ram each, two 15K 73 GB drives in RAID-0, (2 onboard GB NICs),

two quad port Intel GB NICs. We have also purchased one Dell MD3000i

(iSCSI), with fifteen 15k 300 GB drives and it has four GB NICs.

We are not sure if we should run these in RAID-5 or 10, as we do not

think space will be an issue.

We have a HP 5406zl network switch and have purchased a new 24 port

100/1000 blade and we also have more ports available on other blades

as well. Our 20 servers, 13 edge wiring closets switches, internet

connection from the firewall, etc. all run through this switch.

Our network has a star topology with this being the core.

Once we have the new servers installed, we plan on initially adding

six more guest servers. This will include two medium usage SQL servers,

one medium usage Exchange server (250 users), ISA Server, and our other

two domain controllers. We plan on using the VMWare P2V Convertor for this.

I was thinking that I would have two VMotion capable clusters.

One cluster would have two PE 2850s and the other cluster would have

two PE 2950s. All storage would come from the Dell MD3000i.

Questions

1a) Right now, we have physical servers in different VLANs for security

reasons. Our web server, email gateway server, other internet facing

servers all live in a DMZ'd VLAN. Can these servers reside in the same

cluster as our regular, more secure servers (file server, database server,

etc)? One of our main concerns is security since we store sensitive data.

1b) Is it OK to run a domain controller as a guest? Is it OK to have

all 3 of our domain controllers as guests? It is OK to use P2V on

domain controllers?

1c) Does anything special need to be done to make an Exchange server run as a guest and

is it OK to use P2V on an Exchange server?

2a) What will be the best way to carve up our MD3000i? I don't really

know (yet) how VMWare handles storage. We will want our configuration

to be capable of using VMotion. My initial thinking is to create storage

groups that span all 14 or 15 drives to get the best performance.

2b) Under the free VMWare Server, I have configured the guests to

split the vmdk files at 2GB and grow them on demand - only as needed.

Are these same options available under ESX?

2c) Is it possible to grow a Windows 2003 file system that is not a

Dynamic Disk? Or do we have to estimate future growth and use this for

our initial disk sizes?

3) About how many physical NICs will be needed for each server? I have

heard numbers ranging anywhere from 6 to 10.

4) What are the 'best practices' with respect to backups? We also want

to revamp our backup methodology and are open to suggestions.

5) Are there any books on Amazon that cover ESX 3.5 that are worth

purchasing?

Any help would be greatly appreciated!

Thanks,

-John

[Texiwill]

Hello,

1a) Right now, we have physical servers in different VLANs for security

reasons. Our web server, email gateway server, other internet facing

servers all live in a DMZ'd VLAN. Can these servers reside in the same

cluster as our regular, more secure servers (file server, database server,

etc)? One of our main concerns is security since we store sensitive data.

They can. I would suggest you segregate by vSwitch not VLAN on the vSwitch however. Others consider VLANs safe enough, and they are currently but that can change in a heartbeat..... The most secure installations, use Host Separation however. I.e. DMZ ESX Servers and Production ESX Servers.

1b) Is it OK to run a domain controller as a guest? Is it OK to have

all 3 of our domain controllers as guests? It is OK to use P2V on

domain controllers?

Yes. and not sure. I have heard success and horror stories.

1c) Does anything special need to be done to make an Exchange server run as a guest and

is it OK to use P2V on an Exchange server?

Plan Exchange very well, your disk layout will be the most important aspect of using Exchange in a VM. It is a disk intensive application. As for P2V, you can but I would not do this for the data aspects.

2a) What will be the best way to carve up our MD3000i? I don't really

know (yet) how VMWare handles storage. We will want our configuration

to be capable of using VMotion. My initial thinking is to create storage

groups that span all 14 or 15 drives to get the best performance.

I would not use all your drives in one LUN. Most LUNs for VMs are no more than 500-600GBs with no more than 12-15 VMs on each. More LUNs implies generally better throughput.

2b) Under the free VMWare Server, I have configured the guests to

split the vmdk files at 2GB and grow them on demand - only as needed.

Are these same options available under ESX?

Yes, but you really do not want to use these. Use VMware Converter to import these VMs into ESX. But make them full disks as split drives will adversely affect performance.

2c) Is it possible to grow a Windows 2003 file system that is not a

Dynamic Disk? Or do we have to estimate future growth and use this for

our initial disk sizes?

You can grow the disks using multiple methods, either by using something like PartitionMagic or adding more drives.

3) About how many physical NICs will be needed for each server? I have

heard numbers ranging anywhere from 6 to 10.

For full redundancy, security, and performance when using iSCSI you may want ten:

2 for SC

2 for vMotion

2 for VM Network

2 for iSCSI

2 for DMZ

However many people condense SC/vMotion to the same pNICs using VLANs, and run DMZ/Production on the same pNICs using VLANs. This depends on how you feel about using VLANs. You absolutely want your iSCSI network to be isolated however.

4) What are the 'best practices' with respect to backups? We also want

to revamp our backup methodology and are open to suggestions.

Backup as you normally do, the tools however generally change. You can do Within the VM backups (traditional tools) or full VMDK backups (Virtualization specific tools...) Check out http://vmprofessional.com/index.php?content=esx3backups.

5) Are there any books on Amazon that cover ESX 3.5 that are worth

purchasing?

There are several.... Mine of course (listed in signature below), and the Ogelsby, Herold, Laverick, and Madden book which is not yet available on Amazon.

Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education. As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization

[ctfoster]

Questions

1a) Right now, we have physical servers in different VLANs for security

reasons. Our web server, email gateway server, other internet facing

servers all live in a DMZ'd VLAN. Can these servers reside in the same

cluster as our regular, more secure servers (file server, database server,

etc)? One of our main concerns is security since we store sensitive data.

VMWare tells us that the vSwitches are as isolated as pSwitches so I'd be happy to run a DMZ on a host that also runs production VM's. The problem lies not so much in isolation created by the software but in the management of those devices. If you run a datacenter often have secure and insecure servers in the same rack. There is nothing stopping an over eager manager swapping cables over/assigning the wrong vLAN or IP address and exposing something they shouldn't. Same is true in the the ESX world. You are protected by your procedures not your software.

1b) Is it OK to run a domain controller as a guest? Is it OK to have

all 3 of our domain controllers as guests? It is OK to use P2V on

domain controllers?

Sure. However, my personal preference is to always run run a physical DC as the the PDC. I find time runs better that way. AD is a transactional database that you can't turn off under Windows 2003. For that reason I wouldn't bother cloning the DC's - just build a new server and DCpromo it. If it's running some key system for you - cold clone it.

1c) Does anything special need to be done to make an Exchange server run as a guest and

is it OK to use P2V on an Exchange server?

You might consider giving stores with high i/o there own LUNS. For P2V your either need to unmount the store or cold clone the server. Either way Exchange is offline.

2a) What will be the best way to carve up our MD3000i? I don't really

know (yet) how VMWare handles storage. We will want our configuration

to be capable of using VMotion. My initial thinking is to create storage

groups that span all 14 or 15 drives to get the best performance.

Cut the LUNS across a many spindles as you can. The 'sweet spot' for VMFS is aroung 500-700Gb so unless you have a really good reason to build a bigger LUN I'd stick to that.

2b) Under the free VMWare Server, I have configured the guests to

split the vmdk files at 2GB and grow them on demand - only as needed.

Are these same options available under ESX?

I't easy to expand vmdk files under ESX - and a total pain to shrink them. During the Convertor process you have the option to reduce the size of your drives. Therefore if Im faced with as server with a 60Gb drive with only 15 Gb of data on it I'll set it up with 20Gb and maybe expand it later if you need to. Unless you know for sure you are planning for some clear data growth on a drive (Exchange/SQL) manage your space proactively. You may be sitting on 1Tb now ... but in a few years time it'll seem like a thumb drive.

2c) Is it possible to grow a Windows 2003 file system that is not a

Dynamic Disk? Or do we have to estimate future growth and use this for

our initial disk sizes?

See above

3) About how many physical NICs will be needed for each server? I have

heard numbers ranging anywhere from 6 to 10.

Lets work out the minimum you need in production.

2 - Production VLAN

2 - Storage iSCSI

1 - Console

1 - VMotion

So thats six to start with - It's also nice to have a segment dedicated for backup traffic. You cant have too many pNICs on a host. How you use the rest is down to where your priorities lie.

> 4) What are the 'best practices' with respect to backups? We also want

to revamp our backup methodology and are open to suggestions.

There are as many options on this as there are posters on this forum. You have the licence so Consolidated Backup is clearly an option. I use esXpress and am very happy with it. What every you choose you will need a mix of 'file level' and VM vmdk strategies.