VMware Cloud Community
RogerSillars
Contributor
Contributor

Small branch office server RAID setup

Hi,

We have 4 remote offices (from 15 to 40 users each) and are planning on putting a VMware setup in for each site.

Were going to go for an IBM x3650 (2x quad cores) with 6GB RAM and 6x 146GB 15k SAS disks and VMware ESX Starter for each site

Each of these machines will be running the 3 (or 4) VMs:

File, Print & AD

Exchange

Misc (voice mail, etc)

In the future we may want to put Terminal Server or Citrix VMs on these boxes and replicate these VMs to our head office for DR.

Does this seem a reasonable solution?

Would one of these machines handle all these services and potentially TS or Citrix? I realise it would be one machine running everything, that is why we would consider replication.

I have been recommended to setup 2x disk for ESX OS and 4 for data (RAID 5 or 10). However I think it would be better to setup a single RAID 5 with 1 hot spare. My theory is that the ESX would hardly be using the disks so it would be better for all round performance to have more spindles for the VMs. How would you setup the disks for best performance and storage space?

Thanks,

Roger

Reply
0 Kudos
13 Replies
GBromage
Expert
Expert

Would one of these machines handle all these services

and potentially TS or Citrix? I realise it would be one

machine running everything, that is why we would

consider replication.

Yeah, it should be OK. Provided your Exchange server isn't too busy. Voicemail may be a problem, because that's fairly time-critical as far as voice processing goes.

I have been recommended to setup 2x disk for ESX OS

and 4 for data (RAID 5 or 10). However I think it

would be better to setup a single RAID 5 with 1 hot

spare. My theory is that the ESX would hardly be

using the disks so it would be better for all round

performance to have more spindles for the VMs. How

would you setup the disks for best performance and

storage space?

Theoretically, you get better RAID5 performance with either 5 or 9 disks. 4 or 8 data stripes plus parity)

More spindles is usually better, but if you have a drive failure, it becomes much slower because the controller has to read EVERY disk to reconstruct the missing data.

Depending how much space you need, you might find it more efficient to have more spindles but smaller drives. Smaller drives are generally more reliable, because the have fewer platters and less sector density.

I hope this information helps you. If it does, please consider awarding points with the 'Helpful' or 'Correct' buttons. If it doesn't help you, please ask for clarification!
Reply
0 Kudos
RogerSillars
Contributor
Contributor

Thanks for your help.

Were planning on having 6 disks at this stage, we can add other drives externally at a later stage if we need them.

Which option would be better?

Option 1:

2x disks for VI3 (RAID1)

3x disks for VMs (RAID5)

1x disk for hot spare

Option 2:

5x disks for VI3 and VMs (RAID5)

1x disk for hot spare

I'm thinking option 2 sounds better as we get more spindles. However we would no longer have dedicated drives for the OS (VI3). My thoughts are that it won't be using the disks much anyway so the VMs could do with the extra spindles.

Thanks,

Roger

Reply
0 Kudos
dalepa
Enthusiast
Enthusiast

I'de go with a Netapp S500 (1-6TB) for CIFS and use NFS on the S500 for your vmware datastores. Snapmirror all the CIFS and vmware volumes back to the head office (fas2020) for a tapeless backup/DR solution . Simple and expandable.

Reply
0 Kudos
PerryWhittle
Enthusiast
Enthusiast

when you say 2 disks for ESX OS i presume you mean 2 x 146Gb mirrored? That is overkill as the ESX OS will use no where near any of that amount of storage. As pointed out, if you have the option go for smaller disks and more of them. You could then have 3 separate arrays

mirrored OS,

RAID5 for VM's,

RAID5 for exchange data

Pez

Please rememeber to award points if you found this helpful or correct
Reply
0 Kudos
e12pilot
Enthusiast
Enthusiast

We have essentially the same setup, but with HP hardware.

We went with the 2x146GB disks for ESX and 4x146GB disks (RAID 5) for the VMs. The RAID 1 partition allows us to segment the setup a little bit, do backups to RAID1, etc. The RAID controller in the server has 512MB of cache, and does about 60-80MB/sec on the RAID5 volume, plenty fast for our remote needs.

Reply
0 Kudos
gmjulian
Contributor
Contributor

I have struggled with a similar situation in my env. We use HP hardware so I went with a mirrored (raid 1) OS of 72 Gb drives (overkill I know but it is what came with the server) and then a Raid 5 or the ADG (you can lose 2 drives) data array to store my vmx and vmdk files. One thing to keep in mind is that this type of senario only protects you from disk failure. I actually had a server die on me a month after installation. It brought the whole office down. Thank fullly i had 24x7 support with 4 hour response time and we were able to get the box back up and running in short time but there are other issue to keep in mind when spec'ing a single server to handle an entire site. I have since moved some of my larger remote offices to a citrix solution out of my main datacenter to try to mitigate downtime. Now I am dependent on my WAN connection Smiley Happy

Reply
0 Kudos
PerryWhittle
Enthusiast
Enthusiast

the thing with ADG is it sucks up disc space and when that's at a premium you're stuck

think about it for a minute, the OS doesnt actually store much does it.

just use 1 x 72GB disc for the OS (you can always recover/reinstall from the cd in about 10-15 mins)

raid5 for VM's

raid5 for exch data

it really goes back to the old age adage "there's more than 1 way to skin a cat" Smiley Wink

Please rememeber to award points if you found this helpful or correct
Reply
0 Kudos
RogerSillars
Contributor
Contributor

Thanks everyone for your help.

We are still considering centralising our system or not so this may or may not be a temporary solution. These are small sites of 15 to 35 users so putting in a SAN at each site is out of the question due to the cost. Since we may end up pulling these machines back into our data center we need to stick with drives we can re-use that's why we are going for the 3.5" SAS internal drives as we could then put them in our main site SAN (DS3400) if we centralise.

For now we have access to 6 drives max and are unlikely to get funding for any more. The current servers at each site are actually using 2x RAID1 arrays with 4x ATA drives and a P4 of some kind. I suspect that this solution should be heaps faster.

I am reluctant to use 1 disk for ESX as we have no onsite IT people and rebuilding would be a real pain, having redundancy is critical. A hot spare drive isn't really necessary but would be nice, I guess we could always have one sitting next to the server and get someone to swap it if it fails (assuming IBM can't replace it quickly).

Since we are limited to 6 disks then having multiple RAID5 for the VMs and Exchange are out unless we do 2x 3 disk RAID5.

Perhaps the best question is why should we NOT put the VMs and ESX on the same RAID array? Doing so gives us more options.

There are likely to be 3 VMs. File/Print/DC, Exchange & Misc.Load on these isn't that high.

Options we have that give enough disk space are:

  1. 2 disk RAID1, 3 disk RAID5, 1 disk HS (50GB spare)

  2. 2 disk RAID1, 4 disk RAID5 (190GB spare)

  3. 2 disk RAID1, 4 disk RAID10 (50GB spare)

  4. 3 disk RAID5, 3 disk RAID5 (150GB & 160GB spare)

  5. 5 disk RAID5, 1 disk HS (330GB spare)

  6. 6 disk RAID5 (470GB spare)

  7. 6 disk RAID10 (190GB spare)

I think option 5 is the best overall, my second would be option 2. If anyone can give me the perfect answer that would be nice.

Reply
0 Kudos
RogerSillars
Contributor
Contributor

Hi Everyone. I was just looking at ESX 3i. I assume when they are talking about this it is some sort of flash device inside the server or via USB flash disk (http://blogs.vmware.com/vmtn/2007/09/more-on-esx-ser.html)

If this is the case then I assume the hypervisor doesn't use much disk activity (if any). If this assumption is correct they why not have ESX and the VMs on the same RAID array for small setups?

Thanks,

Roger

Reply
0 Kudos
e12pilot
Enthusiast
Enthusiast

3i is still Beta, but yes, it essentially boots the hypervisor off of a flash based device (USB key, flash disk, etc). No linux OS to manage/patch, etc. It would be ideal for this type of environment, but the only drawback is that it is beta, so if support is essential for you, it isn't the best solution out there. With that said, it is the future.

To answer your question regarding which RAID setup is best, the answer is: "it depends" (flashes of VMWorld VCB anyone?). The seperate RAID 1 volume is nice to store templates on, etc, etc. It is also a nice spot to run backups/snaps to without impacting your "production spindles".

If it were me I would try and allocate as much ram as possible to your DC to get it off of your disk. Try allocating enough memory to the DC to store your AD database in memory.

For the print server, definitely assign a second drive for the print queues, and you can play around with putting this on RAID5 or RAID1. The benefit to having these two arrays is that your are segmenting your spindles. In our setup the controller is very powerful and runs at about 1-5% CPU during peak hours, so can tolerate a split array type setup with no issues (512MB of cache is nice).

In this situation you just need to test, test, test.....It really depends on what you want out of the environment (i.e. ideal redundancy, performance, etc), and that will really drive your decision.

Good luck

PerryWhittle
Enthusiast
Enthusiast

i prefer option 4 as you definitely want to keep your exchange databases on a separate array Smiley Wink

Please rememeber to award points if you found this helpful or correct
RogerSillars
Contributor
Contributor

Thanks everyone. I still don't know what to do but will work that out. I appreciate all your help.

Reply
0 Kudos
ccandia
Contributor
Contributor

Roger,

I read through your post from 2007 about branch offices and am interested in what you ended up with and where you are now. We are an IBM shop with iSeries and xSeries. I am also very interested in the DS3400 as well. So your post mentioned all the right things as you said keeping disks compatible with your DS system.

I'm in a slightly similar situation looking at disks, but I intend to have shared storage for our small, but singe site office.

My ultimate goal is running two ESX4 servers on x3550's with 2x 73GB SAS drives in RAID1. As you and everyone else says, over kill, but I can't think of any better way to protect the hypervisor, regardless of the fact that it indeed can be re-installed quickly. We want to connect the two x3550s to DS3400 and IBM San switch. We have a 3650 with internal disk to run vCenter and do san backup if needed. We are looking at the new backup in version 4.

I am trying to size our SAN and am interested in your DS experience. I'm currently looking at running around 6-8 146GB SAS disks in RAID5 for the datastores and throwing a few high capacity sata's in there for templates and backup.

So with all that said, everyone in the world says it all depends on your needs and what you're running. We are looking at running about the same things you described, DCs, email servers, etc. I anticipate about 5-6 production vms and maybe 5-10 test servers, which may be up or down.

Do you have any new experience since you posted this that might help us in selecting the SAN configuration? Are you still running on DS3400 and getting good performance?

Thanks in advance.

Reply
0 Kudos