VMware Cloud Community
KJDzikowicz
Contributor
Contributor
Jump to solution

ESXi 4.0U1 on MCP55

Ive got a Tyan MCP55 dual 1207 board.

I will shortly be upgrading from 2x2218 (Santa Rosa 2 Core) to either 2x2300 (Shanghai 4 core) or 2x2400 (Istanbul 6 core) series opterons.

I have 16GB of DDR2 667 Reg ECC already installed and proven.

Im looking to run 4-6 VM's to process Video streams on this machine and I am wondering how much of an improvement a LSI 3081E-R (HBA -NO RAID-, IT Firmware, PCI-E 8x) controller will have over the MCP55 for hard drive access times and throughput. Each VM will have its own datastore on its own hard drive.

RAID 5 or 10 IS a possibility for the setup but I am unsure of a GOOD controller with hot-swap and hot-rebuild to use that will not cost me an Arm or Leg (the 3081 is $200 and as i understand, a reliable fast controller).

If anyone has any good suggestions of how to set this up to get the best performance, please let me know.

Thanks in advance.

Reply
0 Kudos
1 Solution

Accepted Solutions
DSTAVERT
Immortal
Immortal
Jump to solution

That shouldn't be a big problem but it will certainly depend. Single drives would not be a good choice for safety.

-- David -- VMware Communities Moderator

View solution in original post

Reply
0 Kudos
15 Replies
DSTAVERT
Immortal
Immortal
Jump to solution

Not knowing what exactly you will be doing but video streams are usually VERY demanding of disk subsystems. Many disks in raid 10 is most common. This may not be the best use of virtualization and certainly not if you are skimping on the disk side of things. Again not knowing what you will be doing.

-- David -- VMware Communities Moderator
Reply
0 Kudos
KJDzikowicz
Contributor
Contributor
Jump to solution

In the simple world, Ill be pulling data off of a NAS unit and re-encoding it to the local drives.

Ethernet will be a bottleneck at some point but i expect that to be 60-80 MB/sec over gigabit.

I can bond the 2 network channels if need be.

Im just looking for a good disk subsystem. If i need a mid-range to high end controller, then i will get it. Im just curious how badly the MCP will do with 6 VM's and 6 hard drives. I remember there being a bottleneck in the controller, i just dont know how bad it is.

If there is a good simple 6 disk setup you could recommend, I will deifinitely look into it.

Thanks.

Reply
0 Kudos
DSTAVERT
Immortal
Immortal
Jump to solution

I would use the Hardware Compatibility List to make sure you get a supported one. I would look at LSI since they do have monitoring support already in ESXi. The controller you choose will need a battery module and write caching enabled.

As for performance only you will know. If you have something currently running I would do some performance monitoring over time to see whether this is a good candidate for virtualization. 6 disk and processor and network hungry virtual machines may not do that well on the same host.

-- David -- VMware Communities Moderator
Reply
0 Kudos
KJDzikowicz
Contributor
Contributor
Jump to solution

Im definitely sticking with LSI.

Would you recommend an individual disk per VM or a sliced up 4 drive RAID 5?

Reply
0 Kudos
DSTAVERT
Immortal
Immortal
Jump to solution

Storage performance usually means RAID 10 and the more disks/spindles you have in an array the better the performance. ESX(i) is limited to slightly less than 2TB per LUN so you will need to have a controller that allows you to create virtual disks / luns on top of an array if you have more than 2TB total storage. You can also do RAID 10 pairs.

-- David -- VMware Communities Moderator
KJDzikowicz
Contributor
Contributor
Jump to solution

How do you think a 4 drive Raid 10 would stack up against 4 individual drives, one per VM?

Ive got a supported LSI controller in mind, LSI MegaRAID 9260-4i.

And im only going to be able to support 4 drives, possibly 6 with some work, in this chassis.

Ill be using WD RE3 3.5 drives, probably 320's.

Reply
0 Kudos
DSTAVERT
Immortal
Immortal
Jump to solution

Only you will be able to determine whether performance is acceptable or not in any configuration. As I said RAID 10 spread across many disks is the generally accepted performance winner for streaming. Again, I question whether virtualization is good choice in this situation and before you have too much invested do some performance monitoring on a single machine over an extended period of time. Look at disk access and processor utilization. If you have sustained periods of heavy disk IO it won't work well when you have 6 VMs fighting for disk and controller access.

-- David -- VMware Communities Moderator
KJDzikowicz
Contributor
Contributor
Jump to solution

I guess ill bench test this setup on the MCP55 with a single VM. Im relatively new to ESXi so if i post my results will anyone be able to give me a better "works or wont work" opinion?

Ill get the results as soon as i can. Thanks.

Reply
0 Kudos
DSTAVERT
Immortal
Immortal
Jump to solution

We can certainly give you opinions. Please do post your test results.

What OS will this be running from?

-- David -- VMware Communities Moderator
Reply
0 Kudos
DSTAVERT
Immortal
Immortal
Jump to solution

I would look at using the hardware directly rather than as a Virtual machine on ESXi for your testing. How have you been doing this?

-- David -- VMware Communities Moderator
Reply
0 Kudos
KJDzikowicz
Contributor
Contributor
Jump to solution

VM's will all be barebones installs of Windows Server 2003.

Up until now ive had only old P3 machines running encodes occasionally.

The plan for the machine is to run my download server in one VM giving it a high priority and a single hard drive. I was then going to put another VM for a possible email server. The left over power was then going to be pushed into the WIndows Encoder VM's with a low hardware priority to run in teh background.

Realistically i dont have any expectation of the encoders to be any more powerful than a dual P3 system. As long as my linux server VM's can happily run, im content with the encoders just burning excess unused CPU power.

I would just like to get the most bang for money invested.

Reply
0 Kudos
DSTAVERT
Immortal
Immortal
Jump to solution

If you aren't concerned about encoding / streaming performance then install away. Start with a minimal CPU count for each VM and as much RAM as they need. I would do the array but you decide. If this is for any type of production I would make an array controller a higher priority and be sure to get the battery option and make sure write caching is enabled. It makes a huge performance improvement.

-- David -- VMware Communities Moderator
Reply
0 Kudos
KJDzikowicz
Contributor
Contributor
Jump to solution

So a 4 drive RAID 1+0 on an LSI 9260-4i should hold back 4 VM's without much hassle, if 2 of the 4 VM's are not going to be heavily drive dependent.

Ill definitly be under 2TB, probably closer to 1TB at most.

Reply
0 Kudos
DSTAVERT
Immortal
Immortal
Jump to solution

That shouldn't be a big problem but it will certainly depend. Single drives would not be a good choice for safety.

-- David -- VMware Communities Moderator
Reply
0 Kudos
J1mbo
Virtuoso
Virtuoso
Jump to solution

MCP55 RAID is not supported in ESXi, you'd need to use them as individual drives as I think you suggest. That controller seems to do about 60MB/s with RE2's in my test box.

A cheap way into the LSI controllers is the Dell Perc 5i (look on eBay) - 8 SATA/SAS channels and not far short of 300MB/s write performance in RAID-5. But whatever controller you get MUST have battery-backed write-cache (BBWC) installed.

However RAID might not be the best solution here - would need to test. For example if these workloads all depend on high-volume sequential IO, individual physical disks for each VM will give better performance (head movement kills throughput - 100% random at 4K block size will give about 0.5MB/s on a 3-drive RE2 RAID-5 set for example).

If you do go RAID, try some different configurations. RAID-5 can be just as quick as RAID-10 when the disks are not able to saturate the controllers XOR engine, but will give much more space.

Please award points to any useful answer.

Reply
0 Kudos