HP c3000 + ESX + Datacore San Melody

[TommyFCP]

Ive been trying to figure out the best way to setup my new servers and need some advice.

So I have the HP c3000 chassis with 3 BL460 Blades and 1 SB440c Storage Blade. One of the BL460 blade is parntered with the Storage Blade. So only that blade sees the storage on that storage blade.

I would like to make that one BL460 server blade the management for everything. Now my question is should I install Windows 2003 Server directly to that BL460 Blade, or make the Windows 2003 Server a Virtual Machine and run it on that blade? Right now I have ESX install on that blade and 2003 Server a VM. How do I setup San Melody to see the storage on the storage blade?

[kjb007]

I'm not sure if your sanmelody is an appliance or is an application that lives on an os. Either way, if your ESX server see the storage from the storage blade. Create a vm and allocate that storage to the vm. Then if the san melody is an application, then load the app on your windows or host application, and you should be able to configure that app to provide the SAN presentation. You can then use that storage to be given to the other blades as iSCSI.

I haven't used the software, so can't speak to it exactly, but the concept is similar to the other free and commercial ip SAN appliances and applications.

Hope that helps,

-KjB

[cjt]

SANMelody is great for creating a cheap "software" iSCSI SAN for a VMware testbed.

As far as I understand, you have two choices,

A - Install Win2003 server (or XP??) to the blade that is dedicated to the storage. Install SANMelody. This becomes a glorified "physical" SAN Controller. The two remaining 2 blades then become ESX hosts which have access via iSCSI to the presented storage volumes.

B - Install 3xESX hosts to the respective blades. To the blade dedicated to the storage blade - Install SANMelody inside a guest OS 2003/XP (as a VM) - or use a virtual appliance released by Datacore. This becomes a glorified "virtual" SAN Controller, which is bound to that blade, as it is the only blade with "physical" access the storage. This VM will then present storage to all 3 ESX servers. This is the option I would try, as it gives you an additional ESX host, however, IO is the concern, as the Gateway to the "virtual SAN" is the SANMelody VM.

I have not tested this - but theoretically should work, providing all hardware has been checked via the VMware HCL.

It is for a Dev/Test environment I hope ?? ... which has no high IO requirements ??

useful link ...

I am interested to see how you go.

[TommyFCP]

KJB,

This was what I was thinking but didn't know if it was the right way of doing it. San Melody is an application that runs in Windows. So I should create a VM Storage that is the same size of my storage blade? I believe 700GB. And attach it to the VM that is the management?

[TommyFCP]

CJT,

Plan B was what I was trying to pull off to work. But this will be a production setup if all goes well. Why is this setup not good for production you think?

The blade has 6 Network ports, a Quad Core, and 4 Gigs of ram. Do you think I should bump up the specs? or am I better off going with Plan A for a production setup?

[kjb007]

Yes. In your scenario, I think the plan is sound. You don't want to leave that blade just as management, otherwise, you lose 1/3rd of your processing power to only handle your SAN. I would use a VM on that host, and give the storage blade's storage completely to that vm, so it can manage your SAN for you. Since it will be on the ESX host, and will be presented to the vm as a vmdk file, I would try to break up the 700 GB into smaller chunks. Just so you have more LUNs to spread the I/O a little bit, and so all of your vm's aren't in one location, in case something unforseen occurrs. Also, if you plan to do DRS/VMotion, which I assume you do, make sure that VM is configured to not be part of any vmotion. Since it's using local disk, it won't vmotion anyway, but remove it anyway to save on the DRS calculation. Since you have to run Windows for your software, you can use this vm to host your vc as well.

-KjB

[cjt]

I would still go plan B (if IO permits) - as this gives you the most flexibility.

6 Network ports is quite ample, I have always liked dedicated ports - SC+VMKernel (2), VMotion (2), VMTraffic (2) - for full redundancy.

since it is a blade, and a single point of failure anyway, you could swing towards an aggressive 4gb/s pipe for your VMTraffic - SC+VMKernel (1), VMotion (1), VMTraffic (4).

I still believe in using a dedicated port for VMotion, as per VMware best practice.

I would increase the RAM. In all virtualisation projects I have done since ESX2, the first thing you run out of is RAM. Even with transparent page memory sharing, you will run out of physical RAM rather quickly.

Regarding CPU? One question - are the blades 2-way (ie 2 sockets), quad core (ie. 8 cores per blade)?

The reason why i am concerned for this design as a production setup...

--> VMware best practice - iSCSI should only be used in environments which have Low IO requirement and have a Low data ROC (Rate Of Change). Especially, since you are adding another virtualisation layer between VMware and the storage - SANMelody. You are putting a virtualisation solution (VMware), on top of a storage virtualisation solution (Datacore), on top of a consolidation solution (blade technology) - then running that storage virtualisation solution (Datacore) inside a VM on the original virtualisation solution (VMware), and running it in production. WOW !!

Do not expect a hefty IO intensive SQL database to give you impressive results.

Have you performed a virtualisation assessment? Free tool - VMware Cap Planner for under 25 servers. This will give you a good blueprint of your data type and your requirements for a virtual environment.

--> A Production VI environment is to contain minimal SPOF (Single point of failure). The direct attached storage (storage blade) is a big one. You lose that blade, or the storage blade, your entire environment is down, regardless of the VMware enterprise goodies. Additionally, if the VM that is acting as the "virtual" SAN Controller becomes unstable or goes offline, again, your environment is compromised. Datacore best practice is to use 2 servers. Remember - VMware ESX Enterprise is expensive for a reason - it minimises planned + unplanned downtime. Entry level SANs are cheap these days, example - EMC AX4. If you have invested in VI3 enterprise, then do not go cheap on the SAN component. The SAN is the engine for your virtual environment.

--> SANMelody in a VM may produce an IO bottleneck. If you have a low IO requirement, with a reasonably low data rate of change, then you should be fine.

Hope this answers your question.

[TommyFCP]

KJB,

What do you recommend I break the 700GB into? Two parts? 350GB?

I do plan on doing Vmotion and DRS.

[TommyFCP]

CJT,

I currently have a DL360 G4 with 146GB of Hard drive running my current ESX 2.5 Production, Once I get my new setup up and running I plan on migrating over the VM's on that machine to the new one. Now I also plan to get bigger hard drives for that Dl360 and installing ESX 3.x on it and making it the DR. Using SanMelody to mirrior to it. And if anything should happen to that storage blade or parner blade, I could use the DL360.

I have not performed a virtualisation assessment, Where would I find that tool? I'm trying to reduce my SPOF as much as possible I did have a SAN in my budget but my budget was reduced, so now I have to work with what I have until I can get my SAN.

[kjb007 · Accepted solution]

Yes, enough to hold 10-20 vm's each. 350GB should be a good number, but will vary depending on your vm images.

-KjB

[cjt]

good, you seem to have the SANMelody sorted.

I have just read your other post, and noticed the VMware environment is 5 VM's running on the existing HPDL360. Do not worry about a Virtualisation assessment, as between 3 ESX servers, I cannot see there being a problem.

Regarding the SAN - what can I say... damn beancounters :smileyangry:

dont worry, it only takes one incident where the entire company is offline - you will get a SAN approved very quickly !!

Regarding the carving of the storage - It wont make much difference performance wise, I would go with the 2 x 350GB.

[TommyFCP]

I just tried adding a new Hard Drive to the VM, and it tells me the file is too big for the filesystem. Why is that?

[meistermn]

Maybe you should read this blog

Do-It-Yourself Storage

[kjb007]

When you create the VMFS filesystem, the block size you choose determines how big of a file (or disk) you can create. See below, but you'll need a 2 MB block size.

Block Size = Max File Size

1 MB = 256 GB

2 MB = 512 GB

4 MB = 1024 GB

8 MB = 2048 GB

-KjB

[TommyFCP]

KJB,

Bingo! I remember setting this part up and picking 1MB... 😛

[Berniebgf]

Hi.....

I have looked over the previous posts and got a little confused as to where you left things.

My 2Cents.

I would dedicate one of the Blade for SANMelody and NOT create a "SANmelody VM"

Why?: As your environment grows, you will want your storage solution to grow with it, you will most likely start using extra features and functionality.

If you created a virtual machine as the datacore box, are you gong to MAP Thin provisioned volumes to the VM's directly via iSCSI software initiators or BACK to the ESX Host via iSCSI?

If back to the ESX hosts, this sounds messy to me....

Sorry if this has been covered off.

Bernie.

sanmelody.blogspot.com

[TommyFCP]

Bernie,

Thank you for your input. Only reason I'm going with a SANmelody VM is because if the blade that is partnered with the storage blade should go down. I could swap out that blade for one of the other blades, hoping for little downtime. Once I have some kind of HA setup, maybe my only option.

[virtualesxer]

Moved....