Netapp Storage Planning - Questions

[virtech]

I'm planning a VMware design using iscsi HBA's (QLA4052C) connected to FAS270 or FAS3000 storage processors.

How have people been configuring their storage for VM's? I have a requirement to use Snap Manager to backup some SQL databases.

Previosuly I have used RDM's for each VM using its own LUN but want to move away from this configuration and use VMFS-3 datastores for ease of manageabilty etc.

If I use Netapp San technology to Snap the LUN containing the VMFS-3 Datastore which has multiple VM's within, will I be able to restore individual VM's from this LUN? or will it be restore the whole LUN or nothing?

[Jae_Ellers]

Here's the deal with NetApp:

If you want lightning fast restores, use RDM and one VM per volume. This will allow you to restore from a previous snapshot in seconds. This incurs the overhead of setting up a volume, qtree, lun for each VM, but ultimately makes the snap, restore, replication, etc. very simple.

If you want flexibility and restore capability, use VMFS on bigger luns to host shared storage (typically 300-800 Gb) among several to many VMs. Stagger higher I/O among different luns for the obvious reasons. If you need to restore a vm you present the appropriate snapshot as a new lun, enable resignature on a single host, then attach to the snapshot. Using local ESX tools copy the vmdk back to the original lun, overwriting the broken disk. Here you have the ESX file transfer time to contend with, so this is probably on the order of 5-30 minutes for restore depending on vmdk size & connection protocol & bw.

Right now I use large shared storage luns, but am going to start migrating critical VMs to RDM for the instant restore functionalitiy. By using both methods you can keep you lun count down but have quick restores when you need it.

Netapp is working on various software tools to make this easier, but right now to do it right takes some scripting. They do provide sample scripts which are VC aware, so it's simple to suspend or VM-snapshot all VMs on a lun then NetApp-snapshot the VMs without tracking down which witch is which.

As to SnapManager for MS SQL it will need a VM side iSCSI initiator. Contrary to what I'd thought this actually works as well or better than iSCSI in the vmkernel, so unless you're pushing 50+ MB/s all the time I doubt you'll have any problems with the client side tools. We do this as well.

Some good things to remember when spec'ing your platform:

1. 30xx are much more expandable. New 3040 & 3070 will support SAS in the head unit and supplant the 3020 & 3050 RSN. 4Gb SAS shelves are coming soon or out now. 270 are EOA soon, but a 20xx series is available/coming soon.

2. Don't be cheap and try to getaway with SATA. We got bit by this early on. Get as many FC/SAS spindles as you can for the aggregate, use it for other things as well.

3. NetApp supports TOE on the head nics if you order the right interfaces. This is supported in Data ONTap 7.22.

[shaneyoder]

Jae,

I have seen on multiple sites that NetApp is "working" on a SnapDrive/SnapManager product that will integrate with ESX to perform the VM snapshotting functionality that you must use scripts to acomplish today. Can you shed any more light on this? Have you discussed this with NetApp? Is a product forthcoming? I have talked to the NetApp folks in my area and they keep telling me they haven't heard of anything like this.

Thanks

[Jae_Ellers]

I have no more visibility of this than you, I've just been to a couple NetApp dog-N-pony shows in the last couple weeks and we use a fair amount of NetApp here in engineering.

My reading of the tea leaves is that NetApp recognizes the need for this and is working hard to fill the void. However, they also realize they can't promise anything and not deliver on time. So at this point there are no committments, nor any other details that I have.

The scripting isn't too bad and sample scripts are available. I'll make sure all the scripts you might want are available and post some links for those.

[virtech]

Jan thanks for the reply.

I don't think the restore process needs to be lighting fast but it needs to be simple so the customer can manage their restores of VM's if required easily and efficiently..

I'm not too keen on the idea of presenting a snaphot as a new LUN, enable resignature on host then copy the vmdk around.

My thinking is at the moment i'll create the SAN storage in the following way unless there is a better way?

Create a Flex Volume

Create a Qtree for each LUN

Create a LUN for each VM, use RDM to configure the VM.

I guess with the above approach I should then be able to restore an individual LUN from the Volume after a snapshot?

I haven't used a iSCSI initator inside a VM before, would I just allocate storage for the C drive of the SQL server and then use the iSCSI initiator to connect to another LUN to hold the SQL databases?

[Jae_Ellers]

Sounds like a good middle ground and approach to try. The only reason to separate volumes is that that is where the snapshots are controlled. So if you have a bunch of vms on a volume and you want to ensure some consistency you need to quiesce all of them before pulling the snapshot trigger. Let us know how your testing goes. Leverage your NetApp guys, as they should be able to get a lot of help for this.

I haven't configured the latest SnapManager, but the process goes something like this:

1. Bring the system up with a typical c: drive.

2. Read the SnapManager docs regarding sizing of the lun(s). Create & present luns.

3. Create the database with the data on c:

4. Install the iSCSI initiator. Mount luns.

5. Install SnapManager

6. Run SnapManager. RTFM, but basically you select a db to be managed & it is migrated onto the iSCSI lun for you. You can then setup scheduled snapshots after it's migrated.

[christianZ]

What kind of servers will you use?

I heard here about support problems by hp with qla405x.

[virtech]

Probably HP, not sure of the model yet though

[Jae_Ellers]

Good point, Christian. Thanks for the reminder.

HP hardware iSCSI support for VMware ESX is nonexistent. http://www.vmware.com/community/thread.jspa?messageID=660475#669504

Looks like HP is hanging onto their Storageworks market share by not allowing people to use iSCSI hardware. Others have posted that HP's policy is that if it's not in the quickspec it's not supported. I only see HP rebranded FC hbas here: http://h18004.www1.hp.com/products/quickspecs/12575_na/12575_na.html

I guess HP is the new IBM. IBM, however, offers these options and doesn't seem to care if they're IBM rebranded or Qlogic.

Today your only option for HP and iSCSI seems to be SW initiator. IBM offers a full selection of Qlogic 405x adapters in their blades and rack-mount systems.

That will keep our iSCSI implementation on our IBMs and limit our future use of HP blades. Good thing you mentioned this. We're slated to purchase $125k of new HP blades for ESX in a couple weeks. I'd better review this with our operations lead and see if she still wants to stay with the HP blades.

[virtech]

Wow thats going to be an issue!

[virtech]

Jae, I have heard that you can only use Snap Manager with the software initiator ? not the iSCSI HBA ?

[virtech]

I heard from a good source its on the way but no ETA

[shaneyoder]

SnapManager only works with software initiator becuase SnapDrive has to be the one managing the LUNs. The only way to get SnapDrive to manage the LUNs inside a VM is to use the software initiator because the VM can't access the HBAs directly. I heard that in a future release ESX will support some virtual HBAs for FC, maybe iSCSI is being planned as well.

[shaneyoder]

I just heard yesterday from a reseller that we should see an announcement in 3rd or 4th quarter. I am very interested in what the capabilities will be. Maybe some info can be found out a VMworld. We'll have to wait and see!

[virtech]

Hi, I posted a seperate question regarding this:

http://www.vmware.com/community/thread.jspa?threadID=96623&tstart=0

I understand what your saying but are you referring to the MS Software initatior or the ESX software initiator ?

[Jae_Ellers]

Answered your MS vs. HBA question in the other thread.

As far as NetApp productizing the interface for VMware, right now they are very tight with the info other than saying they recognize the gaps in making it easy to use right now and they're working on it.