Fiber and SCSI SAN/DAS connections directly to VM's (RDM)

[leejb]

We currently have a very generic 3 host implementation of ESX 4 where all the VM's are using vmdk's for disk or mapped drives to external storage.  We are looking into updating to ESXi 4.1 and potentially virtualizing two of our primary file servers, however these boxes connect to multiple storage volumes on Fiber and SCSI units.   These volumes are 4TB and Over.  For example, Server A connects to 2 x 4TB volumes via Fiber (switched) and 2x2TB volumes via SCSI DAS.  Server B connects to 1 x 4TB volume via Fiber and 1 x 4TB volume via SCSI DAS

I'm trying to get a handle on directly connecting storage to VM's, but not finding a resource that speaks directly (at least that I'm able to decern in quick reading).   My understanding is that RDM or Fiber NPIV are potential methods.   NPIV seems to be out of reach as the switch must be a in NPIV mode that rules out standard SAN usage (i.e. I would have to have a dedicated switch just for NPIV.)  On top of that, Brocade switches seem to only support NPIV if they are fully stocked (i.e. a E300 must have all 24 ports active... that's expensive to say the least).

RDM's seem to require total ownership of the connector.  i.e. I would have to have an HBA per VM per storage volume.  On a ESX host that I virtualize both Server A and B, then have 1 VMFS volume via Fiber connection, I would have to have 4 single port/ 2 dual port HBA's and 3 channels for SCSI.

Basically, my understanding is there is no way of using a single HBA or SCSI card to connect direct multiple external storage to various VM's is just not possible (i.e. like NAT is to network, NPIV is the closet to FIber and nothing available for SCSI).

iSCSI is one solution, but I currently have no iSCSI storage nor the funds to migrate current storage to such a medium.  Also have some concerns regarding performance of iSCSI... but I think that's mostly from ignorance of the tech than reality.  I've looked at some Fiber to iSCSI routers (e.g. sandbox 6140), but these seem very cost ineffective as well.

Do I have any options here, or is remaning physical with these server solutions my best option for now?

[idle-jam · Accepted solution]

With RDM, you could have the LUN present to the HBA WWD then from that stage onwards you can only add it to the particual VM. So it's like Many LUN, One HBA, and Many VM. so it would feed your needs.

[leejb]

Thanks for the response and sorry for the delay.  I will do some more reading up.  I've upgraded one of my hosts to ESX4.1 to do some testing with a few spare LUNS on the storage.  I will do some more reading of WWD, but frustrated that I've taken the opposite from  what I've read so far.  First though I have to fix another issue from the upgrade.

If you happen to have handy any KB or posts that detail Many storage LUN <> 1HBA <> VM's w/o using NPIV, please post.

[opbz]

hi,

as stated before with RDM you can present mulpile RDMs to multiple VMs.

there are several limitations though.

1: a RDM can only be presented to 1 VM on a ESX hosts. If you want to present it to multiple VMs those VMs need to be all on different ESX hosts.

2: you do still have a 2TB limitation. so your RDM LUN needs to be be slightly less than 2TB this is the messy bit.Only ways I know to get past this are

     a: use a software iscsi inititator within the VM itself

     b: use software raid within the vm.

[leejb]

As long as I can pose multiple RDM's over the same HBA, I only want 1 VM to see it.  The 2TB limit is a huge road block though, I didn't realize this applied to RDM's, I thought this was a VMFS limitation only.

[idle-jam]

yes it's applies to both virtual mode and physical mode of RDM. the only datastore that does not get a limit if NFS.