Red Hat Cluster Suite, GFS2 and VMware

[norfolknog]

Hello,

I want to implement Red Hat Cluster Suite with a shared GFS2 filesystem on VMware guests running RHEL 5.7. I want the implementation to be fully supported by both Red Hat and VMware. 

This Red Hat document https://access.redhat.com/kb/docs/DOC-46375 states that VMDKs are not supported, but that RDMs (in either Virtual Compatibility Mode or Physical Compatibility Mode) are fully supported when the fence_vmware_soap fence device is used. 

Therefore I intend to place the GFS2 filesystem on one of the two types of RDM disk (and obviously have the Red Hat OS on another disk). I'm a bit confused about the options though! My questions are:

1) Does the RDM have to be added to an additional SCSI controller on the guest, or can it be on the same controller as the local (OS) disk? i.e. can I just have one SCSI controller for the guest, with both OS and GFS2 storage?

2) How is the RDM "Compatibility Mode" option (virtual or physical) affected by the SCSI controller's "SCSI Bus Sharing" option (none, virtual or physical)? Are these just different ways of setting the same thing? If so, which takes precedence?

3) Is it at all possible to vMotion these guests without powering off?

Many thanks in anticipation, experts!
Andy

[weinstein5 · Accepted solution]

norfolknog wrote:

1) Does the RDM have to be added to an additional SCSI controller on the guest, or can it be on the same controller as the local (OS) disk? i.e. can I just have one SCSI controller for the guest, with both OS and GFS2 storage?

No RDMs do not need to be on their virtual SCSI controller

2) How is the RDM "Compatibility Mode" option (virtual or physical) affected by the SCSI controller's "SCSI Bus Sharing" option (none, virtual or physical)? Are these just different ways of setting the same thing? If so, which takes precedence?

RDM Compatability mode is chosen based on the cluster you propose to run - if you are planning to have have the cluster span to ESXi hosts than you will have to select physical compatability. Likewise the SCSI Bus sharing is also selected based on the cluster configuration - physical if you are planning to have a 'cluster -across-boxe' or virtual if you are planning a 'cluster-in-a-box'.

3) Is it at all possible to vMotion these guests without powering off?

No you are not able to vMotion a VM that is part of an OS Cluster -

[norfolknog]

Many thanks for your help David, that has clarified things for me.

For interest, I tried to add an RDM with "Physical" compat mode to the guest's existing SCSI controller, which I had changed to "Physical" Bus Sharing. Unfortunately it didn't boot due to the OS disk being thin-provisioned; the error message seemed to indicate that only thick-provisioned disks (and RDMs, obviously) are allowed on controllers which have Bus Sharing set to "Physical". No matter, I'll just have a second controller.

Now I just need Red Hat to update fence_vmware_soap to work with vCenter 5 - it currently only works with vCenter 4.1. Unfortunately I've been told by Red Hat that this won't be till RHEL 5.9 is released...

Thanks again,

Andy