ESX SAS/SCSI filesystem (ext3) and multiple paths

[namtrok]

Total VMWare ESX Noob here.

Running a Proof of Concept.

IBM x3550 redundantly connected to a IBM DS3200 using SAS cables. (dual HBAs dual Controllers)

The DS3200 is used as our SAN, the LUNs/Logical Drives appear as local storage not iSCSI.

I have 15+ LUNs created formatted as ext3 and data stored on each ~2TB.

The Plan/Desire:

1. Install ESX (3.5.0 Update 3) to the x3550 use a new LUN as VMFS to store my Virtual Machines. (Done)

2. Build a Linux server as a VM (Done)

3. Serve a LUN with existing data from the DS3200 to the ESX server to the Linux server. (Not possible?)

From what I have read I need to either connect via NFS to a datastore or have it (re)formatted as VMFS destroying all the data on it, then create a virtual disk on it. Am I correct in this? Is there any way to pass a SCSI device from the ESK server to the VM Linux without making it a VMFS?

Also, Is ESX capible of redundant paths, Multipathing-RDAC-etc? my DS3200 is complaining that the second path is not connected and only one hba is showing up in my ESX console.

Thank You for your help. I need to know this before moving forward. Copying all my data from ext3 to vmfs is doable but extremly time consuming.

[Texiwill]

Hello,

1. Install ESX (3.5.0 Update 3) to the x3550 use a new LUN as VMFS to store my Virtual Machines. (Done)

Good plan.

2. Build a Linux server as a VM (Done)

Great.

3. Serve a LUN with existing data from the DS3200 to the ESX server to the Linux server. (Not possible?)

Possible but not through the VIC as an RDM but you should be able to as a SCSI Generic Device. If not that way then you would use vmkfstools from the CLI to make an RDM that points to the LUN. Then use that RDM. It may also be possible to create a passthru-scsi device. You can try using the following to make a passthru rdm. or the -r option instead of -z to make a real RDM. One should work.

vmkfstools -z /vmfs/devices/disks/vmhbaW\:X\:Y\:Z /vmfs/volumes/STORAGE1/VMNAME/VMNAME-rdm.vmdk

There used to be a more convoluted method but that does not work any more either.

From what I have read I need to either connect via NFS to a datastore or have it (re)formatted as VMFS destroying all the data on it, then create a virtual disk on it. Am I correct in this? Is there any way to pass a SCSI device from the ESK server to the VM Linux without making it a VMFS? This used to be possible

Also, Is ESX capible of redundant paths, Multipathing-RDAC-etc? my DS3200 is complaining that the second path is not connected and only one hba is showing up in my ESX console.

Not for SAS, only for iSCSI/FC-HBA.

Thank You for your help. I need to know this before moving forward. Copying all my data from ext3 to vmfs is doable but extremly time consuming.

Actually you would end up copying it from ext3 to a VMDK running ontop a VMFS, not into a VMFS.

Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education.

Blue Gears and SearchVMware Pro Blogs: http://www.astroarch.com/wiki/index.php/Blog_Roll

Top Virtualization Security Links: http://www.astroarch.com/wiki/index.php/Top_Virtualization_Security_Links

[namtrok]

looking into the RDM choice... why is it that my choice is greyed out (in the GUI) ? similar to the discussion in this thread except the fix does not work for me: http://communities.vmware.com/thread/142008

As far as SAS and redundancy this KB seems to suggest its possible (this is not the problem I'm having, my problem is that only 1 hba shows up)

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=100431...

as to the correction on vmdk - vmfsthanks, that is what I meant.

-Paul

[teovmware]

Is there an answer for that? Is SAS multipathing supported for VMware ESX (assume latest version) or it works but not supported?

[Texiwill]

Hello,

3. Serve a LUN with existing data from the DS3200 to the ESX server to the Linux server. (Not possible?)

This is possible in some cases, you need to create a LUN on the DS3200 and then through the VIC or direct editing of the VMX access this LUN as a SCSI generic device. Other than that, there is no real capability to access a LUN on local attached storage as a RDM.

From what I have read I need to either connect via NFS to a datastore or have it (re)formatted as VMFS destroying all the data on it, then create a virtual disk on it. Am I correct in this? Is there any way to pass a SCSI device from the ESK server to the VM Linux without making it a VMFS?

DS3200 is locally attached storage so it is either VMDK or LUN, not NFS.

Also, Is ESX capible of redundant paths, Multipathing-RDAC-etc? my DS3200 is complaining that the second path is not connected and only one hba is showing up in my ESX console.

Not with SCSI.

Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education.

Blue Gears and SearchVMware Pro Blogs: http://www.astroarch.com/wiki/index.php/Blog_Roll

Top Virtualization Security Links: http://www.astroarch.com/wiki/index.php/Top_Virtualization_Security_Links

[namtrok]

As a follow up.

Texiwill you were right with the vmfstools command. It worked beautifully even on a reiserfs partition. I did have to use the -r option.

I still have the RDM option greyed out in my GUI and don't know why that is.

Also on the two paths issue I have that solved and here is what I know:

1. My problem was in the physical server (IBM 7978) where the two SAS HBA's were showing up in the bios but one was "disabled" There was a PCI device resource allocation failure (1801) which caused the second hba to be disabled. After changing a BIOS setting to disable loading the PCI bios into bios/ram (sorry cannot remember the exact setting) The 1801 error went away and my second HBA showed up in ESX server.

2. From this I have found that for sure all luns have to be on the same path in a SAS SAN (DS3200) situation. But they operate as a failover to the second path.

3. Both hbas do show up in the GUI/ VI3 Client. You can verify that they can see all the appropriate LUNS from the Configuration tab > Storage Adapters.

4. You can right click on a LUN and choose Manage Paths which shows the two paths as Active and ON, this is conforting to me! You can then also click the "Change" button which shows three options "Fixed" "Most Recently Used" and "Round Robin (Experimental)" - I'm currently set to Most Recently Used. I'd love to see that RR get out of the lab and into production. But I'm willing to wait.

So yes, load balancing is not possible. But failover is with SAS SAN devices like the IBM ds3200.

And yes you can offer any LUN with any partition on it to the ESX server and then create an RDM pointer for it and access it via a VM. This is Great news!

[RickySR]

Hi namtrok,

We can see both HBAs on the ESX server, but only one has the LUNs. I suspect this is normal with our setup since the DS3200 is using one HBA for all LUNs.

I have a question, we have the same setup but with a 7979 instead. I can right click the LUNs ending with 31 and see active/on and even change them. But cannot with any of the other LUNs. You described below that you can see active/on, was that for all the LUNs? What steps did you take to be able to do this?

As far as failover, am I right to understand that failover takes place on the DS3200? Then we would just change the prefered path on the ESX?

[dbucher]

Hello,

Your discussion is very interesting to me as we are planning to have the same config.

If we have VMWare ESXi 4 on one server (and ESXi or VMWare server on the other) what filesystem do you suggest on the IBM DS3200 and is it resizable ?

(I saw that vmfs is not free, therefore are there some good alternatives ? )

Thanks a lot for any suggestion !

Denis