Massimo1980
Contributor
Contributor

ESX clustering with (virtual) shared SCSI disks?

I'm testing Virtual Infrastructure in a virtual environment, running ESX 3.5 U4 and VirtualCenter 2.5 U4 inside VMWare Workstation 6.5.2.

It works, if the hardware supports it (Intel VT) and if the ESX VMs are properly configured (monitor_control.restrict_backdoor, etc.). VMs in a virtual ESX server can be powered up, and they run with good enough performances, given this setup.

I want to test the whole Virtual Infrastructure here, so I created two ESX VMs and a VirtualCenter VM; I joined the two ESXs to the VC cluster, created an additional virtual network for vMotion and configured everything, both in the ESX servers and in the VC. What I need now is shared storage.

I'm familiar with the cluster-in-a-box model with virtual Microsoft clusters, but I also know ESX officially supports only SAN, iSCSI or NFS storage for clustering, while a shared VMDK file can only appear as a shared SCSI disk to the VMs. What I want to know is:

  • Do ESX clustering work (even if totally unsupported) with physical shared SCSI disks?

  • If the above is true, can it also work with virtual shared SCSI disks (cluster-in-a-box)?

I of course tried this setup, but I got quite mixed results:

  • The two virtual ESX servers both see the shared virtual disk

  • The datastore on the shared disk is recognized by VirtualCenter as a shared datastore

  • VirtualCenter allows creating a clustered VM on the datastore

  • The VM can be powered up on a single host

  • When trying vMotion, validation is successful if moving it to the other host

  • The vMotion operation howewer fails with a "invalid fault" error

  • Even if the VM is powered off, moving it from one host to the other also fails

Can someone please help here?

P.S.: Please, don't answer "use iSCSI"; I know that, I just want to know if virtual (or physical) shared SCSI disks can work.

0 Kudos
7 Replies
weinstein5
Immortal
Immortal

No you will not be able to use a shared virtual SCSI disk - what you will need to do is build a machine and use an Open Source iSCSI/NAS type programe to provide shared storage like OpenFiler or FreeNAS -

If you find this or any other answer useful please consider awarding points by marking the answer correct or helpful

If you find this or any other answer useful please consider awarding points by marking the answer correct or helpful
0 Kudos
Massimo1980
Contributor
Contributor

No you will not be able to use a shared virtual SCSI disk

Is there any specific reason for that?

I'm actually encountering problems with this setup, so you're probably right; but can you (or someone else) confirm that shared SCSI (be it physical or virtual) doen's work for ESX clustering?

0 Kudos
NWhiley
Enthusiast
Enthusiast

I've not used Workstation for this, so it may be something related there.

But I am running a two ESX cluster with VIC inside Server 2.0.

I haven't tried clustered VMs inside my (virtual) ESX cluster, but Vmotion, DRS and HA all work.

I can send you the vmx files if you want to take a look at the config?






VCP

Neil VCP
Massimo1980
Contributor
Contributor

I've not used Workstation for this, so it may be something related there.

But I am running a two ESX cluster with VIC inside Server 2.0.

I haven't tried clustered VMs inside my (virtual) ESX cluster, but Vmotion, DRS and HA all work.

I can send you the vmx files if you want to take a look at the config?

It could very well be a Workstation issue, as it officially doesn't support VM clustering (and it warns you about it). I'll try building a Microsoft cluster in it, and/or running everything in Server 2.

My .vmx files include the standard setup for cluster-in-a-box (scsi1.sharedBus = "virtual", disk.locking = "false") and ESX-specific settings (monitor_control.restrict_backdoor = "true", monitor_control.vt32 = "true", guestOS = "vmkernel"); feel free to post your ones.

What are you using for shared storage?

0 Kudos
NWhiley
Enthusiast
Enthusiast

My shared storage is just local disks inside the physical host. This is just a POC set up for me and to allow me access to ESX at home so I can work through the CLI stuff in my own time.

The only setting you didn't mention is the SCSI mode, which I have set to independent-persistent.

So for disk 1.1 it would read

scsi1:1 mode = "independent-persistent"






VCP

Neil VCP
0 Kudos
Massimo1980
Contributor
Contributor

You're right about the "indipendent-persistent" stuff, I didn't mention it but it was configured.

The idea of using a physical disk as shared storage is also nice, but... how do the VMs see it? Isn't it presented to them as... a shared SCSI disk?!?

0 Kudos
Massimo1980
Contributor
Contributor

I've not used Workstation for this, so it may be something related there.

Turned out it actually was a Workstation issue: cluster-in-a-box just doesn't work in Workstation 6.5.2. I tried installing a standard 2-nodes Microsoft cluster, and I had exactly the same problem.

I re-created everything in Server 2.0, and now vMotion works great.

So, the bottom line is: shared SCSI disks, be them physical or virtual, do work for ESX clustering; they just don't work inside Workstation.

Now, one could wonder why exactly the premium consumer VMWare product doesn't support internal clustering, which is often used for testing and developing (which is the main focus of Workstation)... but I can't even imegine a valid reason for that. I'm just glad I didn't buy Workstation.

Thank you for all help.

0 Kudos