Microsoft Clustering Fail (Shared Virtual Disk Issue)

[AlexSmith]

Hello,

Getting an issue similar to this guy:

http://communities.vmware.com/message/928200;jsessionid=3AA6E6986093A68E3CB386AE95CC0E19

Where I try and change the SCSI bus to using Physical, and I get 'Invalid Configuration for Device 0'. I, however, am not using that version of esx, and it seems to be pretty supported on this one(?) - VMWare ESX Server, 3.5.0, 110268.

Any ideas?

Cheers

[Texiwill]

Hello,

Moved to VI: Virtual Machine and Guest OS forum.

Setting up MSCS requires careful reading of http://www.vmware.com/pdf/vi3_35/esx_3/vi3_35_25_u1_mscs.pdf as some of the steps you need to perform are not within the text, but the examples. Switching from Virtual to Physical is not the only steps you need to take when you go from Cluster in a Box to across boxes.

Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education.

CIO Virtualization Blog: http://www.cio.com/blog/index/topic/168354

As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization

[hphuhtin · Accepted solution]

Don't use NFS or iSCSI for the shared storage

See http://www.vmware.com/pdf/vi3_35/esx_3/r35u2/vi3_35_25_u2_mscs.pdf

[AlexSmith]

Hello.

This was followed up on IRC, and I managed to grab an answer that "satisfied" me

It seems that when using NFS or iSCSI as the datastore, setting LSI to using 'Physical' or 'Virtual' isn't supported. The only method around this for my implementation is to use an iSCSI initiator from within the VM.

Thanks again!

Cheers,

Alex.

[SquareVM]

I also posted this at http://communities.vmware.com/message/1027128#1027128, but wanted to post in this "Virtual Machine and Guest OS" forum too.

Has this issue been resolved? I am not using shared to network storage. I only have local server storage, but I still see the "Invalid configuration for device '0'." error pop-up

message....

I'm also having this same issue on 4 different ESX 3.5 Update 3 Build 123629 servers and 1 ESXi 3.5 Update 3 Build 123629 server. Both have no extra patches installed. These servers only have local SCSI drives that are in a RAID 5 config - no external, shared storage. All VMs and extra virual drives are on the servers' local storage.

My steps are:

1. Logging onto a command session of an ESX (and ESXi) server as root and creating a virtual disk with the following command:
   
   vmkfstools -d eagerzeroedthick -a lsilogic -c 1G /vmfs/volumes/storage1/SHARED-DRIVES/test-disk.vmdk

2. Then I open an existing 2003 Standard Server VM's setting dialogs and add the virtual disk to the VM by connecting it to a separate SCSI controller that what the VM operating system disk is connected to . This works OK.
   
   My OS VM disk file is located at /vmfs/volumes/storage1/<hostname>/<hostname>.vmdk

3. Then I open the VM's settings again and set the new hard disks SCSI controller's "SCSI Bus Sharing" to "Virtual" and select the OK button. Then I get the "Invalid configuration for device '0'." error pop-up message.

Is this supported in ESX or ESXi 3.5 Update 3 Build 123629? Am I doing something wrong?

This used to work for me on ESX 3.01. ... 😐

[SquareVM]

I believe that OlivR in discussion http://communities.vmware.com/message/1068640#1068640 found my issue.

When I had a snapshot of my VM, I could not change the extra virtual disk's "SCSI Bus Sharing" to "Virtual" in the VM's settings. When I deleted my snapshot, I was able to set "SCSI Bus Sharing" to "Virtual". This means that I cannot take a snapshot of a VM with extra virtual disks like for a MS cluster. 😐

I know a snapshot cannot be taken while a VM's "SCSI Bus Sharing" setting is set to "Virtual", but with ESX 3.01, I used to switch the "SCSI Bus Sharing" setting over to "None", take a snapshot, and then switch "SCSI Bus Sharing" back over to "Virtual" before booting up the cluster again. This worked fine in 3.01, but in 3.5 it seems like a snapshot cannot exist for MS cluster VMs or VMs that need to use a "Virtual" "SCSI Bus Sharing" setting. .... right??????

Is this a bug in ESX 3.5 that will be fixed? I need to be able to take snapshots of my MS clustered VMs and use them!

[marcelo_soares]

I really think snapshots are not possible in MSCS, and more in separate ESXs. You can imagine the problem by thinking in what will happen if you take a snapshot of node A and after sometime this node fails. The node B, when coming back, will obviously not see the changes made to the delta disk, which is only attached to node A.

You can say that to resolve the problem is easy: attach the snapshot simultaneosly on both VMs, but this may involve more difficulties in all process - that's should be the reason that VMWare did not support it yet. I really don't know if this is supported on 3.0.x, but if someone asks me I would really say it don't.

Marcelo Soares

VMWare Certified Professional 310

Technical Support Engineer

Linux Server Senior Administrator

[SquareVM]

My clustered VMs are only for development purposes - ie. not a production MSCS cluster of VMs - to test my software in a MSCS cluster environment. I use snapshots of MS clusters to get back to a clean (none-of-my-software) install state. This worked **great** on ESX 3.01 and saved me and my dev. team a great deal of validation and test time because we could always get back to a "clean"/known clustered system state, but now it seems impossible with ESX 3.5 Update 3. But, even besides clustering 2 VMs, a stand-alone VM cannot even have a virtual disk if that VM has a snapshot stored with it. I know this worked on ESX 3.01.

Are there any ways to work around this limitation in 3.5?

Is there even a fix for 3.5 to allow setting of "SCSI Bus Sharing" setting to "Virtual" when that VM has snapshots?

When I take snapshots of my VMs (back on ESX 3.01) with virtual disks, I switch the VM's "SCSI Bus Sharing" setting to "None" and then take a snapshot. When reverting to a snapshot, I would power the clustered VMs down via the Windows operating system shutdown, switch the VMs' "SCSI Bus Sharing" setting to "None", and then revert. Before the VM is powered back up, I would switch "SCSI Bus Sharing" back to "Virtual". Again, all of this worked great in ESX 3.01.

There is a web page () that kind of implies that snapshots are possible with ESX 3.5 Update 2-3 for cluster-in-a-box for local storage VM disks (which is what I'm doing), but I cannot get this to work. ..... well, I'm wrong about this web page implying snapshots can be used with cluster-in-a-box (CIB) on ESX 3.5. Sorry...

I have tried to 'see' what removing a snapshot from a VM does to the VM's config files and then manually simulate snapshot removal without actually removing the snapshot files - to try and hack around this virtual drive setting limitation and use snapshots with CIB, but my manual edits of the VM configs and rename (not delete) of snapshot files do not seem to have an affect.