Microsoft Clusters - Majority Node Set (MNS) Vi3

[conyards]

Posting this to hopefully generate some discussion on MS Clustering within Vi3 infrastructure

Following on from the London user group meeting and brief discussion regarding MS clustering on Vi3, I decided to have a go a configuring an Majority Node Set cluster. The aim being to move away from the requirement to have a shared Quorum raw device mapping. and therefore be able to provide basic cluster group fail over services in tandem with VMotion, DRS and HA.

The setup was as follows;

\- Two ESX 3.0.1 hosts running upon IBM LS41 blades.

\- Network traffic was handled by one vswitch, consiting of two teamed nics, trunked to two Vlans (one for private and one for public traffic) (I know the document states that teams cannot be used for MS Clusters in Vi3, but there is no choice when dealing with the LS41)

\- VMs where running MS Windows 2003 Enterprise, 1 vCpu, 1024MB RAM, 2 vnics, 2 LSI logic scsi controllers, VMDK1 housing the OS attached on scsi 0:0, VMDK2 housing MS volume attached on scsi 1:0. (NODE2 in the cluster was a clone of NODE1). nb. For increased stability MS recommends creating an MNS cluster utilising 3 nodes.

Note neither SCSI card was engaged in BUS sharing.

After the OS' where configured with the relevent IP, partition, Naming etc and connectivity was tested on the public and private interfaces, I created the cluster.

For the inital configuration I accepted the defaults on the first node, and made sure that the quorum resource was set to local, pointing at pre labelled Q:\ volume. I then followed the documentation at the following link;

http://technet2.microsoft.com/WindowsServer/en/library/d58091c1-f31e-46ec-b7e1-40751add28a71033.mspx...

to create a MNS resource and ensured the Clusters Quorum volume was changed to reflect this.

the second node was added into the cluster, it automatically picks up the MNS settings.

I created a new cluster group under the protection of the cluster and created a number of dummy applications, in reality these where just instances of notepad & calc. I then checked that I could successfully initiate failover of this group, which it would let me do.

I then checked that VMotion worked and yet again it let me VMotion VMs between hosts, the assumption here is that DRS will also work. Regretabley, I'm unable to test the response to HA in this environment.

I will be rebuilding this test environment and adding a third node to the MNS cluster so I can test MS Cluster fail over response when I start turning off active cluster nodes. I hope to also add a more meaningful application/service to the Cluster.

Hopefully results from there will show that an MNS cluster can provide some sort of resilience to basic application layers, and perhaps more advanced applications when twinned with a form of data replication.

Any thoughts?

Anybody use Virtual MNS clusters in production, with VMotion, HA, DRS? what type of applications?

Anyone replicating data for use within a Virtual MNS cluster?

Thanks for reading.

Simon

[conyards]

A three node MNS cluster appears to work also.

Any thoughts?

[conyards]

Attempted the installation of more complex environments today, with mixed success, but essentially back where I was at the start...

I wanted to install SQL onto the 3 node cluster. For this install to work in required the creation of an MSDTC resource, which unfortunatly requires... Physical disk resource, which was kind of why I wanted to test an MNS cluster to avoid that requirement...

I continued in the attempt just to see what functionallity I could get.

I installed a third SCSI card into the Nodes, and created a thick VMDK file for use;

vmkfstools -c \{size}m -d eagerzeroedthick \{vmfs path}\{vmdk name}.vmdk[/i]

I then initially put the thrid scsi adaptor into Virtual Bus sharing mode and alligned all the VMs on the same host.

I configured MSDTC upon the cluster and initiated a failover of the group, the shared physical disk resource failed over as espected with the MSDTC resource.

VMotion in the above mode wouldn't work, but HA did restart the nodes. Although afterwards node 2 saw the shared resource as corrupted and required some manual intervention, the cluster still came up.

So possibly with such a configuration it may be possible to provide VMware HA functionality with Microsoft Clustering.

If I get time this week I'll try and confirm how physical mode bus sharing works.

[wobbly1]

Have you tried to fail a server yet rather than just a failover. Being a mns cluster will you not need a min of 3 nodes so that if 1 fails you still have 2 (majority)? Is your plan in this to have 3 or more?

[juchestyle]

Hey Simon,

I built an exchange cluster on my lab box this weekend using cluster in a box and a shared disk. Inside the box works like a champ! Can't test drs or HA on the lab box though.

One thing I noticed, you didn't specify lsi logic in your creation of the quorum disk; it defaults to bus logic if you don't. And I believe that vm requires you to use lsi logic.

Not sure if it matters, but wanted to contribute to your success.

Respectfully,

[conyards]

Hi Guys thanks for responding

Yes, I have failed over servers, this was my motivation for getting a third node into the MNS cluster, works just fine

With MNS the Quorum volume is just another standard VMDK, when it was created it defaulted to LSI logic. The Virtual shared SCSI bus disk on the other hand did default to BusLogic and did as you state require intervention to set them as LSI Logic.

Anyone else tried MNS clusters?

[stvkpln]

So possibly with such a configuration it may be

possible to provide VMware HA functionality with

Microsoft Clustering.

If you're planning to run your clustered Windows instances in a supported (by VMware and Microsoft) state, the boot volumes for your VM's \*MUST* reside on local disk. This is more so a Microsoft requirement, but VMware has stated it clearly in their documentation that the only supported configuration for Microsoft clusters is this way..

Therefore, HA/DRS don't come into play, they have to be excluded.

[conyards]

Thank you for the reply, I have read the whitepaper and am aware of the caveats that are placed upon virtualising clusters, this is more an exercise in what seems to work and what doesn't... supportability doesn't come into it.

lets face it, if I wanted a 100% vendor supported platform for everything, then it wouldn't be using VMware...

[conyards]

SQL 2005 was installed onto the 3 node cluster today

The application installed as expected, all cluster resource was created during installation. Group failover works, as does hard frailover when a node goes offline.

tommorrow may provide an opertunity to test VMware HA functionality with this cluster.

[conyards]

OK, after a week of contiuous operation SQL is starting to get some write errors to the shared VMDKs...

I'm going to look more in depth to see why this has happened and will post here again.

Still very interested to hear any other experiences with Virtual MSCS.

[trentg]

I think that saying Microsoft has a local disk requirement for cluster nodes boot volumes is misinformed and I'm very far from a MS expert. To quote this MS document,

"

Server Clusters : Storage Area Networks - For Windows 2000 and Windows Server 2003

Microsoft Corporation

Published: January 1, 2003

...

Booting from a SAN

Microsoft supports booting from a SAN in limited environments. There are a set of configuration restrictions around how Windows boots from a storage area network, see KB article 305547.

Windows 2000 Server clusters require that the startup disk, page file disk and system disk be on a different storage bus to the cluster server disks. To boot from a SAN, you must have a separate HBA for the boot, system and pagefile disks than the cluster disks. You MUST ensure that the cluster disks are isolated from the boot, system and pagefile disks by zoning the cluster disks into their own zone.

Note

Windows Server 2003 will allow for startup disk and the cluster server disks hosted on the same bus. However, you would need to use Storport minioprt HBA drivers for this functionality to work. This is NOT supported configuration with in any other combination (for example, SCSI port miniport or Full port drivers).

"

That was almost 5 years ago. Our shop has booted non-clustered Windows servers from SAN for years with zero issues. I don't see a roadblock yet as we take MSCS clustering on.

[Dave_Mishchenko]

Here's a good explanation of why VMware requires the boot disk of servers in a MS cluster to be on local VMFS: http://www.vmware.com/community/message.jspa?messageID=639571#639571