how to set up vmware channel between primary and secondary vCenter servers on two different ESXi hosts

[thuang2]

Hi,

I am trying to set up create a vDS (for VMware Channel IPs) for the two different ESXi hosts but it does not work.

[ssbkang]

Make sure the VLAN is in layer 2, otherwise, Channel Network won't work.

[thuang2]

> Make sure the VLAN is in layer 2, otherwise, Channel Network won't work.

Would you elaborate on this? How?

In this case, I did not add any physical adapter as dvUplink and tried to configure PVLAN. Is there a VMware KB that

provides instructions for configuring vDS in such scenario? Thanks!

[ssbkang]

What I meant by Layer 2 VLAN is no gateway, no DNS.

So when you configure network settings on the adapter (Both Primary and Secondary):

• IP Address: x.x.x.x
• Subnet Mask: x.x.x.x
• Gateway: NULL
• DNS: NULL

Also, the Channel Network uses port 57348 by default and this has to be manually opened from the Windows Firewall (Heartbeat installation doesn't do this automatically).

If there is no dvUplink, obviously, packets cannot be transferred from one ESXi to another. You need the uplinks.

In my case, I didn't configure PVLAN, only a portgroup with VLAN used.

Could you provide me more details:

• How many and which VLANs are configured (from the switch side) on the interfaces
• Which VLAN is being used for Channel Network

-Steven

[thuang2]

Thanks. 

I added uplinks (i.e., physical adapters) to the vDS. I have configured the network settings on the adapter (as mentioned above) and turned off the firewall on both primary and secondardy.

On my vDS, I have two port groups (one for public and management IPs and one for VMware Channel)

On one of the port groups (the one for VMware Channel), I've configured a VLAN (with ID 10).

At this point, VMware Channel is still not working

[ssbkang]

So both Primary and Secondary are using VLAN10 portgroup right?

Two things to check:

• How many uplinks have you added?
  • If more than 2, have you configured LAG? (Etherchannel...etc)
• VLAN10, is there a gateway?

[thuang2]

> How many uplinks have you added?

> If more than 2, have you configured LAG? (Etherchannel...etc)

I've reduced number of uplinks to 1. But, why do I need to configure

LAG (Link Aggregation Group) when uplinks are more than 2?

> VLAN10, is there a gateway?

How do I configure a gateway for VLAN10? I have two ESXi that are

on two different subnets. Which gateway should I use for the VLAN?

[thuang2]

BTW, I am using vCenter server 5.1.

[ssbkang]

No, you don't have to configure LAG.

If LAG is configured, then the portgroup's load balancing should be configured to Route based on IP hash, that's why I asked.

You should not use a gateway for Channel Network which means, two channel adapters must be in a same subnet.

ESXi servers can be on two different subnets but for Channel Network, they must be in a same subnet!

For example:

• Primary Channel Adapter=> 130.216.1.10
• Secondary Channel Adapter => 130.216.1.11

As per the screenshot attached, no gateway no DNS!

[thuang2]

What I am trying to accomplish is to set up vCSHB in WAN environment:

I have two ESXi hosts in two different subnets (172.x.x.x and 10.226.x.x). I have the primary and secondary (both are virtual machines)

servers with primary on one ESXi host and secondary on the other.

To set up a Channel network, I've created a vDS with vLAN ID 10. On primary (VM), Channel adapter => 192.168.1.100 (no gateway, no DNS); on secondary (VM), Channel adapter => 192.168.1.101 (no gateway, no DNS). Did I miss something here?

Also, what do you mean by "two channel adapters must be in a same subnet"? Are you referring to vNIC from the VM or physical network

adapter from ESXi host? Thanks!

[ssbkang]

So the VLAN10 is spanned across two data centres, right?

Simply, can you ping 192.168.1.101 from 192.168.1.100?

I meant that as long as the Primary can reach Secondary on Layer 2, it should work.

[thuang2]

The two esxi hosts are managed by the same vCenter server and the same vDC but they are in two different physical locations (i.e.

two different physical data centers). Yes, VLAN 10 is spanned across two (physical) data centers.

But, I still cannot ping 192.168.1.101 from 192.168.1.100 or vice versa? What went wrong here?

[ssbkang]

Then it's quite strange, it should work.

Have you configured VLAN10 on the dvSwitch port group?

1. Right click on the portgroup
2. Click Edit Settings
3. Click VLAN
4. Ensure VLAN 10 is assigned

[thuang2]

In this case, if I use a non-private IP with default gateway and DNS (on both primary and secondary) for Channel network, that should work, right?

That means, Channel IPs on primary and secondary are not on the same subnet. 

I believe the Channel network is just for syncing files, registry keys and sending hearbeat back and forth.

[thuang2]

> Have you configured VLAN10 on the dvSwitch port group?

1. Right click on the portgroup
2. Click Edit Settings
3. Click VLAN
4. Ensure VLAN 10 is assigned

Yes, I have done that, but it does not work.

[ssbkang]

Not quite sure what the root cause is...

The following is from the vCSHB documentation.

Channel Network connection(s) configured with the following:

• Static IP address in a different subnet than the Principal (Public) network with a different IP address than the Secondary server channel NIC
• Correct network mask
• No Gateway IP address
• No DNS server address
• NetBIOS enabled (disabled during the installation process)

Hence, if you configure non-private IP with Gateway, the servers will be able to talk to each other but Channel Network won't work as stated above.

[thuang2]

> VLAN10, is there a gateway?

I don't think there is gateway for VLAN10. Do I need to config one for the vDS? If so, how?

[thuang2]

BTW, I've configured a multiple-NIC on both primary and secondary.

Both vNICs (on both primary and secondary) for public IP/mgmt IP and Channel IP use the same dvUplink (i.e., physical

network adapters).

BTW, I only have two physical network adapters on each ESXi host (one for vmk/mgmt console and one for vCSHB).

Would that make any difference?

[ssbkang]

Wait a sec, so do you use both Standard switch and Distributed switch?

Standard switch for ESXi Management and dvSwitch for vCSHB?

Also, could Primary server ping Secondary management IP? Is it only Channel Network they can't communicate?

Number of physical network adapters won't matter as long as they are configured correctly.

[thuang2]

> Wait a sec, so do you use both Standard switch and Distributed switch?

> Standard switch for ESXi Management and dvSwitch for vCSHB?

Yes, I use vSS for ESXi mgmt and vDS for vCSHB. But, does that matter?

> Also, could Primary server ping Secondary management IP? Is it only Channel Network they can't communicate?

I can ping secondary mgmt IP from primary and vice versa. I believe it is only Channel Network (IP with no gateway and no DNS) cannot communicate.

Thanks!