- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi community,
we have a problem (?).
We have a VSAN enabled cluster with four hosts. Everything looks fine,
- the configuration is good,
- the VSAN status page displays "Network status: (green arrow) Normal",
- the Disk Management page displays "Status: Healthy" for all of our disk groups.
- Even a "esxcli vsan cluster get" on every host returns a "HEALTHY"
But we have a littly yellow exclamation mark on every host "Host cannot communicate with all other nodes in the VSAN enabeld cluster".
Anyone with the same problem? Anyone with an idea or just a hint?
Thank you!
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
This is a physical network switch configuration problem. Make sure you have enabled igmp snooping on your vSAN vlan.
Lars
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Its not simply an alarm left over from when you were configuring the cluster?
What I mean, is that was the VSAN cluster formed before the network was configured? This might explain why all the hosts have this alarm.
Now you have resolved the network issue, but the alarm still needs to be acknowledge.
If that is the case, then all you need to do is acknowledge the alarms and reset them to green.
HTH
Cormac
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thank you for your reply.
@larstr
IGMP snooping is disabled on the vsan vlan because on this vlan is only vsan traffic.
@CHogan
Unfortunately not that easy. ![]()
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Do you have all servers in the cluster with a VMK tagged with VSAN traffic?
Can you ping from all servers to all server in the cluster?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Every host has one vmk with vsan traffic and every host can ping all other hosts in the cluster via the vsan network.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Have you tried to reapply vsan configuration from RVC?
How many nodes do you see in the esxcli vsan cluster get?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Mrhaseins,
What type of switches do you use? IGMP is required for vSAN to work.
I saw the exact same from esxcli as you (everything reported ok), but error message in GUI until we got the switch config correct.
I wrote a bit about my experience on HP switches here: http://www.core-four.info/2014/11/vsan-and-hp-5400-switches.html
Lars
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I have Vsan working with no igmp snopping enabled at all.
Do you refeer to igmp as a protocol or igmp snooping?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
P er http://www.yellow-bricks.com/2014/03/31/vsan-misconfiguration-detected-2/ :
Here are your two options:
- Enable IGMP Snooping for your VSAN network (VLAN) and define an IGMP Snooping Querier. Default setting on most Cisco switches is IGMP Snooping enabled but without an IGMP Snooping Querier. In this configuration VSAN will not be able to configure correctly!
- Disable IGMP Snooping for your VSAN network (VLAN). Please note that you can typically disable IGMP Snooping globally and per VLAN, in this case if you want to disable it… disable it on your VLAN!
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
We use Juniper EX4500 for our storage network. And of course only IGMP snooping is disabled on the vsan vlan.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Mrhaseins,
Sound good. As vSAN communicates by using multicast you should then atleast make sure igmp is enabled. By using "show igmp group" on your vsan vlan you should two multicast groups.
Lars
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Lars,
I dont have IGMP enabled on my EX4550 and the VSAN cluster is working fine.
ezequiel@COBOGWBP4500CLOUDx1> show igmp group
ezequiel@COBOGWBP4500CLOUDx1>
of course all ports are in access and the entire virtual chassis is dedicated for VSAN
ezeq.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Updating vcenter to the latest build and the errors disappears. Problem solved! The "old" vcenter version was from September 2014, strange.
Thank you for your help!
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Mrhaseins,
Did you remember which version were you running and what you have now?
I'm experiencing the same problem.
C:\Windows\system32>"C:\Program Files\VMware\Infrastructure\VirtualCenter Server\vpxd.exe" -v
VMware VirtualCenter 5.5.0 build-2001466
Control Panel > Program and Features
VMware vCenter Server 5.5.0.42389
The installation was updated the last time with "VMware-VIMSetup-all-5.5.0-2105955-20140901-update02.iso".
Thanks!
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I upgraded our vcenter last time in September 2014 so the "old" version could be the "vCenter Server 5.5 Update 1c for Windows | 22 JUL 2014 | Build 1945274". Now we are on the latest version "vCenter Server 5.5 Update 2b | 09 OCT 2014 | Build 2183111". But I think the reconnect of the hosts after upgrading der vcenter solved the problem not just the upgrade.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thanks for your fast reply.
I think then in my case it's not related to the version.
The problem started to appear after changed the Management vmkernel IP address of the vSAN hosts members.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi All,
I have seen the same issue in my environment while I was configuring VSAN at new site. I have worked for two days to resolving this issue. Now i can advise you how to troubleshoot it.
You can follow the below point. Its up to you if you want to follow step by step or any individual which suits to your infrastructure.
1. Restart management services.
2. Restart ESXI host.
3. Login on ESXI and try to ping the IPs which you are using for handle VSAN traffic.
4. Recreation of virtual adapter, use for VSAN traffic.
5. Recreation of Vlans(Port Group)use for VSAN.
6. Recreation of DVswitch use for VSAN.
7. Enable\disable VSAN,
8. Recreation and deletion of disk group.
If you are following step 3 and you will be successful to ping it. Then approach to your network Team\person to make changes on physical switch. The ports used for Virtual SAN have multicast enabled. Enabling multicast can be done in one of two way on your physical switches:
• Disabling IGMP snooping.
• Configure IGMP snooping for selective traffic.
I am pasting a link which can help you more in resolving above issue or other related to VSAN. https://www.vmware.com/files/pdf/products/vsan/VMware_Virtual_SAN_Quick_Monitoring_Reference_Guide.p...
I was able to perform all step because my infrastructure was in implementation phase. Please consider you infrastructure and then follow the step carefully and at your own risk.
Thanks
Suresh Siwach
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Just a quick note: A lot of the checking (ping tests, deducing multicast issues) are now part of the VSAN Health check. All 6.0 deployments should install the plugin. It makes life much easier.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Recently, I have noticed the same issue and found I missed to select virtual san traffic in one of the host. The following link gives step by step guide to configure vSAN https://www.virtuallyboring.com/vmware-virtual-san-6-setup-and-configuration-part-2/