VMware Cloud Community
Guybrush_Threep
Contributor
Contributor

Issue with vSAN 6.6.1 new lab cluster setup - network partitioned going into unicast mode and vSAN cluster configuration consistency warning

Hi all, wondering if anyone here have encountered the same issue I’m facing in my lab testing vSAN 6.6.1:

I was just setting up the vSAN from scratch, a brand new vSAN cluster setup with the latest 6.5U1 ESXi host install with the latest vCenter 6.5U1 install on a separate host – all with the latest updates and patches as of today.

I ended up getting the same “vSAN cluster configuration consistency” warning that also listed the issue being each of the 6 hosts showing “invalid request (Correct version of vSAN Health installed?)”

I noticed the following behavior:
– initial setup of the cluster with vSAN without disks claimed had them in multicast mode without any network partition problem (tested multicast traffic was working with tcpdump-uw)

– upon claiming disks, it tried to go into unicast mode and then every single host became partitioned. Unicastagent list came up empty. Cluster complains of being partitioned.

– manually adding the unicastagent entries resolved the partition problem, and that’s how I ended up with the “vSAN cluster configuration consistency”

Anyone have similar issues and can advise how to proceed? Thanks in advance!

Tags (1)
7 Replies
Guybrush_Threep
Contributor
Contributor

Some progress:

Funny enough, if I were to bootstrap the VCSA install using the installer into a single host vSAN directly, it comes up just fine into unicast mode without any of the above-mentioned symptoms.

Testing further to migrate the cluster into the earlier VCSA to see if it sticks.

Reply
0 Kudos
Guybrush_Threep
Contributor
Contributor

Further progress:

Moving the hosts to the earlier VCSA (in a non-vSAN host) resulted in the same errors and warnings, so that didn't help.

Moving the hosts back to the bootstrapped VCSA works fine.

Moved the bootstrapped VCSA into the same non-vSAN host (as the earlier VCSA) and it continued to work just fine.

Adding a second, separate cluster of 3 hosts in this VCSA and it works fine too.

Compared the services running on the earlier VCSA and the bootstrapped VCSA to no difference.

Gave up, deleted the earlier VCSA.

In a way, it's resolved, but I'm still confused as to what could be the difference that caused this problem to begin with.

mhampto
VMware Employee
VMware Employee

Hi,

It looks like there may be some minute difference in the network config of the new VCSA. Is the bootstrapped VC the same version as the hosts? All hosts and VC running the same major version? Newer hosts + older VC will not play nice.

Maybe some firewall options were enabled in the non-bootstrapped instance?

Glad you have got it working now. Thanks for posting your experience with the deployment.

Reply
0 Kudos
Guybrush_Threep
Contributor
Contributor

It’s the same VCSA version and all, as I had reinstalled from scratch since it’s a lab.

I didn’t expect any firewall differences either since the steps I did was all identical, as well as the virtual network setup, down to the same vlan and port config on the physical switch.

Reply
0 Kudos
sarikrizvi
Enthusiast
Enthusiast

Please check output of below cmd and match it , it should match if not then correct it please.

esxcli vsan network list

Also can you confirm build no. for ESXi and vCenter please which you have in your LAB.

Regards,
SARIK (Infrastructure Architect)
vExpert 2018-2020 | vExpert - Pro | NSX | Security
vCAP-DCD 6.5 | vCP-DCV 5.0 | 5.5 | 6.0 | vCA-DCV 5 | vCA-Cloud 5 | RHCSA & RHCE 6 | A+ (HW & NW)
__________________
Please Mark "Helpful" or "Correct" if It'll help you
_____________________________________
@Follow:
Blog# https://vmwarevtech.com
vExpert# https://vexpert.vmware.com/directory/1997
Badge# https://www.youracclaim.com/users/sarik
Reply
0 Kudos
TheBobkin
Champion
Champion

Hello sarikrizvi​,

Is there any particular reason you are necro'ing this thread that hasn't been active since 2017?

Also, the OP hasn't logged into Communities since May 2019 in case you didn't see.

Bob

Reply
0 Kudos
sarikrizvi
Enthusiast
Enthusiast

oh I didn't check date for post Bobkins​ and I just faced same issue so updated there earlier.

Regards,
SARIK (Infrastructure Architect)
vExpert 2018-2020 | vExpert - Pro | NSX | Security
vCAP-DCD 6.5 | vCP-DCV 5.0 | 5.5 | 6.0 | vCA-DCV 5 | vCA-Cloud 5 | RHCSA & RHCE 6 | A+ (HW & NW)
__________________
Please Mark "Helpful" or "Correct" if It'll help you
_____________________________________
@Follow:
Blog# https://vmwarevtech.com
vExpert# https://vexpert.vmware.com/directory/1997
Badge# https://www.youracclaim.com/users/sarik
Reply
0 Kudos