VMware Cloud Community
Trobinox
Enthusiast
Enthusiast

Failed to activate VSAN on VMkernel

Hi all

I'm setting up vSAN in my own lab. I have 2 servers: 1 HP DL380 G7 and one G6. (I know that maybe not all components are on the HCL) but:

I was able to activate vSAN Traffic on vmk1 on the G7 server but if I enable it on the G6 server I only get a general system error during the refresh of the vsan configuration. I'm looking at vmkernel log and some vSAN logs, but I didn't get it out what's the problem.

ESXi Version is 6.5.

If someone has any ideas, let me know.

Thank you

Update 1:
I'm able to activate all other services like vMotion, management, FT, etc. only vSAN fails.

Update 2:

Error Message:

Task: Update Vsan

Target: ha-host

Initiator: VC Internal

Result:

Key: haTask--vim.host.VsanSystem.update-158810592

Description: Updates the Virtual SAN configuration for this host

State: Failed  - A general system error occurred:

20 Replies
GreatWhiteTec
VMware Employee
VMware Employee

Hard to say without a complete picture of the environment. Check below:

  • vCenter needs to be at 6.5
  • All hosts at 6.5
  • Disable HA before enabling vSAN
  • Create vSAN vmkernel ports prior to enabling vSAN
  • Enable vSAN service on vSAN vmkernel
  • NIC assigned to vSAN vswitch (ideally vDS) (license - vDS included with vSAN)
  • Minimum # of host for vSAN is 3
  • You will need a virtual Witness Appliance to do 1+1+1 cluster (2-node)
    • or set FTT=0 (for Lab only, but still wouldn't do it this way)

Alternatively you can use CLI from the host

esxcli vsan cluster get

If already part of it, then may be comms with vCenter. You can leave/join the cluster via CLI.

Reply
0 Kudos
Trobinox
Enthusiast
Enthusiast

  • vCenter needs to be at 6.5 -> OK
  • All hosts at 6.5 -> OK
  • Disable HA before enabling vSAN -> OK
  • Create vSAN vmkernel ports prior to enabling vSAN -> OK
  • Enable vSAN service on vSAN vmkernel -> my issue, does not work on one host
  • NIC assigned to vSAN vswitch (ideally vDS) (license - vDS included with vSAN) -> Actually it is a standard switch
  • Minimum # of host for vSAN is 3
  • You will need a virtual Witness Appliance to do 1+1+1 cluster (2-node) -> Witness is ready
    • or set FTT=0 (for Lab only, but still wouldn't do it this way)

Alternatively you can use CLI from the host

esxcli vsan cluster get

I can't enable vsan until the vmkernel is set correctly.

Reply
0 Kudos
Trobinox
Enthusiast
Enthusiast

Update: I migrated to a DSwitch - but same issue.

Reply
0 Kudos
admin
Immortal
Immortal

Greetings!

Could you please try to add vSAN tag on VMkernel adapter through CLI using below command:

To check the tags set on the given VMkernel network interface

# esxcli network ip interface tag get -i vmk0

Note: Please change vmk0 to the vmkx as per your requirement.

To add a tag on a given VMkernel network interface:

# esxcli network ip interface tag add -i vmk0 -t VSAN

Note: Please change vmk0 to the vmkx as per your requirement.


Supported tags are on latest ESXi version: Management, VMotion, faultToleranceLogging, vSphereReplication, vSphereReplicationNFC, vSphereProvisioning, VSAN, VSANWitness

Hope this helps you to configure VMkernel adapter for vSAN.

_________________________

Was your question answered correctly? If so, please remember to mark your question as "Correct" or "Helpful" when you get the correct answer. This helps others searching for a similar issue.


Cheers!

-Shivam

Reply
0 Kudos
Trobinox
Enthusiast
Enthusiast

Hi Shivam

Thank you for the input, but seems to be strange. CLI output:

[root@ESX01:/var/log] esxcli network ip interface tag get -i vmk1

  Tags: VMotion

[root@ESX01:/var/log] esxcli network ip interface tag add -i vmk1 -t VSAN

[root@ESX01:/var/log] esxcli network ip interface tag get -i vmk1

  Tags: VSAN, VMotion

But in the UI VSAN is still not enabled and I'm not able to configure VSAN.

Reply
0 Kudos
admin
Immortal
Immortal

I think there is some problem with the vSAN mgmt daemon which is not allowing this ESXi host to talk to vCenter and vCenter to talk to ESXi.

Could you please try restarting all the services on ESXi host:

# pwd

/

# service.sh restart

It will restart all the services on ESXi hosts (including vSAN services). Let me know how it goes.

Cheers!

-Shivam

Reply
0 Kudos
Trobinox
Enthusiast
Enthusiast

Problem still remains and vmk vsan options still is disabled in the UI and I can't enable VSAN

Reply
0 Kudos
Trobinox
Enthusiast
Enthusiast

I installed ESXi from scratch (and on another USB stick) but the problem still exists.

Reply
0 Kudos
admin
Immortal
Immortal

This is weird. If not already done, can you please remove the host from vCenter Server and then add it back again?

ESXi version and build?

vCenter version and build?

Reply
0 Kudos
Trobinox
Enthusiast
Enthusiast

I tried this way already

vCenter Version: 4602587

ESXi Version (tried with an without HPE Image): 4564106

Reply
0 Kudos
dhanarajramesh

Hi have you configured VMkernal as normal or with an IP stack?

Reply
0 Kudos
Trobinox
Enthusiast
Enthusiast

As normal in the webinterface and also a try with esxcli.

Reply
0 Kudos
huskies007
Contributor
Contributor

Hi all,

i'm facing the same issue, did anyone find the solution ?

Actually i use a nested configuration to test the VSAN upgrade path from VSAN 5.5 to 6.6.

Do you know if there  is an ESXi check to know if cpu ram and much other prerequisite are ok ?

I'll be very happy if someone can help me on this topics to find the troubleshooting path to retrieve vsan logs etc ......

Thanks a lot.

Regards

Franck

Reply
0 Kudos
huskies007
Contributor
Contributor

Hi all

My issue has been resolved.

I have increased the memory of my virtual ESXi host from 5 GB to 8 GB and rebooted the Host.

The Warning message “Host is in a VSAN enabled cluster but does not have VSAN service enabled” disappeared after ESXi host came up.

Regards

Franck

Ali_Akbar1
Contributor
Contributor

I have the same issue, even i increased memory from 4 GB to 6 GB still the message was popping up - I am using vmware workstation.

I have total memory of 16 Gig on my whitebox. I was able to increase memory for each host to 6 Gig the workstation didn't complain - VCSA 6.5 with 4 gig RAM i reduce RAM after installation since it's just a testing Lab and AD server with 1 Gig RAM.

Following resolved my issue assuming all the Network settings etc is correct .

1. Created a New cluster - Configure vSAN on it.

2. Added one host at a time - Added Esxi01 first, created a disk group on it. putty to esxi host and run the following command on first host to enable vSAN on host level.

esxcli vsan cluster get - this command will give information for vSAN cluster - the results it return was

"Virtual SAN Clustering is not enabled on this host"

So i RUN below command

esxcli vsan cluster new

ironically it resolved my problem. So i was missing this.

Later i add the other two hosts to vSAN cluster - and configure only disk group for the remaining two hosts one by one and it go added to my vSAN datastores.

Again when i run Get command it gives following info.

[root@Esxi01:~] esxcli vsan cluster get

Cluster Information

   Enabled: true

   Current Local Time: 2017-09-01T09:18:37Z

   Local Node UUID: 588e11f8-b864-1893-5019-000c29f551d8

   Local Node Type: NORMAL

   Local Node State: MASTER

   Local Node Health State: HEALTHY

   Sub-Cluster Master UUID: 588e11f8-b864-1893-5019-000c29f551d8

   Sub-Cluster Backup UUID: 588e1239-e54f-fc6b-0853-000c2933ba4d

   Sub-Cluster UUID: 52b486d4-e790-190c-4f4e-97e502a4425c

   Sub-Cluster Membership Entry Revision: 2

   Sub-Cluster Member Count: 3

   Sub-Cluster Member UUIDs: 588e11f8-b864-1893-5019-000c29f551d8, 588e1239-e54f-fc6b-0853-000c2933ba4d, 59a790bd-3959-57e5-8968-000c29c7cacf

   Sub-Cluster Membership UUID: 891ca959-c500-9c7b-e8dc-000c29f1ab15

Hopefully it help someone like me.

Thanks,

Ali

Reply
0 Kudos
CGat01
Contributor
Contributor

were you able to get the problematic host into maintenance mode before you rebooted? i'm not able to do that since the vsan service won't enable on my problematic host. there are no running vm's on it, but that host is part of the datastore as well. a 3 node datastore is what we have.

Reply
0 Kudos
CGat01
Contributor
Contributor

were you able to get the problematic host into maintenance mode before you rebooted? i'm not able to do that since the vsan service won't enable on my problematic host. there are no running vm's on it, but that host is part of the datastore as well. a 3 node datastore is what we have.

Reply
0 Kudos
ac4gc
VMware Employee
VMware Employee

Hello,

Didn't you get any kind of warning that the minimum requirement for ESXi memory is 6Gb or something like this?

Reply
0 Kudos
creker
Contributor
Contributor

Had the same problem on ESXi, 6.7.0, 11675023 and finally managed to solve it.

I noticed that every time I try to enable vSAN I get something like this in the logs from hostd process

Invoke method vim.host.VsanSystem.update failed: N7Vmacore3Ssl12SSLExceptionE(SSL Exception: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca)

I tried everything. I wanted check the certificate but couldn't find any sign of this request in the traffic. tcpdump didn't show anything on either lo0 or vmk. I checked hostd and other services configuration. No sign of anything useful. Finally I remembered that vcenter and all my hosts use custom certificate from Let's Encrypt. I reverted everything back to vmware self-signed certificate and the problem was fixed. Looks like the documentation for installing custom certificates doesn't cover everything and some services still expect self-signed certificate.

Reply
0 Kudos