VMware Cloud Community
apendo
Contributor
Contributor

vSphere 6.0 NTP Service and SSO malfunction

A couple of weeks ago I installed a fresh vSphere 5.5 U2 from HP on a completely new HP ProLiant DL380 Gen9 without any problems. Everything was working flawlessly, and I was just waiting for some disks before starting to use the server in our production environment.

Last week, when 6.0 was released, I decided to give the new version a try. As last time, I used HP’s customized version.

When configuring the server, I found two problems:

  1. Firstly, the NTP Service failed to start. I entered a couple of NTP servers and told the service to start and stop with the host, but the service always failed to start, no matter what values I used. The error message was “Call "HostServiceSystem.Start" for object "serviceSystem" on ESXi "server.domain.com" failed”. I started troubleshooting and and found that there were no server entries in the /etc/ntp.conf file. Therefore I manually created two of them, and after that the service started successfully. It’s a mystery where the names of the NTP servers I entered in the vSphere Client GUI was saved. Obviously not in the right place, but nonetheless they managed to stay persistent between several restarts of the host.
  2. Secondly, SSO didn’t function properly. I joined the server to our Windows domain to be able to manage permissions using our AD and to enable SSO for our users. At first everything worked perfectly, but after a host restart I had to enter my domain credentials manually in vSphere Client instead of using the checkbox Use Windows session credentials. The authentication between the vSphere Server and the AD is obviously working, but the automation of it isn’t. The error message using the checkbox is “A general system error occurred: gss_acquire_cred failed”.
    I’ve seen this exact problem several times before, and according to the Internet I’m not alone. We have three other vSphere servers using version 4.1, 5.0 and 5.5. The server using version 4.1 has the exact same problem, while the server using version 5.0 used to have it. Fortunately, I was able to follow some instructions given by another user and got rid of the problem on that particular server. The server using version 5.5 has never had the problems at all, nor did this new one until upgrading it to version 6.0.

I then performed a fresh install of 6.0 instead of upgrading 5.5 U2 to see if it made any difference, but unfortunately it didn’t. Next step was to see if there was something wrong with HP’s vSphere image, so I started over using VMware’s general image. This didn’t make any difference either. Both times I used a wiped USB key as installation target. Has anyone else experienced these problems with vSphere 6.0?

Tags (3)
17 Replies
Hammer68
Contributor
Contributor

I got the very same problem trying to configure NTP with ESXi 6 today.

The service was not starting with the same error reported.

Resolved manually configuring /etc/ntp.conf adding (in my case)

server 0.it.pool.ntp.org

server 1.it.pool.ntp.org

server 2.it.pool.ntp.org

The configuration seems working and persistent across reboots.

So again .. never ever try to install a new release 😞 

majerus
Contributor
Contributor

Install new releases, just dont deploy into production Smiley Wink

Reply
0 Kudos
Hammer68
Contributor
Contributor

of course, it was in a development environment, I told my colleague "expect some surprise" ... easy win  😉

I hope there are no other suprises ... !

rago60
Contributor
Contributor

I've got the same problem and manual edited the file /etc/ntp.conf

restrict 127.0.0.1
restrict default kod nomodify notrap
driftfile /etc/ntp.drift
server 192.168.0.10 version 3

After you add the last line with the ntp-server you have to restart the host. After restart take a closer look on the time configuration at the host.

Reply
0 Kudos
LHarris
Contributor
Contributor

Same problem here. I too had to manually edit /etc/ntp.conf before the service would start up.

Reply
0 Kudos
reub
Enthusiast
Enthusiast

I raised a support ticket against this in October last year during the 6.0 Beta - 14538274610 .  It was reproduced in house but obviously not deemed to be important enough to have fixed in the final GA...

jlanders
VMware Employee
VMware Employee

Odd, I see you and others reported this and 2 PRs were opened and marked fixed.

Can you confirm that you're using the C# client? I don't see this happening with the web client.

Reply
0 Kudos
Hammer68
Contributor
Contributor

Yes, I think it is a problem with the C# client only. Not a problem of ESXi hypervisor "per se".

Probably due to the choice of leaving the C# in the dust and developing the web client. Thus the C# client didn't have the minimal not regression test.

Reply
0 Kudos
apendo
Contributor
Contributor

I have only been using vSphere Client. Unfortunately, I’m not able to test these functions using the vSphere Web Client. Otherwise it would have been easier for me to check if the problems are caused by the server or the client.

Regarding the problem with automated SSO not working in vSphere Client, everything is functioning fine until I restart the server. This naturally implies that the problem is caused by the server. It doesn’t matter how many times I restart the vSphere Client or if I log on using another computer, it’s after a restart of the server the problems start.

Comparing my new 6.0 host with a fully functional 5.5 U2 host, I notice that the three AD related services “I/O Redirector”, “Local Security Authentication Server” and “Network Login Server” are replaced by a single service called “Active Directory Service” in the new version.

Reply
0 Kudos
rmoat
Contributor
Contributor

I'm getting this as well. I was able to fix it once, I thought it was through deleting the host_0 file on /scratch/var/tmp and making sure the NTP service was running, but that only fixed it the first time. I can log in with the AD account just as long as I don't use the "Use windows session credentials". This is very annoying.

Reply
0 Kudos
eos11
Contributor
Contributor

Hi, I also get Call 'HostServiceSystem.Start' for object "serviceSystem" when I try to start ntp using the vsphere client on a brand new install of vsphere 6.0.

I dont have access to the physical console (nothing connected to the VGA port).

How can I fix this issue remotely?

Without NTP, the time is already drifting out, which is not good.

Reply
0 Kudos
unsichtbare
Expert
Expert

This may be unrelated, but we saw similar symptoms in-house until we disabled IPv6 on the host. When left unconfigured, it was causing errors with various modules loading.

Incidentally, we always run the command: ntpq -p

To verify the functionality of NTP, including firewall configs for port 123, if you are configuring to a public NTP server

+The Invisible Admin+ If you find me useful, follow my blog: http://johnborhek.com/
Reply
0 Kudos
hostasaurus
Enthusiast
Enthusiast

Apparently fixing this isn't a priority.  Just installed a fresh build 6.0.0, 2809209 standalone and while the fat client will accept an NTP server, and apparently stores it somewhere because the value survives a reboot, the value never ends up in ntp.conf.  I had to add it via ssh.  I didn't try the web client since using that piece of garbage is slower than enabling ssh, logging in, editing and starting.

Reply
0 Kudos
unsichtbare
Expert
Expert

We have not seen this. I wonder if what you are experiencing is a symptom of a diskless install and storing data/values on "non-persistent storage?" Our standard build procedure is roughly as follows:

  1. Install ESXi with vendor-customized ISO
  2. Assign IPv4 IP, DNS, Hostname
  3. Disable IPv6 if/when not in use
  4. Enable SSH & Shell (where permitted)
  5. Update host using SSH*
  6. Configure storage (SAN, NFS)
  7. Redirect logs to SAN (VMware KB: Configuring syslog on ESXi 5.x and 6.0)
  8. Redirect scratch to SAN (VMware KB: Creating a persistent scratch location for ESXi 4.x/5.x/6.0)
  9. Configure NTP
  10. Configure AD (where appropriate)

This procedure seems to circumvent some of the issues you have experienced.

-J

*Online update for HP (lines 1-5 should work for Dell, IBM, etc - basically any server)

vim-cmd hostsvc/maintenance_mode_enter

esxcli network firewall ruleset set -e true -r httpClient

esxcli software sources profile list -d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml | grep ESXi-6

# Determine latest appropriate build of ESXi for your host and insert it in place of 'ESXi-X.X.X-XXXXXXXXXXX-standard' on the next line

esxcli software profile update -d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml -p ESXi-X.X.X-XXXXXXXXXXX-standard

esxcli software vib install -d http://vibsdepot.hp.com/hpq/latest/index-drv.xml

esxcli software vib install -d http://vibsdepot.hp.com/hpq/latest/index.xml

esxcli network firewall ruleset set -e false -r httpClient

reboot

+The Invisible Admin+ If you find me useful, follow my blog: http://johnborhek.com/
Reply
0 Kudos
Abdullah55
Contributor
Contributor

I am having the same issue and tried everything I know nothing worked. I could not get into the /etc/ntp.conf. It says Permission denied

Reply
0 Kudos
CaliColombia
Contributor
Contributor

I had the same error on ESXi, 6.0.0, 2494585

Using SSH to the VMWare server edit /etc/ntp.conf

add (according to your timezone):

server 0.pool.ntp.org

server 1.pool.ntp.org

server 2.pool.ntp.org

server 3.pool.ntp.org

In the server configuration -> Software -> Security Profile -> Services -> Properties -> Remote access

Pick NTP Daemon (for me was Stopped)

Below under Service properties click on Options:

Setup as in Time configuration

The NTP client start working.

Reply
0 Kudos
zloetelo
Contributor
Contributor

this work for 6.0.0 3073146 to:smileycheck:

Reply
0 Kudos