vCenter & Hosts in Domain (Hosts not responding)

[vPatrickS]

Hello everybody

I'm working on a new problem in my homelab.

Today I installed a new server 2008 VM and configured it as DC. After that I put my two ESXi 4.1 hosts and my vCenter Server into the new domain.

So far no problems.

Some minutes later the hosts changed their "state" and now vCenter tells me that the hosts are "not responding" and all VMs are "disconnected".

KB1011878 gave me the tip with the time configuration.

At this time the ESXi hosts were still configured as NTP-Client & the vCenter server to sync. time with the host.

In order to the KB article I reconnected my hosts and disabled the NTP client configuration and also the time sync. between vCenter server and host.

So all of them should receive the right/same time from the domain controller!?

But the hosts still switch to "not responding" after ~30 seconds.

At the moment I'm not sure if I configured something wrong or if I missed some importend configurations.

Maybe you guys have some recommendations for me.

In some blogs I read something about re-installing vCenter server and such tips, but I think there are some other ways to fix that.

Thanks in advance

Patrick

[vPatrickS]

Hey Guys

I'm still working on this problem.

I found another article (KB1011647) which did'nt worked for me.

So I left the domain with the vCenter server and both hosts what solved to problem.

If I rejoin the domain with the vCenter server the problem occour again.

I also rejoined the domain with one of the hosts and this seems to be ok, so the problem just occur if the vCenter server is a member of a/my domain.

Could there be some other reasons as just the time configuration?

Regards

Patrick

[a_p_]

Did you see any errors/messages in the vpxa.log?

When adding the vCenter Server to the domain, the FQDN of the vCenter Server may be modified. Did you verify the DNS and name resolution work correctly?

You may also have group policies and/or security policies which apply when you add the server to the domain.

André

[vPatrickS]

Hi

To ensure that the time settings are not the problem I did the following:

- Configured my PDC as NTP client to obtain the time from an external NTP server

- Configured my hosts again as NTP client to obtain the time from the same external NTP server

- Rejoind my vCenter server to the domain.

So all systems should use time. But this did'nt solved the problem.

Then I tried your recommendations and checked the dns.

- From my vCenter server I can ping all hosts with <name> and  FQDN <name>.<domain>

- From the ESXi hosts I can also ping/resolv the name & FQDN of the vCenter server

So DNS should work properly.

Then I looked into the vpxa.log.

The last entry was at 13:45, but it should be at 14:45.

The vCenter server shows me still the right time for this host.

Then I used "date" to have a look at the actual time on the ESXi and it seems that the host is one hour behind.

So the time on the esxi differs from the time what vCenter shows me.

Could this be the reason?

The vpxa.log overwhelmes me a little bit, to much information. :smileyblush:

[vPatrickS]

I just found this thread:

Duncan wrote:
it's not possible to set the timezone in ESXi. 
ESXi runs in UTC and the time you see is converted by the VIC, 
which uses the Windows info to 
convert the time to their own region.

So it seems that's standard esxi behavior.

So there must be a other reason for this problem.

And how I said, everytime I leave the domain with the vCenter server the problem is solved, but this can't be the final solution :smileyconfused:

[ankitnarchal]

Yes, You cannot configure time synch in ESi servers as there is no service console and does not have capability of syching the time with the NTP servers. The UTC time it shows is picked by the host from its BIOS. Now if yo have VMware tools time synch enabled for the DC VM and you have also made DC to synch the time from NTP server, then this could be the problem.

The VMware tools will pick the time from ES servers and will give it to the VMs so i would suggest you to disable the vmware tools Time SYNC and let DC VM to SYNC the time from external NTP server.

Monitor it for a day and check what errors do you get in VPxD logs and VPxa logs. Make sure that the VC version is either same or higher then the version of ESx because they will install the agent on the host which cannot be of older version.

If you still get errors then you upload the VPxd and vpxa logs on this thread. i will let you know the reason which is causing that issue.

[sudhakar13]

hi,

It's not recommended you to install vCenter on the Domain Controller. So make sure that you are installed vCenter in the seperat VM. One for DC and other for vCenter Server. This may help you.

Regards

S.Sudhakar

[ankitnarchal]

Thanks for the comment Sudhakar. But FYI, its not about recommendation but it is not a supported configuration by VMware to install VC on DC as you have IIS installed on DC servers and same is not supported by VC. you cannot have VC instaklled on the same server where IIS is installed. This would lead to some port conflicts as well.

[vPatrickS]

Thank you

How I wrote at the moment I have no VM <-> host time sync enabled.

DC >> external NTP

HOSTS >> same external NTP

vCenter >> As domain member from DC

And I use vCenter server 4.1 aswell ESXi 4.1

//EDIT:

I have two different VM's, one for vCenter and one for my domain.

On the vCenter VM there is just vCenter server + standalone converter + update manager installed.

On my domain VM just PDC role + DNS.

Also I attached the two log files.

Thanks in advance

Patrick

[ankitnarchal]

Thanks for the logs. Giveme some time to review them so that i could find the cause of it.

Meanwhile you can disable the Host time synch from the client and monitor it for a while.

[ankitnarchal]

From the logs i can say that it does not seem to be an issue with the Time Synch. It could be the cluster configuration that may be causing this issue. There is a heartbeat going between the hosts and the VC which might be breaking. Make sure that the VC version is either equal or higher version than the ESx because when you add the host to the VC it installs the agent on to the esx server.

This agent mismatch can cause such issues. How many host in the cluster are affected by this? will it be possible for you to disconnect the host from the cluster and remove it. Once done you can readd the host using the FQDN and monitor it. This will uinstall and reinstall the vpxa agent on the esx server.

Do you have HA enabled on the cluster? Make sure that the HOSTS file on the esx server has the entries for all the hosts in the cluster and of the VC as well.

[vPatrickS]

Good morning everybody

I have good and bad news :smileylaugh:

The bad news first:

I rechecked everything and I also followed your latest reccomendations but what should I say? It doesn't matter, one minute after I had all my hosts out of the cluster, both switched to "not responding" :smileyangry:

Now the good news:

Yesterday I read on a blog about exacly the same problem and they also were'nt able to find a proper solution.

They decided to reinstall the vCenter Server and I did the same this morning. And? It works.

I readded my two hosts and they are still connected to my vCenter. :smileygrin:

Lessons learned: Join your potential vCenter Server to a domain. After that if your server is a domain member, install the vCenter server.

I have no idea what happens to the vCenter services and host agents in the background if you join a domain, but this causes the problems.

Nevertheless thank you very much for your help!!

Regards

Patrick

[ankitnarchal]

well that is something very strange as you can make the VC part of domain later as well. But one of the reasons could be that when you make the VC part of domain then ODBC connection you create using DSN will depend on the name of the server's FQDN and if you make any change to that it would cause issues even connecting to the VC.

But in your case you were able to connect to the VC and Hosts gets disconnected. I guess that is the issue with the agents that were installed on the ESx servers when they were added to the VC cluster. Well reinstalling the VC did reinstalled the agents as well but you could have tried avoiding it by just reinstalling the vpxa and aam agent on the Host.