This article is going to showcase you how to approach this issue as I have seen during my days that the customers have tried restarting the services / server so many times that the logs have been rolled over and there is no way left to identify what caused the problem in the first place.
When we are talking about vCenter Server services being inaccessible, thumb-rule still remains same, which is, identify as much as possible about the problem, as this will simplify the issue and help you in identifying the correct solution.
So here are a set of questions that must be asked and made note off to begin isolating the problem.
- Is it a physical server or a virtual machine.
- Is it a Windows based vCenter server or is it an appliance.
- Version and build of your vCenter Server (This is a very key piece of information as a lot have changed over the last couple of versions starting with dividing the services into more refined micro-services, there is services interactions, then there is logging and much more)
- How is the vCenter Server being accessed vSphere client (C# client or fat client), vSphere Web Client (flash based client) or Web client(HTML5).
- What is the message reported, when you try logging in using the above connectivity tools.
- If vCenter server is a VM, do we know on which host is this VM running on.
- Is the vCenter server accessible over network.
- If it is a VM, log on to the ESXi host where we have the vCenter server service hosted, to find out if it is up and running or not.
Note : If you do not know, start checking all the ESXi hosts one by one.
Recommendation for future : Setup an affinity rule to restrict the vCenter Server’s movement in between few hosts in a host cluster. - If it is a physical server, log on to the remote server management tool like iLO(for HP server), iDRAC (for DELL server), etc, to check the system state.
- Next, you should check if the vCenter server IP address is reachable or not, from a VM which is on the same network as vCenter Server machine via ping.
- Also, check if it is accessible from another VM which is in same network(subnet / vlan / portgroup) on the same ESXi host.
- Do a forward and reverse lookup for the FQDN & IP address of the server.
- Next you need to check if port 80 & 443 are accessible from the local system.
- Log on to the vCenter server machine (If it is Microsoft Windows based then RDP into it else SSH into the machine).
Note : SSH service on the appliance needs to be up and running, if it is not up, then log on to the following URL : https://<vCenter_server_IP_or_fqdn>:5480 and navigate to Access tab (on the left hand side of the screen) and then click on Edit button on the right hand side corner, a pop up will appear for Edit Access Setting click on checkbox for Enable SSH login and click OK and you can continue to use putty or other ssh tool to log on the command line of the vCenter Server appliance. - Check for the vCenter Server service state.
- Look into the last log file to find out about the rough time-stamp of the issue(If you are well aware off vCenter Servers log files).
- Try starting the service for the vCenter server or other related services, in case if they are down.
Note: Do not try more than 2-3 times to start services as logs for some services may get rolled over after couple of restart attempts or it may get buried too deep in the log files. - If the services have not come up then it is a red flag and you should get in touch with VMware support team for assistance.
- Once the services are up and running, your job does not end here, you need to find what caused the services to go down in the first place and how to avoid it in future.
- Specific information about the issue and symptoms.
- How did you get to know about the issue.
- Log bundle for the vCenter server services.
- Windows Event viewer logs for system and Application view (In case if vCenter Server is Windows based).
With all the above it will help in problem identification and to come up with a workable solution.