First thing first, I have to admit we are a very inexperienced team, following after a lot of senior engineers left our company. We are rebuilding our staging servers, which havent been touched for about 250 days. So after clogging and rebuilding our veeam server, we wanted to start on the vcenter infrastructure.
After rebooting our Vcenter server VM, we can access the Webgui of vsphere, but it doesnt show anything. The webgui of Vcenter gave the following error: "Could not connect to one or more vCenter Server Systems:vcenter1.local:443/sdk error in the vSphere Web Client. It shows nothing but a white inventory etc.
Also the first ESXi host cant be accessed anymore from the webgui. We have 3 hosts of which 2 of them can be accessed, (also the one with the vcenter server). The host that cant be accessed does response to Ping request.
Cause of this all the whole setup doesnt work anymore. Veeam cant access the VM's to backup etc. Does someone know what to do?
Thanks in advance,
Henri
The iDRAC is a feature that allows you to connect to the server using an out-of-band management network.
It requires a separate IP address, which needs to be configured from the server’s BIOS, and of course, requires a physical network connectivity from the network switch to a specific port on the server.
If you don’t have a physical or remote access to the server, then I don’t think there’s much you can do right now.
You might want to wait till you or somebody else have a way to access to this server to restart the management agents to see if this fixes the issue or not.
If the issue persists, then you might need to safely shut down the VMs using SSH if they’re Linux or RDP if they’re Windows, then reboot the server.
Forgot to mention: when typing vcenter server ip address it goes to the empty Webgui. But when typing its hostname its gives the following error:
Hi
Regarding the host that doesn’t respond to ping requests, you’ll need to open its console (using iLO, iDRAC, KVM, etc. ) to confirm that it is up and running, and is configured with the correct TCP/IP settings.
Regarding the vCenter Server, the error message might indicate that one of the services is not running correctly, specifically the VPXD service.
Hot check the service status, you’ll need to log into the vCenter Server machine using SSH if it is an appliance, or RDP if it is Windows then run the following command:
service-control —status vpxd
if it is stopped, then run the following command:
service-control —start —all
Then run the following command to confirm that the VPXD is running :
service-control —status —all
You can find more info about this command in the following KB article:
Thanks Ahmed,
This steps have resolve the issue with the vcenter server as the vmware-vpxd was indeed not running. Still left with one of the isxhosts not connected with the error:
Cannot contact the specified host (172.16.91.51). The host may not be available on the network, a network configuration problem may exist, or the management services on this host may not be responding.
Ping request to the host does work, but thats it. No ssh, and no webgui. When nmap the host it gives the following output:
PORT STATE SERVICE
22/tcp closed ssh
80/tcp open http
427/tcp open svrloc
443/tcp open https
902/tcp open iss-realsecure
2222/tcp closed EtherNetIP-1
5988/tcp closed wbem-http
5989/tcp open wbem-https
8000/tcp open http-alt
8080/tcp closed http-proxy
8300/tcp open tmi
9080/tcp open glrpc
(edit) Something to add. I can ping the VM's that are running on the host.
Log in the ESXi shell, either physically or remotely using a console like iLO, iDRAC, etc., then run the following command:
services.sh restart
Read the following KB for cautions that should be taken into consideration before you run this command:
https://kb.vmware.com/s/article/1003490
Once the command finishes, try to type the IP address of the ESXi into a web browser, to see how it goes.
If you successfully connect to the ESXi web interface, then check the connectivity from vCenter Server client
Thanks again! Maybe this are some basic questions, but as i said we dont have much experience with this issues, so this is helping me a lot.
The hosts are located off site, so i cant access them physically right away. This will be the last resort i guess. But i found out that there are iDRAC on them. (it are Dell PowerEdge R710's)
I have installed the srvadmin-all package on my linux laptop, start the service and try to connect to the host? with https://<ipadress>:1311 This doesn't seem to work. Is the iDRAC IP the same as the host IP?
The only other IP-adresses that i can find in my network that are unknown to us (which are exactly 6, as there are 6 hosts) have no open ports what so ever. Can this be that the iDRAC's aren't configured or something
Thanks in advance
The iDRAC is a feature that allows you to connect to the server using an out-of-band management network.
It requires a separate IP address, which needs to be configured from the server’s BIOS, and of course, requires a physical network connectivity from the network switch to a specific port on the server.
If you don’t have a physical or remote access to the server, then I don’t think there’s much you can do right now.
You might want to wait till you or somebody else have a way to access to this server to restart the management agents to see if this fixes the issue or not.
If the issue persists, then you might need to safely shut down the VMs using SSH if they’re Linux or RDP if they’re Windows, then reboot the server.
Thanks again, yeah the problem is, this is our staging and testing environment and there is zero documentation on it whatsoever. The person who managed this has left the company and we are left with it without much knowledge how the infrastructure is build up etc. I will try to find out if there is someone who knows about the physical network connection of the iDRAC.
If that doesn't yield anything i guess the only option is to shut down the VM's that are on it (I know i can reach them through SSH), and then go to the server location to see what we can do there.
With a last resort in shutting it down and power it on manually.
Thanks for all your answers
So wanted to let you know, and maybe someone who has the same problem later on.
I have been to the datacenter and rebooted the ESXi-host. This solved the problems, as it restarted all the services.
I also found the iDRAC Ip adresses of all the hosts in the cluster on the little screen in front of the racks. (And documented it..)
So in the future we will be able to resolve this kind of issues on a remote location.
Henri