7 Replies Latest reply on Dec 4, 2019 3:19 AM by Foxdemon

    Cant login to vsphere webgui and 1 of esxi hosts after reboot of vcenter server.

    Foxdemon Lurker

      First thing first, I have to admit we are a very inexperienced team, following after a lot of senior engineers left our company. We are rebuilding our staging servers, which havent been touched for about 250 days. So after clogging and rebuilding our veeam server, we wanted to start on the vcenter infrastructure.

       

      After rebooting our Vcenter server VM, we can access the Webgui of vsphere, but it doesnt show anything. The webgui of Vcenter gave the following error: "Could not connect to one or more vCenter Server Systems:vcenter1.local:443/sdk error in the vSphere Web Client. It shows nothing but a white inventory etc.

       

      Also the first ESXi host cant be accessed anymore from the webgui. We have 3 hosts of which 2 of them can be accessed, (also the one with the vcenter server). The host that cant be accessed does response to Ping request.

      Cause of this all the whole setup doesnt work anymore. Veeam cant access the VM's to backup etc. Does someone know what to do?

       

      Thanks in advance,

       

      Henri

        • 1. Re: Cant login to vsphere webgui and 1 of esxi hosts after reboot of vcenter server.
          Foxdemon Lurker

          Forgot to mention: when typing vcenter server ip address it goes to the empty Webgui. But when typing its hostname its gives the following error:

          503 Service Unavailable (Failed to connect to endpoint: [N7Vmacore4Http20NamedPipeServiceSpecE:0x00007f7f1007b7d0] _serverNamespace = / action = Allow _pipeName =/var/run/vmware/vpxd-webserver-pipe)

          • 2. Re: Cant login to vsphere webgui and 1 of esxi hosts after reboot of vcenter server.
            AhmedIbrahimVMware Enthusiast

            Hi

             

            Regarding the host that doesn’t respond to ping requests, you’ll need to open its console (using iLO, iDRAC, KVM, etc. ) to confirm that it is up and running, and is configured with the correct TCP/IP settings.

             

             

            Regarding the vCenter Server, the error message might indicate that one of the services is not running correctly, specifically the VPXD service.

             

             

            Hot check the service status, you’ll need to log into the vCenter Server machine using SSH if it is an appliance, or RDP if it is Windows then run the following command:

            service-control —status vpxd

             

            if it is stopped, then run the following command:

            service-control —start —all

             

            Then run the following command to confirm that the VPXD is running :

            service-control —status —all

             

             

            You can find more info  about this command in the following KB article:

            https://kb.vmware.com/s/article/2109887

            • 3. Re: Cant login to vsphere webgui and 1 of esxi hosts after reboot of vcenter server.
              Foxdemon Lurker

              Thanks Ahmed,

               

              This steps have resolve the issue with the vcenter server as the vmware-vpxd was indeed not running. Still left with one of the isxhosts not connected with the error:

               

              Cannot contact the specified host (172.16.91.51). The host may not be available on the network, a network configuration problem may exist, or the management services on this host may not be responding.

               

              Ping request to the host does work, but thats it. No ssh, and no webgui. When nmap the host it gives the following output:

              PORT     STATE  SERVICE

              22/tcp   closed ssh

              80/tcp   open   http

              427/tcp  open   svrloc

              443/tcp  open   https

              902/tcp  open   iss-realsecure

              2222/tcp closed EtherNetIP-1

              5988/tcp closed wbem-http

              5989/tcp open   wbem-https

              8000/tcp open   http-alt

              8080/tcp closed http-proxy

              8300/tcp open   tmi

              9080/tcp open   glrpc

               

              (edit) Something to add. I can ping the VM's that are running on the host.

              • 4. Re: Cant login to vsphere webgui and 1 of esxi hosts after reboot of vcenter server.
                AhmedIbrahimVMware Enthusiast

                Log in the ESXi shell, either physically or remotely using a console like iLO, iDRAC, etc., then run the following command:

                services.sh restart

                 

                Read the following KB for cautions that should be taken into consideration before you run this command:

                https://kb.vmware.com/s/article/1003490

                 

                Once the command finishes, try to type the IP address of the ESXi into a web browser, to see how it goes.

                 

                If you successfully connect to the ESXi web interface, then check the connectivity from vCenter Server client

                • 5. Re: Cant login to vsphere webgui and 1 of esxi hosts after reboot of vcenter server.
                  Foxdemon Lurker

                  Thanks again! Maybe this are some basic questions, but as i said we dont have much experience with this issues, so this is helping me a lot.

                   

                  The hosts are located off site, so i cant access them physically right away. This will be the last resort i guess. But i found out that there are iDRAC on them. (it are Dell PowerEdge R710's)

                  I have installed the srvadmin-all package on my linux laptop, start the service and try to connect to the host? with https://<ipadress>:1311 This doesn't seem to work. Is the iDRAC IP the same as the host IP?

                   

                  The only other IP-adresses that i can find in my network that are unknown to us (which are exactly 6, as there are 6 hosts) have no open ports what so ever. Can this be that the iDRAC's aren't configured or something

                   

                  Thanks in advance

                  • 6. Re: Cant login to vsphere webgui and 1 of esxi hosts after reboot of vcenter server.
                    AhmedIbrahimVMware Enthusiast

                    The iDRAC is a feature that allows you to connect to the server using an out-of-band management network.

                     

                     

                    It requires a separate IP address, which needs to be configured from the server’s BIOS, and of course, requires a physical network connectivity from the network switch to a specific port on the server.

                     

                     

                    If you don’t have a physical or remote access to the server, then I don’t think there’s much you can do right now.

                     

                    You might want to wait till you or somebody else have a way to access to this server to restart the management agents to see if this fixes the issue or not.

                    If the issue persists, then you might need to safely shut down the VMs using SSH if they’re Linux or RDP if they’re Windows, then reboot the server.

                    • 7. Re: Cant login to vsphere webgui and 1 of esxi hosts after reboot of vcenter server.
                      Foxdemon Lurker

                      Thanks again, yeah the problem is, this is our staging and testing environment and there is zero documentation on it whatsoever. The person who managed this has left the company and we are left with it without much knowledge how the infrastructure is build up etc. I will try to find out if there is someone who knows about the physical network connection of the iDRAC.

                       

                      If that doesn't yield anything i guess the only option is to shut down the VM's that are on it (I know i can reach them through SSH), and then go to the server location to see what we can do there.

                      With a last resort in shutting it down and power it on manually.

                       

                      Thanks for all your answers