0 Replies Latest reply on Oct 19, 2018 6:06 AM by hkmix

    Vcenter Down Without any change setting

    hkmix Lurker

      Dear All,

       

      Need some help, our vcenter just down today....already tried couple of hours, but stll not fix....

       

      error as

       

      ==

      503 Service Unavailable (Failed to connect to endpoint: [N7Vmacore4Http20NamedPipeServiceSpecE:0x00007f30cc0cf330] _serverNamespace = / action = Allow _pipeName =/var/run/vmware/vpxd-webserver-pipe)

      ==

       

      Tried to start and stop all service....only below showing error

       

      ==

      root@localhost [ /var/log/vmware/vapi/endpoint ]# service-control --start vmware-vpxd

      Perform start operation. vmon_profile=None, svc_names=['vmware-vpxd'], include_coreossvcs=False, include_leafossvcs=False

      2018-10-19T12:57:46.438Z   Service vpxd state STOPPED

      Error executing start on service vpxd. Details {

          "resolution": null,

          "detail": [

              {

                  "args": [

                      "vpxd"

                  ],

                  "id": "install.ciscommon.service.failstart",

                  "localized": "An error occurred while starting service 'vpxd'",

                  "translatable": "An error occurred while starting service '%(0)s'"

              }

          ],

          "componentKey": null,

          "problemId": null

      }

      Service-control failed. Error {

          "resolution": null,

          "detail": [

              {

                  "args": [

                      "vpxd"

                  ],

                  "id": "install.ciscommon.service.failstart",

                  "localized": "An error occurred while starting service 'vpxd'",

                  "translatable": "An error occurred while starting service '%(0)s'"

              }

          ],

          "componentKey": null,

          "problemId": null

      }

      ====

       

      checked  /var/log/vmware/vpxd/vpxd.log

       

       

      ==

       

      2018-10-19T12:57:52.193Z info vpxd[7FEF8DA80800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.storage.vmfs* - list* requires System.Read

      2018-10-19T12:57:52.193Z info vpxd[7FEF8DA80800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.storage.vmfs* - get* requires System.Read

      2018-10-19T12:57:52.193Z info vpxd[7FEF8DA80800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.storage.vmfs* - * requires Host.Config.Storage

      2018-10-19T12:57:52.193Z info vpxd[7FEF8DA80800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.storage.nfs* - list* requires System.Read

      2018-10-19T12:57:52.193Z info vpxd[7FEF8DA80800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.storage.nfs* - get* requires System.Read

      2018-10-19T12:57:52.193Z info vpxd[7FEF8DA80800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.storage.nfs* - * requires Host.Config.Storage

      2018-10-19T12:57:52.193Z info vpxd[7FEF8DA80800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.vm.process.list* - * requires System.View

      2018-10-19T12:57:52.193Z info vpxd[7FEF8DA80800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.vm.process.kill* - * requires VirtualMachine.Interact.PowerOff

      2018-10-19T12:57:52.193Z info vpxd[7FEF8DA80800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.license* - * requires Global.Licenses

      2018-10-19T12:57:52.193Z info vpxd[7FEF8DA80800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.system.account* - * requires Global.Settings

      2018-10-19T12:57:52.193Z info vpxd[7FEF8DA80800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.esxcli* - * requires System.View

      2018-10-19T12:57:52.193Z info vpxd[7FEF8DA80800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.system.permission* - * requires Global.Settings

      2018-10-19T12:57:52.193Z info vpxd[7FEF8DA80800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.system* - list* requires System.Read

      2018-10-19T12:57:52.193Z info vpxd[7FEF8DA80800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.system* - get* requires System.Read

      2018-10-19T12:57:52.193Z info vpxd[7FEF8DA80800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.system* - * requires Host.Config.Settings

      2018-10-19T12:57:52.193Z info vpxd[7FEF8DA80800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.system.settings* - list* requires System.Read

      2018-10-19T12:57:52.193Z info vpxd[7FEF8DA80800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.system.settings* - get* requires System.Read

      2018-10-19T12:57:52.193Z info vpxd[7FEF8DA80800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.system.settings* - * requires Host.Config.AdvancedConfig

      2018-10-19T12:57:52.194Z info vpxd[7FEF8DA80800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.hardware* - * requires Host.Config.Settings

      2018-10-19T12:57:52.194Z info vpxd[7FEF8DA80800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.network* - list* requires System.Read

      2018-10-19T12:57:52.194Z info vpxd[7FEF8DA80800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.network* - get* requires System.Read

      2018-10-19T12:57:52.194Z info vpxd[7FEF8DA80800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.network* - * requires Host.Config.Network

      2018-10-19T12:57:52.194Z info vpxd[7FEF8DA80800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.system.coredump* - list* requires System.Read

      2018-10-19T12:57:52.194Z info vpxd[7FEF8DA80800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.system.coredump* - get* requires System.Read

      2018-10-19T12:57:52.194Z info vpxd[7FEF8DA80800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.system.coredump* - * requires Host.Config.Storage

      2018-10-19T12:57:52.194Z info vpxd[7FEF8DA80800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.storage.core* - list* requires System.Read

      2018-10-19T12:57:52.194Z info vpxd[7FEF8DA80800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.storage.core* - get* requires System.Read

      2018-10-19T12:57:52.194Z info vpxd[7FEF8DA80800] [Originator@6876 sub=dynamicMethodValidator] [Init] Method vim.EsxCLI.storage.core* - * requires Host.Config.Storage

      2018-10-19T12:57:52.198Z info vpxd[7FEF7E1BB700] [Originator@6876 sub=vpxLro opID=lro-1-fc4111] [VpxLRO] -- BEGIN lro-1 --  -- VpxdDrmMainLoop --

      2018-10-19T12:57:52.199Z info vpxd[7FEF7DD32700] [Originator@6876 sub=vpxLro opID=lro-2-3113c398] [VpxLRO] -- BEGIN lro-2 --  -- VpxdDrmMainLoop --

      2018-10-19T12:57:52.199Z error vpxd[7FEF8DA80800] [Originator@6876 sub=HostUpgrader] [VpxdHostUpgrader] Could not find vpxa bundle for embeddedEsx version e.x.p: vpx-upgrade-eesx-1-linux-*

      2018-10-19T12:57:52.200Z error vpxd[7FEF8DA80800] [Originator@6876 sub=HostUpgrader] [VpxdHostUpgrader] Could not find vpxa bundle for esx version e.x.p: vpx-upgrade-esx-7-linux-*

      2018-10-19T12:57:52.285Z info vpxd[7FEF8DA80800] [Originator@6876 sub=Default] Wrote vpxd process map to /var/log/vmware/vpxd/vpxd-map-20811.txt.

      2018-10-19T12:57:52.285Z info vpxd[7FEF8DA80800] [Originator@6876 sub=Vpxd.MemChecker] vpxd memChecker disabled

      2018-10-19T12:57:52.286Z warning vpxd[7FEF8DA80800] [Originator@6876 sub=VpxProfiler] ServerApp::Init [TotalTime] took 5059 ms

      2018-10-19T12:57:52.286Z info vpxd[7FEF8DA80800] [Originator@6876 sub=MoOptionMgr] [OptionManagerMo] Invoking callbacks for key VirtualCenter.MaxDBConnection, pre commit

      2018-10-19T12:57:52.287Z info vpxd[7FEF8DA80800] [Originator@6876 sub=MoOptionMgr] [OptionManagerMo] Invoking callbacks for key VirtualCenter.MaxDBConnection, pre commit

      2018-10-19T12:57:52.287Z info vpxd[7FEF8DA80800] [Originator@6876 sub=MoOptionMgr] [OptionManagerMo] No change to VirtualCenter.MaxDBConnection

      2018-10-19T12:57:52.318Z error vpxd[7FEF8DA80800] [Originator@6876 sub=Default] [VdbStatement] Execute result code: -1

      2018-10-19T12:57:52.318Z error vpxd[7FEF8DA80800] [Originator@6876 sub=Default] [VdbStatement] SQL execution failed: SELECT TASK_ID, NAME, DESCRIPTIONID, ENTITY_ID, ENTITY_TYPE, ENTITY_NAME, LOCKED_DATA, COMPLETE_STATE, CANCELLED, CANCELLABLE, ERROR_DATA, RESULT_DATA, REASON_DATA, QUEUE_TIME, START_TIME, COMPLETE_TIME, EVENT_CHAIN_ID, USERNAME, VM_ID, HOST_ID, COMPUTERESOURCE_ID, DATACENTER_ID, RESOURCEPOOL_ID, FOLDER_ID, ALARM_ID, SCHEDULEDTASK_ID, CHANGE_TAG_ID, PARENT_TASK_ID, ROOT_TASK_ID, DESCRIPTION, ACTIVATION_ID FROM VPX_TASK  where ENTITY_TYPE in (22,23,24,25,26,27) and COMPLETE_STATE='running' LIMIT 1000;

      2018-10-19T12:57:52.318Z error vpxd[7FEF8DA80800] [Originator@6876 sub=Default] [VdbStatement] Execution elapsed time: 13 ms

      2018-10-19T12:57:52.318Z error vpxd[7FEF8DA80800] [Originator@6876 sub=Default] [VdbStatement] Statement diagnostic data from driver is XX001:0:7:ERROR: invalid page in block 182250 of relation pg_tblspc/16392/PG_9.4_201409291/16385/19856;

      --> Error while executing the query

      2018-10-19T12:57:52.318Z error vpxd[7FEF8DA80800] [Originator@6876 sub=Default] [VdbStatement] Connection diagnostic data from driver is XX001:0:110:ERROR: invalid page in block 182250 of relation pg_tblspc/16392/PG_9.4_201409291/16385/19856

      2018-10-19T12:57:52.318Z error vpxd[7FEF8DA80800] [Originator@6876 sub=Default] [VdbStatement] Bind parameters:

      2018-10-19T12:57:52.318Z error vpxd[7FEF8DA80800] [Originator@6876 sub=Default] [Vdb::IsRecoverableErrorCode] Unable to recover from XX001:7

      2018-10-19T12:57:52.318Z error vpxd[7FEF8DA80800] [Originator@6876 sub=Default] [Vdb::IsRecoverableErrorCode] Unable to recover from XX001:110

      2018-10-19T12:57:52.318Z error vpxd[7FEF8DA80800] [Originator@6876 sub=Default] [VdbStatement] SQLError was thrown: "ODBC error: (XX001) - ERROR: invalid page in block 182250 of relation pg_tblspc/16392/PG_9.4_201409291/16385/19856;

      --> Error while executing the query" is returned when executing SQL statement "SELECT TASK_ID, NAME, DESCRIPTIONID, ENTITY_ID, ENTITY_TYPE, ENTITY_NAME, LOCKED_DATA, COMPLETE_STATE, CANCELLED, CANCELLABLE, ERROR_DATA, RESULT_DATA, REASON_DATA, QUEUE_TIME, START_TIME, COMPLETE_TIME, EVENT_CHAIN_ID, USERNAME, VM_ID, HOST_ID, COMPUTERESOURCE_ID, DATACENTER_ID, RESOURCEPOOL_ID, FOLDER_ID, ALARM_ID, SCHEDULEDTASK_ID, CHANGE_TAG_ID, PARENT_TASK_ID, ROOT_TASK_ID, DESCRIPTION, ACTIVATION_ID FROM VPX_TASK  where ENTITY_TYPE in (22,23,24,25,26,27) and COMPLETE_STATE='running' LIMIT 1000;"

      2018-10-19T12:57:52.318Z error vpxd[7FEF8DA80800] [Originator@6876 sub=MoTask] [TaskManagerMoImpl] Failed to load tasks: "ODBC error: (XX001) - ERROR: invalid page in block 182250 of relation pg_tblspc/16392/PG_9.4_201409291/16385/19856;

      --> Error while executing the query" is returned when executing SQL statement "SELECT TASK_ID, NAME, DESCRIPTIONID, ENTITY_ID, ENTITY_TYPE, ENTITY_NAME, LOCKED_DATA, COMPLETE_STATE, CANCELLED, CANCELLABLE, ERROR_DATA, RESULT_DATA, REASON_DATA, QUEUE_TIME, START_TIME, COMPLETE_TIME, EVENT_CHAIN_ID, USERNAME, VM_ID, HOST_ID, COMPUTERESOURCE_ID, DATACENTER_ID, RESOURCEPOOL_ID, FOLDER_ID, ALARM_ID, SCHEDULEDTASK_ID, CHANGE_TAG_ID, PARENT_TASK_ID, ROOT_TASK_ID, DESCRIPTION, ACTIVATION_ID FROM VPX_TASK  where ENTITY_TYPE in (22,23,24,25,26,27) and COMPLETE_STATE='running' LIMIT 1000;"

      2018-10-19T12:57:52.318Z error vpxd[7FEF8DA80800] [Originator@6876 sub=Daemon] Unhandled exception: N3Vim5Fault13DatabaseError9ExceptionE(vim.fault.DatabaseError)

      --> [context]zKq7AVECAAAAAEklWwANdnB4ZAAAeF4rbGlidm1hY29yZS5zbwAAEBcbAMppGAEu0VR2cHhkAAEA0lUBUHKTASJ6kwEBfZMBiu9UAVj5VAGKnFMC4AUCbGliYy5zby42AAGVlFM=[/context]

      2018-10-19T12:57:52.321Z info vpxd[7FEF8DA80800] [Originator@6876 sub=SupportMgr] Wrote uptime information

       

       

       

      ========================

       

      Is that mean the