Skip navigation
1 2 Previous Next

Rajeev's Blog

26 posts

I got issue when I checked  windows authentication  box, same user was able to logon by web client but from vc-client getting issue.

 

When I typed manually user name and password I was able to logon. I went through logs and got use full KB which is applicable 6.0 also.

 

 

2017-11-24T12:55:37.853Z error vpxd[7F3237CF8700] [Originator@6876 sub=GSSAPI opID=F65D9D27-00000004-8a] gss_accept_sec_context failed: (0x00070000, 0x00000000)

2017-11-24T12:55:37.853Z error vpxd[7F3237CF8700] [Originator@6876 sub=GSSAPI opID=F65D9D27-00000004-8a] Supported mechanisms: ({ 1 2 840 113554 1 2 2 }, { 1 3 5 1 5 2 }, { 1 2 840 48018 1 2 2 }, { 1 3 6 1 5 2 5 }, { 1 3 6 1 5 5 2 }, { 1 3 6 1 4 1 311 2 2 10 }, { 1 2 840 113554 1 2 10 })

2017-11-24T12:55:37.861Z info vpxd[7F3237CF8700] [Originator@6876 sub=vpxLro opID=F65D9D27-00000004-8a] [VpxLRO] -- FINISH task-internal-1992

2017-11-24T12:55:37.861Z info vpxd[7F3237CF8700] [Originator@6876 sub=Default opID=F65D9D27-00000004-8a] [VpxLRO] -- ERROR task-internal-1992 -- SessionManager -- vim.SessionManager.loginBySSPI: vim.fault.InvalidLogin:

--> Result:

--> (vim.fault.InvalidLogin) {

-->    faultCause = (vmodl.MethodFault) null,

-->    msg = ""

--> }

--> Args:

-->

--> Arg base64Token:

 

 

Logging in to VMware vCenter Server Appliance 5.1 or 5.5 using the Use Windows session credentials option fails with the…

When we run command

 

Service-control --stop --all

Service-control --start -all

 

We got error cm or proxy service not started that why vpxd have issue.

 

Try this

 

Restart, proxy service then restart vpxd. It because of component manager service not starting.

KB: - https://kb.vmware.com/s/article/2147891

Some time web client start but , facing the issue for login and got the error 503.  Mostly issue in 6.0 U2 which is resolved in U3

 

  1. 1. Temporary  solution :- Increase MaxPermMB size 

 

https://kb.vmware.com/s/article/2148445

  1. 2. Permanent: - Upgrade your infrastructure according to 6.0 U3.

 

          https://kb.vmware.com/s/article/2148355

This issue mostly comes when we upgrade PSC ( appliances) , not able to open PSC in GUI mode.

 

This is certificate issue . there are 2 solution

1. Take snap shot and regenerate certificate then restart web-client service

2. Decommission affected PSC ( first check which PSC is pointing if you have 2 PSC) do not touch pointing PSC

3. Once Decomm delete it from disk and remove it from AD/DNS

4. Recreate DNS record

5. Deploy new PSC and join in Domain

6. Try IP/psc you should able to open it in GUI.

 

 

Using the cmsso command to unregister vCenter Server from Single Sign-On (2106736) | VMware KB

RajeevVCP4 Hot Shot
vExpert

Host not responding

Posted by RajeevVCP4 Sep 8, 2017

When host is not responding and not able to press F2 key but it is ping able

 

Mostly issue from storage side scsci reservation locked host's lock file and not able to release it

 

It depend on storage environment in vnx when we check we got

 

09/08/17 10:22:06 FCDMTL 9 (FE1/SC)  ebd1b60 Target command error: mp context 0x300ff, SCSI status = 0x18 - returned by UL

                       A 09/08/17 10:22:06 TDD f32c7730 GRO:Register/Reserve Reservation RsvKey Mismatch Error.

                       A 09/08/17 10:22:06 FCDMTL 9 (FE1/SC)  ef6f680 Target command error: mp context 0x100f6, SCSI status = 0x18 - returned by UL

                       A 09/08/17 10:22:06 TDD f32c7730 GRO:Register/Reserve Reservation RsvKey Mismatch Error.

 

Here scsi reservation because of HLU and ALU was mismatch

Need to reconfigure from storage side and reboot all host in the cluster

 

After upgrade PSC u1 to u2 PSC disjoin from domain , and user got this error.

 

com.vmware.identity.idm.server.provider.vmwdirectory.VMwareDirectoryProvider]

[2017-08-18T13:23:14.779-05:00 bba76607-42b4-4a15-a3c2-7542f427d12c WARN ] [ActiveDirectoryProvider] There may be a domain join status change since native AD is configured. ActiveDirectoryProvider can function properly only when machine is properly joined

 

Tried to join domain  but got (Idm client exception: Error trying to join AD, error code [31])

 

It was because of smb was disabled on PSC (appliances)

 

run this command

 

/opt/likewise/bin/lwregshell list_values '[HKEY_THIS_MACHINE\Services\lwio\Parameters\Drivers\rdr]'

 

if value is zero of smb  then run this command

 

  1. /opt/likewise/bin/lwregshell set_value '[HKEY_THIS_MACHINE\Services\lwio\Parameters\Drivers\rdr]' Smb2Enabled 1
  2. /opt/likewise/bin/lwsm restart lwio
  3. service-controll --stop --all
  4. service-control --start -all
  5. restart web client on vcenter server
  6. then logon psc and try to add PSC in domain
  7. If you have secondary PSC , it will take some time for showing domain user.

Scenario , Windows based vcenter server with external PSC (appliances)

 

When click any component by using administrator@vsphere.local

 

Error:- Do not have permission to look this object

 

Work around:-

1. Shut down the vCenter Server services
2. Shut down the PSCs services
3. Started up PSC services then verified they were all running
4. Started up vCenter services then verified those services

When user is trying to browse storage (data store/device) through web-client getting error

 

I worked with VMware , support opened internal PR with engineering team and provided solution as

Per development they've root caused the issue. The fix will be included in vCenter 6.0 patch 6. The release of this fix is early Q4 this year.

 

 

The fix will also be in the next releases of 5.5 and 6.5 as well.

This issue occurs if the hostd is not aware of the dynamic rule when auto-deploy attempts to check host compliance after applying the host profile. As a result, the compliance check fails if the host profile contains the dynamic rule set

 

Actulally this issue occurred 5.5 but I saw it is also exist in 6.0 and solution is same.

 

 

To workaround this issue:

 

  1. Disable and enable SNMP on the host to restore the dynamicruleset firewall ruleset.

    • Connect to the affected host using SSH and root credentials. For more information, see Using ESXi Shell in ESXi 5.x and 6.0 (2004746).

    • Run this command to disable SNMP:

      esxcli system snmp set -e 0
    • Run this command to enable SNMP:

      esxcli system snmp set -e 1
    • Apply the Host Profile and check compliance.

  2. Manually check and apply the host profile again to refresh the firewall. This will clear the compliance error.

  3. Reset the firewall on ESXi host using these commands:
  • esxcli network firewall set --enabled false
  • esxcli network firewall set --enabled true

Again detach reference host and attach then check compliance

 

 

See Also

I got these logs

 

2017-05-17T18:21:13.052Z warning vpxd[7FC535D487A0] [Originator@6876 sub=VpxProfiler] Init [Vpx::Common::Sso::SsoFactory_CreateFacade(sslContext, ssoFacadeConstPtr)] took 117827 ms

2017-05-17T18:21:13.052Z error vpxd[7FC535D487A0] [Originator@6876 sub=Main] [Init] Init failed: Vpx::Common::Sso::SsoFactory_CreateFacade(sslContext, ssoFacadeConstPtr)

--> Backtrace:

-->

--> [backtrace begin] product: VMware VirtualCenter, version: 6.0.0, build: build-4541948, tag: vpxd

--> backtrace[00] libvmacore.so[0x003C6994]: Vmacore::System::Stacktrace::CaptureWork(unsigned int)

--> backtrace[01] libvmacore.so[0x001F0883]: Vmacore::System::SystemFactoryImpl::CreateQuickBacktrace(Vmacore::Ref<Vmacore::System::Backtrace>&)

--> backtrace[02] libvmacore.so[0x0019A7DD]: Vmacore::Throwable::Throwable(std::string const&)

--> backtrace[03] vpxd[0x00BCC43E]: Vmomi::Fault::SystemError::Exception::Exception(std::string const&)

--> backtrace[04] vpxd[0x00BC9EBA]

--> backtrace[05] vpxd[0x00BB6180]

--> backtrace[06] vpxd[0x00AF44E9]

--> backtrace[07] libc.so.6[0x0001EC36]

--> backtrace[08] vpxd[0x00AF3F3D]

--> [backtrace end]

-->

 

When I checked VCenter server was not communicating with external psc. Some removed nic from PSC , added back nic and issue resolved.

In my case we have 1 VC (windows based) with 2 PSC (appliances) we got these error suddenly

 

 

Cannot synchronize host (host name) License not available to perform the operation.

 

Cannot complete the license assignment operation cannot assign a license to "Host" XXX Make sure the License Service is available

 

Investigation:- I found there was issue in time between PSC and VC

 

Solution:- Correct the time on PSC and restart vpxd , issue got resolved.

 

 

 

 

 

Error

 

vmware-vpxd: vpxd failed to initialize in time.

vpxd is already starting up. Aborting the request.

 

 

Stderr =

2017-03-27T16:19:06.131Z   {

    "resolution": null,

    "detail": [

        {

            "args": [

                "Command: ['/sbin/service', u'vmware-vpxd', 'start']\nStderr: "

            ],

            "id": "install.ciscommon.command.errinvoke",

            "localized": "An error occurred while invoking external command : 'Command: ['/sbin/service', u'vmware-vpxd', 'start']\nStderr: '",

            "translatable": "An error occurred while invoking external command : '%(0)s'"

        }

    ],

    "componentKey": null,

    "problemId": null

}

ERROR:root:Unable to start service vmware-vpxd, Exception: {

    "resolution": null,

    "detail": [

        {

            "args": [

                "vmware-vpxd"

            ],

            "id": "install.ciscommon.service.failstart",

            "localized": "An error occurred while starting service 'vmware-vpxd'",

            "translatable": "An error occurred while starting service '%(0)s'"

        }

    ],

    "componentKey": null,

    "problemId": null

}

Unable to start service vmware-vpxd, Exception: {

    "resolution": null,

    "detail": [

        {

            "args": [

                "vmware-vpxd"

            ],

            "id": "install.ciscommon.service.failstart",

            "localized": "An error occurred while starting service 'vmware-vpxd'",

            "translatable": "An error occurred while starting service '%(0)s'"

        }

    ],

    "componentKey": null,

    "problemId": null

 

 

vmware-vpxd: Waiting for vpxd to start listening for requests on 8089

 

Re-search here problem was in DNS server , PSC and VC were not able to ping DNS server that why got this error,

 

Originator@6876 sub=HttpConnectionPool-000001] [ConnectComplete] Connect failed to

 

DNS/AD team resolved DNS issue after that vpxd started successfully.

Once primary DNS server down , PSC lost the communication on secondary DNS.

Scenario:- 2 PSC appliances, 2 DNS server (appliances) with NLB

 

error:- The v Sphere Client could not connect to vCenter FQDN or IP Address. The server vCenter FQDN or IP Address took too long to respond. (The command has timed out as the remote server is taking too long to respond.

 

Cause:- This issue occurs if the primary DNS server is unavailable. Due to this, the default timeout & retry values used by vCenter Server Appliance to switch the secondary DNS server cause an excessive delay.

 

Resolution:-

To reduce the timeout value and allow the appliance to fail over to the next available DNS server, modify the /etc/resolv.conf file.

 

Note: If you are using a dispersed vSphere topology with one or more external PSCs, vCenter Management nodes, and Single Single On node, you must perform these steps on all appliances. If you modify the DNS settings through the VAMI or the virtual machine console, these values are lost and need to be reapplied.

Note: Take a snapshot of vCenter Appliance before proceeding.

  1. Take an SSH session to vCenter Server Appliance.
  2. Take a backup of the /etc/resolv.conf file.
  3. Open the /etc/resolv.conf file using a suitable text editor.
  4. Add these values to the end of the file:

    options timeout:1 attempts:1 rotate

    Notes:
    • The timeout value controls the time in seconds before moving on to the next DNS server.
    • The attempts value controls the number of retries before moving to the next DNS server.
    • The rotate value adds a round robin behavior.
    1. Save and close the file.
    2. Reboot the appliance.

     

    RajeevVCP4 Hot Shot
    vExpert

    503 Service Unavailable

    Posted by RajeevVCP4 Mar 5, 2017

    After fresh installation of vcenter server 6.0 ( all components on single virtual machines) I got this error.

     

    I saw vpxd (vcenter) service was stopped I tried to restart it bu got below error.

     

     

    Solution:- I changed it delay restart and restart web client service ( that was also stopped) after 1-2 minutes vpxd started it self.

     

    Now I am able to logon through web client ( it also take 5-10 minutes for initializing )

    RajeevVCP4 Hot Shot
    vExpert

    VMotion failed

    Posted by RajeevVCP4 Feb 22, 2017

    Live example

    If you got these backtrace in vpxd logs

     

    migration to complete before retrying.

    --> Failed to create a migrate heap of size 36297256: Not found

    --> Failed to allocate migration heap.

    --> Failed to initialize migration at source. Error 0xbad00a4. VMotion failed to start due to lack of cpu or memory resources

     

     

    This issue occurred when multiple instances created by DRS for migrate virtual machines , that time resources locked

     

    Only solution shutdown all running virtual machines and try to migrate , if it failed then reboot the host