HB 6.6 failover issue after HB installation

[RAF777]

Hi Guys,

I have P2P HB installation, and after manual failover secondary server cant start vcenter service. I got those in vcenter logs:

2014-01-28T13:24:30.070Z [08620 info '[SSO][CreateSsoFacade]'] [CreateUserDirectory] Admin URI set to: https://xxxxxxx:7444/sso-adminserver/sdk

2014-01-28T13:24:30.070Z [08620 info '[SSO][CreateSsoFacade]'] [CreateUserDirectory] Groupcheck URI set to: https://xxxxxx:7444/sso-adminserver/sdk

2014-01-28T13:24:31.146Z [05312 error 'HttpConnectionPool-000001'] [ConnectComplete] Connect failed to <cs p:000000000c2c1830, TCP:xxxxxxxx:7444>; cnx: (null), error: class Vmacore::SystemException(No connection could be made because the target machine actively refused it. )

2014-01-28T13:24:31.208Z [08620 error '[SSO][SsoCertificateManagerImpl]'] [CreateAdminSsoServiceContent] Failure while trying to connect to SSO Admin server: No connection could be made because the target machine actively refused it.  . Will retry in 10 seconds.

2014-01-28T13:24:32.004Z [07304 warning 'VpxProfiler' opID=SWI-2b8f9c44] VpxUtil_InvokeWithOpId [TotalTime] took 12012 ms

2014-01-28T13:24:42.238Z [08464 error 'HttpConnectionPool-000001'] [ConnectComplete] Connect failed to <cs p:000000000c2c1830, TCP:xxxxxxx:7444>; cnx: (null), error: class Vmacore::SystemException(No connection could be made because the target machine actively refused it. )

2014-01-28T13:24:42.238Z [08620 error '[SSO][SsoCertificateManagerImpl]'] [CreateAdminSsoServiceContent] Failure while trying to connect to SSO Admin server: No connection could be made because the target machine actively refused it.  . Will retry in 10 seconds.

2014-01-28T13:24:44.016Z [05312 warning 'VpxProfiler' opID=SWI-752baa88] VpxUtil_InvokeWithOpId [TotalTime] took 12012 ms

2014-01-28T13:24:53.267Z [08464 error 'HttpConnectionPool-000001'] [ConnectComplete] Connect failed to <cs p:000000000c2c1830, TCP:xxxxxxx:7444>; cnx: (null), error: class Vmacore::SystemException(No connection could be made because the target machine actively refused it. )

2014-01-28T13:24:53.267Z [08620 error '[SSO][SsoCertificateManagerImpl]'] [CreateAdminSsoServiceContent] Failure while trying to connect to SSO Admin server: No connection could be made because the target machine actively refused it.  . Will retry in 10 seconds.

2014-01-28T13:24:56.029Z [02252 warning 'VpxProfiler' opID=SWI-f47830f5] VpxUtil_InvokeWithOpId [TotalTime] took 12012 ms

2014-01-28T13:25:04.297Z [04356 error 'HttpConnectionPool-000001'] [ConnectComplete] Connect failed to <cs p:000000000c2c1830, TCP:xxxxxxx:7444>; cnx: (null), error: class Vmacore::SystemException(No connection could be made because the target machine actively refused it. )

2014-01-28T13:25:04.297Z [08620 error '[SSO][SsoCertificateManagerImpl]'] [CreateAdminSsoServiceContent] Failure while trying to connect to SSO Admin server: No connection could be made because the target machine actively refused it.  . Will retry in 10 seconds.

2:25 PM 1/28/20142:28 PM 1/28/2014

xxxxx - is the name of primary server, and above logs entries are from secondary. For some reason its trying to reach sso admin server on primary. When switched back to primary it works like a charm. Do you have any thoughts? Would appreciate any help. Thanks.

[dmihaescu]

What versions of vCenter, vCSHB are you using?

Were the servers renamed after the install? (both primary and secondary)

[RAF777]

vcenter is 5.1

HB is 6.6

servers were renamed, both have different names now. Im not sure why it asks to rename if I have both servers already with different names.

[dmihaescu]

Is this an upgrade?

When did you rename the Primary?

[RAF777]

its a fresh install, both servers where renamed as HB install program wanted (it will ask for names during install) Both servers are in domain, so for rename purpose I used short names from fqdn so both names will stay the same. I had to manually rename secondary as it failed to rename by HB, but still should do the trick.

[dmihaescu]

I am sorry, I still do not understand when were the servers renamed. Did you let vCSHB setup rename them or did you rename them manually after cloning before running setup?

[RAF777]

primary server was renamed by HB during installation, secondary failed and HB asked to rename it manually which was done after setup finished

[dmihaescu]

Thanks for the reply.

Did you set the SSO password for the vCenter plugin? in the install guide, Post-Instalation Configuration, page 43 you have the following paragraph:

Configuring VirtualCenter Plug-in with the Correct Credentials

When protecting vCenter Server, after installation is complete you must enter the credentials for an account with rights to the Virtual Infrastructure to allow evaluation of rules.

To add the Virtual Infrastructure credentials:

Procedure

1 Using the vCenter Server Heartbeat Console, navigate to the Applications: Plug-ins page.

2 Select the VirtualCenter Plug-in.

3 Click Edit.

4 Type the Username and Password for an account with rights to the Virtual Infrastructure.

Important If vCenter Server 5.1 is installed, you must also enter the MasterPassword for SSO.

5 Click OK.

you will probably also need to refresh the SSO service using the following KB:

VMware KB: After making a change or restarting vCenter Single Sign-On server system, vCenter Server ...

[RAF777]

thanks for the reply. Regarding plugin - yes I set it as guide request. I did also steps from kb you mentioned and it doesnt work on secondary, I tried to list identity sources, after switch to secondary but I got java error - cant connect and it returned fqdn of primary server look like it still points to primary somewhere.....

I browsed some more logs and it seems that sso app server name is set up as primary server fqdn, shouldn't it be service fqdn instead?

[dmihaescu]

yes, it should connect to the SSO using the Public Name (service name).

[rh5592]

Before installing heartbeat, what is your primary vCenter hostname? Is it the Cluster FQDN? I encountered this issue in a previous deployment and I found out that before install, you need to have your primary vCenter server setup in such a way that it will be the end state vCenter with heartbeat, meaning the hostname should be the cluster hostname.

Ive blogged about it here: vCenter Heartbeat Gotchas! | virtXpress

[RAF777]

Primary vcenter was not set up with cluster/service fqdn, instead both servers had standard names used on this project. Is there any way to fix that issue?

[rh5592]

You may try to open a case with GSS and see if they can fix this. But honestly it will take significant time and effort. That's why in my case I just decided to just rebuild it and install the correct way.

[dmihaescu]

you should re-install both HB and vCSHB following the instructions from the install guide.

[RAF777]

I dotn know why you assume I didnt follow the guide. Just cant find info in guide about configuring primary server with service/cluster name prior to installation......

Anyway thanks guys for help, will sent ticket to resolve that.

[dmihaescu]

In the install guide, Section 3 Installing vCenter Server Heartbeat on page 35 you have the following note marked as important:

Before initiating the installation of vCenter Server Heartbeat, ensure that the Primary node is assigned the intended Public name.

vCSHB is the last piece of software that goes on your server after vCenter is installed and configured. vCenter should be installed and configured using the Public Name on the Primary, then cloned to create the secondary and then when you install vCSHB both servers will be renamed to their respective management names.

[RAF777]

Thanks, I missed that bit.... Actually I should get that info from my development team...