TonyWhite
Contributor
Contributor

Manage Endpoint - VC_HOSTS - Collection Failed

Jump to solution

I have installed Skyline 2.0.0.0 in our environment. I keep getting "Collection Failed" on the VC_Hosts Endpoint. Here is the actual details:

MessageAn error occurred while collecting/uploading. Will rethrow to scheduler. The scheduler might cancel this task and not reschedule it again.
StateCOLLECTION_FAILED
Payload File-
Payload Size-
Last upload19 minutes ago

I have restarted the Endpoint multiple times with the same result. At this time I am not getting any inventory data being pulled into the Skyline Advisor. The product has been running for over 24 hours now.

Thanks,

Joe

0 Kudos
1 Solution

Accepted Solutions
VishShah
VMware Employee
VMware Employee

The issue appears on old Collector including after the migration to 2.0 with the Large Inventory Size vCenter added as an endpoint and other symptoms may appear CPU spike on Collector VM or crash of the appliances, a fix will be pushed into the next Skyline release as (hotfix) and release will be announced sometime next week.

Regards Vishwajit Shah Skyline Support Moderator VCA-DCV | VCP5-DCV | VCP6.0; 6.5-DCV & CMA | VCA-DBT | VCAP60-DCV

View solution in original post

0 Kudos
11 Replies
TonyWhite
Contributor
Contributor

Here are some errors I am finding in the logs:

A. java.lang.OutOfMemoryError: GC overhead limit exceeded

B. 2018-11-16 15:40:24,358 INFO [pool-21-thread-2] c.v.s.c.s.SkylineSentry [SubstituteLogger.java:169] [task=VC_HOSTS-xxxxxxxxx.xxx.net] Sentry is not enabled, exception ignored.

C. 2018-11-16 15:40:24,357 ERROR [pool-21-thread-2] EsxCliTopologyCollectionTask [ExternalCollectionTask.java:119] [task=VC_HOSTS-xxxxxxxxx.xxx.net] An error occurred while collecting/uploading. Will rethrow to scheduler. The scheduler might cancel this task and not reschedule it again.

0 Kudos
VishShah
VMware Employee
VMware Employee

Hello Tony,

We will observe this behavior when you have any ESXi host in the VC Inventory lying as "Disconnected  \ Not Responding"  state, VC_HOSTS should get re-connected back the moment you identify the host and change the state and If you do not have such host in the Inventory then I would request you to validate after a couple of hours upon appliance graceful shutdown and then Power On also I would need few details to investigate the current status from my end, I will sent you a personal response.

Regards Vishwajit Shah Skyline Support Moderator VCA-DCV | VCP5-DCV | VCP6.0; 6.5-DCV & CMA | VCA-DBT | VCAP60-DCV
0 Kudos
TonyWhite
Contributor
Contributor

Hello Vishwajit,

We did find a host in a disconnected state so we fixed that and rebooted the Skyline appliance. We are now getting the following errors:

Couldn't collect data: Failed to collect topology from host VcHostConfig[ssoConfig=SsoConfig{ssoHost=''xxxxx.xxx.net', ssoAdminUrl='https://xxxxx.xxx.net:7444/sso-adminserver/sdk/vsphere.local', ssoStsUrl='https://xxxxx.xxx.net:7444/sts/STSService/vsphere.local', ssoLsUrl='https://xxxxx.xxx.net:7444/lookupservice/sdk/vsphere.local', lsCertificateThumbprint='null', stsCertificateThumbprint='null'},hostAddress=xxxxx.xxx.net,expectedCertificateThumbprint=58:D0:CD:74:72:71:B3:73:E0:D8:16:45:14:3B:41:C2:98:35:2F:C0]

2018-11-16 18:56:04,024 ERROR [pool-4-thread-3] EsxCliTopologyCollectionTask [ExternalCollectionTask.java:106] [task=VC_HOSTS-xxxxx.xxx.net] Couldn't collect data: Failed to collect topology from host VcHostConfig[ssoConfig=SsoConfig{ssoHost='xxxxx.xxx.net', ssoAdminUrl='https://xxxxx.xxx.net:7444/sso-adminserver/sdk/vsphere.local', ssoStsUrl='https://xxxxx.xxx.net:7444/sts/STSService/vsphere.local', ssoLsUrl='https://xxxxx.xxx.net:7444/lookupservice/sdk/vsphere.local', lsCertificateThumbprint='null', stsCertificateThumbprint='null'},hostAddress=xxxxx.xxx.net,expectedCertificateThumbprint=58:D0:CD:74:72:71:B3:73:E0:D8:16:45:14:3B:41:C2:98:35:2F:C0]

2018-11-16 18:56:04,015 INFO [pool-4-thread-3] c.v.s.c.s.SkylineSentry [SubstituteLogger.java:169] [task=VC_HOSTS-xxxxx.xxx.net] Sentry is not enabled, exception ignored.

In addition the VC_EVENTS endpoint is now reporting Collection Failed as well.

Thanks,

Joe

0 Kudos
HarishV
Enthusiast
Enthusiast

Hello Joe,

Hope you are doing well.

Just wanted to know if you can provide me your registered email ID and the Account details in a personal email so that we can get on a webex session.

I will send you a personal message.

Sincerely Harish Venkatachalam Skyline Support Moderator
0 Kudos
VishShah
VMware Employee
VMware Employee

Hello Joe,

We have been seeing this after the immediate migration of Skyline Advisor to CSP it's not a bug this is functionality we've added with the latest 2.0 release it does not impact anything because the feature is not used anymore or its currently disabled.

Regards Vishwajit Shah Skyline Support Moderator VCA-DCV | VCP5-DCV | VCP6.0; 6.5-DCV & CMA | VCA-DBT | VCAP60-DCV
0 Kudos
VishShah
VMware Employee
VMware Employee

The issue appears on old Collector including after the migration to 2.0 with the Large Inventory Size vCenter added as an endpoint and other symptoms may appear CPU spike on Collector VM or crash of the appliances, a fix will be pushed into the next Skyline release as (hotfix) and release will be announced sometime next week.

Regards Vishwajit Shah Skyline Support Moderator VCA-DCV | VCP5-DCV | VCP6.0; 6.5-DCV & CMA | VCA-DBT | VCAP60-DCV
0 Kudos
royiversen
Contributor
Contributor

I just deployed Skyline 2.0 for the first time and I'm having the same issue. Is the fix released yet?

0 Kudos
Yuva_1990
Hot Shot
Hot Shot

Hi Royiversen,

Thank you for the response.Can you please confirm the current version of the collector,issue is resolved in 2.0.0.2 which is released today.

Regards,

Yuvaraj.

Skyline Support Moderator

0 Kudos
royiversen
Contributor
Contributor

I downloaded and installed today and I'm still on 2.0.0.0. Where/how can I get 2.0.0.2 ?

0 Kudos
Yuva_1990
Hot Shot
Hot Shot

Hi Royiversen,

Please find the below steps to install the new updates

  1. Open a web browser and enter the URL for the Skyline Collector VAMI: https://skyline_collector_ip_address_or_fqdn:5480
  2. Enter the user name root and associated password.
  3. Click Login.
  4. Click Update.
  5. Click Check Updates.
  6. If a new update is available, click Install Updates

Regards

Yuvaraj

Skyline Support Moderator

0 Kudos
royiversen
Contributor
Contributor

Thank you, that seems to have resolved that issue, but VC_HOSTS has been in a Unknown State for over 10 hours now. I'll try restarting it again.

0 Kudos