Log Insight 3.3 shows multiple host entries consuming licenses - can I clean out?

[BigBjorn]

Hi all,

So I installed Log Insight 3.3 for vCenter and it helped me configure log forwarding. Everything works well except two things:

1. Duplicate hosts (see below) consuming all my OSI licenses. Anyone know how I can clean out one of the entries? (I can of course add FQDN to ESXi host name if that helps, and if supported)
2. ESXi 5.5 hosts are not in host list - double checked configuration and restarted syslog. Possible because of OSI licenses are consumed by duplicated host entries?

From release notes:

The hosts table might display devices more than once.
The hosts table might display devices more than once with each in different formats, including some combination of IP address, hostname, and FQDN. For example, a device named foo.bar.com might appear as both foo and foo.bar.com.
The hosts table uses the hostname field that is defined in the syslog RFC. If an event sent by a device over the syslog protocol does not have a hostname, vRealize Log Insight uses the source as the hostname. This might result in the device being listed more than once because vRealize Log Insight cannot determine if the two formats point to the same device.

Any tips would be highly appreciated.

Thanks

[kpcongdon]

I too am experiencing this, hoping I can get this cleaned up from both a licensing, and reporting stand point

[sflanders · Accepted solution]

For #1 there is no way to manually clear out entries -- for /admin/hosts the entry will be cleared once all data from that host has rotated out (i.e. based on retention period), for /admin/license if you click the question mark next to average active OSIs it says "The Active Average OSI count is the average daily number of hosts sending events to Log Insight." The bigger question is why are you seeing duplicates? Duplicates could be seen if forward AND reverse DNS are or were not configured properly. Duplicates may also be the result in malformed syslog events.

For #2, the issue is not duplicate OSI -- if it is not working then that means something is wrong. It could be network related including DNS resolution on the ESXi host or network firewall configuration (not host firewall configuration). You will likely want to connect to and ESXI 5.5 host and check things like validating syslog configuration, confirming networking connectivity to LI, confirming DNS resolution to the syslog destination is working, etc.

I hope this helps!

[BigBjorn]

sflanders‌ thank you for the reply (and your excellent blog articles of Log Insight!)

Host list in Log Insight includes both short host name and FQDN:

esx-01

esx-01.domain.local

esx-02

esx-02.domain.local

Running command "hostname" on ESXi, it sure shows esx-01 and not FQDN.

I will try to rename the host to FQDN and see if the old short name rotates out.

Thanks

/B

[kpcongdon]

So, in my circumstances, my forward and reverse dns are correct for my hosts.  I do however, have the hosts registered in vcenter using just their host name, and not the fqdn. I thought this might be part of the problem, but I have a lab environment with a separate eval install of loginsight where I tried registering the hosts with the fqdn in vcenter, and even in the lab, I continue to see duplicates for my hosts. 

[kpcongdon]

I believe I've got this resolved in my environment.  My output from "esxcli system hostname get" shows that the FQDN is only set to hostname.  Setting this to the proper fqdn appears to have resolved the issue with multiple names in log insight

[sflanders]

Awesome!

[BigBjorn]

Glad to hear it seems to work for you now! I still having problems though.

14 ESXi 6.0 servers (host profiles) all show duplicates

2 ESXi 5.5 hosts works good, only FQDN reports in

They all have A records and PTR records and uses the same DNS server. No problems with DNS. Looking into DCUI have the DNS/hostname settings on all hosts.

The only difference I can find is if I run hostname from Putty:

ESXi 6.0 returns hostname

ESXi 5.5 returns FQDN

Running esxcli system hostname get displays the same output on both 5.5 and 6.0

[root@esxi-01:~] esxcli system hostname get

   Domain Name: domain.local

   Fully Qualified Domain Name: esxi-01.domain.local

   Host Name: esxi-01

So, I followed KB VMware KB: Changing the name of an ESX or ESXi host rename of the ESXi 6.0 hosts. But it was not really nothing to change, you can't set FQDN in host name in vSphere Client: "The host name cannot include periods. Enter a host name without periods.".

Log Insight hostname behaviour is explained in this article Understanding the Source and Hostname fields in VMware vRealize Log Insight (2053382)   

Those events from ESXi 6.0 hosts that are causing the duplicate entries have FQDN in "source" field and short hostname in the field "hostname".

kpcongdon‌, what exactly did you do to rename?

Thanks

[kpcongdon]

In my circumstance, I didn't have to rename, as my host names were fine.  I simply set the fqdn with "esxcli system hostname set --fqdn=<fqdn>" .   I used esxcli on a few test hosts, then deployed the domain name via Host Profile for the remainder of my hosts.

[BigBjorn]

I will test that and see if it works for me too.

I noticed that host profile is blank for "Domain name portion of DNS name". That might be the issue.

Big thanks!

/B

[BigBjorn]

kpcongdon‌ sflanders‌ So, my hosts have been brewing during the weekend, I think it's safe to say now that my ESXi 6.0 hosts have stopped logging duplicate host names into Log Insight!

"esxcli system hostname set --fqdn=<fqdn>"  solved it.

I think I will rebuild my Log Insight instead of waiting for rotation. Kudos to VMware for making Log Insight setup ridiculously easy!

Thanks y'all!