topic Re: VMware vCSA 6.7 too much noisy ! Need Warning syslog level only! in vCenter™ Server Discussions

VMware vCSA 6.7 too much noisy ! Need Warning syslog level only!

AntexMv — Mon, 30 Nov 2020 09:28:21 GMT

Dear Community
Please advise

Regardless syslog level Warning - it sends tons of syslog messages including Info, Notes and even Debug level

Please advise how it is possible to limit this flood with Waring syslog messages only.
Regards, AntexMv

Re: VMware vCSA 6.7 too much noisy ! Need Warning syslog level only!

scott28tt — Mon, 30 Nov 2020 10:52:20 GMT

@AntexMv

Moderator: Moved to vCenter Server Discussions

Re: VMware vCSA 6.7 too much noisy ! Need Warning syslog level only!

vxprthu — Mon, 30 Nov 2020 13:08:49 GMT

HI @AntexMv

Can you please go to General tab and check the logging settings there?

Regards,

Re: VMware vCSA 6.7 too much noisy ! Need Warning syslog level only!

AntexMv — Mon, 30 Nov 2020 16:00:48 GMT

Dear megotloves

Thank you for your attention

I double check that setting and it is correct -what we need -Warning level

Please kindly checked attached photo

Any other suggestions ?

Regards, AntexMv

Re: VMware vCSA 6.7 too much noisy ! Need Warning syslog level only!

AntexMv — Mon, 07 Dec 2020 05:06:25 GMT

Dear megotloves

Again thank you for your attention

I double check that setting in General tab and it is correct -what we need -Warning level

I tried to extract and change Host Profile in Policies and Profiles tab. I found there some string which mentions syslog logging severity level

Host Profile in Policies -> Host Profiles (need to extract from server/host or create new)->Configure-> Advanced Configuration Settings -> Host Profile Log Configuration -> Log Level WARN

Now start testing. Let's see.

Regards, AntexMv

Re: VMware vCSA 6.7 too much noisy ! Need Warning syslog level only!

vxprthu — Mon, 07 Dec 2020 08:21:32 GMT

HI @AntexMv

With host profile only the hypervisors' configuration could be done, not the vcenter's.

If you can share some log entries, we could figure out what is the source and recommend solution to change.

Regards,

Re: VMware vCSA 6.7 too much noisy ! Need Warning syslog level only!

AntexMv — Mon, 07 Dec 2020 09:32:29 GMT

Dear Enthusiast

Thank you again for your reply

"With host profile only the hypervisors' configuration could be done, not the vcenter's." - I agree

Today I tried to change Syslog Level everywhere I found it

And still some services (updatemgr and vpxd-svcs) send Info and even Debug messages

Kindly look at attached files

I did not find where to change Log Level for those services (if those are correct source) even through CLI

Thank you in advance for your advice

Regards,

Re: VMware vCSA 6.7 too much noisy ! Need Warning syslog level only!

vxprthu — Mon, 07 Dec 2020 10:43:50 GMT

HI @AntexMv

To change updatemgr's log level, you should use the old flex gui. Then go to home > administration > system configuration > services > update manager > manage.

You can check other services here, you may found some where you can change.

Also there is an option to change syslog settings in cli.

You can edit the (r)syslog.conf like this:

*.error;*.crit;*.alert @Syslogserversip:port;RSYSLOG_SyslogProtocol23Format

The restart the service:

systemctl restart rsyslog

Then you can test with the logger:

logger -p syslog.error "This should go through" logger -p syslog.info "This should not go through"

And of course, please take backup/snapshot of the appliance and/or the files you changed.

Regards,

Re: VMware vCSA 6.7 too much noisy ! Need Warning syslog level only!

AntexMv — Tue, 08 Dec 2020 05:25:23 GMT

Dear megotloves

Thank you much for your support

I tried everything. I found INFO in Flex, CLI and replace them for WARNING

I modified 3 files - rsyslog.conf ; rsyslog.conf.orig ; rsyslog.conf.rpmnew

You provide me with useful information - thank you much.

But seems we are missing something as I still receiving INFO and DEBUG messages.

And test shows that INFO goes through

Kindly look at attached files

May be you have other ideas

Regards, AntexMv

################################################################################
############################# VMware Rsyslog Configuration ####################
################################################################################
###### Module declarations ######
module( load="imtcp"
streamdriver.name="gtls"
streamdriver.mode="1"
streamdriver.authmode="anon"
gnutlsprioritystring="NONE:+AES-128-GCM:+AES-256-CBC:+AES-128-CBC:+ECDHE-RSA:+ECDHE-ECDSA:+RSA:+AEAD:+SHA384:+SHA256:+SHA1:+COMP-NULL:+VERS-TLS1.2:+SIGN-RSA-SHA224:+SIGN-RSA-SHA256:+SIGN-RSA-SHA384:+SIGN-RSA-SHA512:+SIGN-DSA-SHA224:+SIGN-DSA-SHA256:+SIGN-ECDSA-SHA224:+SIGN-ECDSA-SHA256:+SIGN-ECDSA-SHA384:+SIGN-ECDSA-SHA512:+CURVE-SECP256R1:+CURVE-SECP384R1:+CURVE-SECP521R1:+CTYPE-OPENPGP:+CTYPE-X509:-CAMELLIA-256-CBC:-CAMELLIA-192-CBC:-CAMELLIA-128-CBC:-CAMELLIA-256-GCM:-CAMELLIA-128-GCM"
)
input(type="imtcp" port="1514")
$ModLoad imuxsock.so

$ModLoad imptcp.so # TCP
$ModLoad imudp.so # UDP
$ModLoad omrelp.so # RELP
###### Common configuration ######
$EscapeControlCharactersOnReceive off
###### Template declarations ######
$template defaultLoc,"/var/log/vmware/%app-name%/%app-name%-syslog.log"
$template defaultFmt,"%timestamp:::date-rfc3339% %syslogseverity-text% %app-name% %msg%\n"
$template vpxdLoc,"/var/log/vmware/%app-name%/%app-name%-syslog.log"
$template vpxdFmt,"%msg%\n"
$template rsyslogadminLoc,"/var/log/vmware/%app-name%/%app-name%-syslog.log"
$template rsyslogadminFmt,"%timestamp:::date-rfc3339% %syslogseverity-text% %app-name% %msg%\n"
$template esxLoc,"/var/log/vmware/esx/%hostname%/%hostname%-syslog.log"
$template esxFmt,"%timestamp:::date-rfc3339% %syslogseverity-text% %hostname% %app-name% %msg%\n"
$template defaultSystemLoc,"/var/log/vmware/messages"
###### Rule declarations ######
# TCP/UDP/rsyslog input ruleset declaration
$RuleSet all
# Make gtls driver the default
$DefaultNetstreamDriver gtls
# Shared certificate authority certificate
$DefaultNetstreamDriverCAFile /etc/vmware/vmware-vmafd/ca.crt
# Client certificate
$DefaultNetstreamDriverCertFile /etc/vmware/vmware-vmafd/machine-ssl.crt
# Client key
$DefaultNetstreamDriverKeyFile /etc/vmware/vmware-vmafd/machine-ssl.key
# Include the configuration for syslog relay
# _must_ be first to relay all messages
$IncludeConfig /etc/vmware-syslog/syslog.conf
# vmware services
:programname, isequal, "applmgmt-audit" ?defaultLoc;defaultFmt
& stop
:programname, isequal, "vmdird" ?defaultLoc;defaultFmt
& stop
:programname, isequal, "vmafdd" ?defaultLoc;defaultFmt
& stop
:programname, isequal, "vmcad" ?defaultLoc;defaultFmt
& stop
:programname, isequal, "vmdnsd" ?defaultLoc;defaultFmt
& stop
:programname, isequal, "rbd" ?defaultLoc;defaultFmt
& stop
:app-name, startswith, "rsyslog" ?rsyslogadminLoc;rsyslogadminFmt
& stop
:programname, isequal, "vmon" ?defaultLoc;defaultFmt
& stop
:programname, isequal, "vmcamd" ?defaultLoc;defaultFmt
& stop
:programname, isequal, "pod" stop
:programname, isequal, "updatemgr" stop
# vpxd-svcs logs to its local logs, hence avoiding duplicate logging.
:programname, isequal, "vpxd-svcs" stop
# vmware-hvc logs to its local logs, hence avoiding duplicate logging.
:programname, isequal, "hvc" stop
# vpxd logs to its local logs, hence avoiding duplicate logging.
:programname, isequal, "vpxd" stop
# For local host's syslog and system logs use the following rules
# localhost
if $fromhost contains $$myhostname then ?defaultSystemLoc
& stop
#localhost
:fromhost-ip, isequal, "127.0.0.1" ?defaultSystemLoc
& stop
# ESX rules
# Define large LinkedList action queue with 2K msgs cap to accomodate 100 ESXs
$ActionQueueSize 2000
# Do not choke ESXs, rather start dropping messages after queue is 97.5% full
$ActionQueueDiscardMark 1950
$ActionQueueDiscardSeverity 0
$ActionQueueTimeoutEnqueue 1
# VC syslog server log collection
*.* ?esxLoc;esxFmt
###### Input server declarations ######
# Setup input flow
$DefaultRuleset all
$InputPTCPServerBindRuleset all
$InputPTCPServerRun 514
$InputUDPServerBindRuleset all
$UDPServerRun 514
$InputTCPServerBindRuleset all
*.warning;*.error;*.crit;*.alert @Syslogserversip:port;RSYSLOG_SyslogProtocol23Format

#
# cron log entries for GEN003160
#
cron.* -/var/log/cron

#
# auth.log entries for GEN003660
#
auth.* -/var/log/auth.log

Re: VMware vCSA 6.7 too much noisy ! Need Warning syslog level only!

vxprthu — Tue, 08 Dec 2020 07:15:44 GMT

HI,)

It's my bad, i wasn't clear. You need to edit /etc/vmware-syslog/syslog.conf instead of rsyslog.conf

Where i said @Syslogserversip:port; you have to update to yours (if it's not already there, but im sure its there.)

Probably you will see something like this in /etc/vmware-syslog/syslog.conf:

*.* 192.168.1.1:514;RSYSLOG_SyslogProtocol23Format

and that needs to be updated to this one:

*.warn;*.err;*.crit;*.alert 192.168.1.1:514;RSYSLOG_SyslogProtocol23Format

Then reboot rsyslog service.

And don't forget to change back the rsyslog files to their original one.

Cheers,

Re: VMware vCSA 6.7 too much noisy ! Need Warning syslog level only!

AntexMv — Tue, 08 Dec 2020 10:04:53 GMT

Dear megotloves

Seems it works now!!!

Thank you very much for your assistance!!!

Test

logger -p syslog.error "This should go through"
logger -p syslog.info "This should not go through"

also clearly works!

This was nice lesson, indeed

If you don't mind, have a look at other post - may be we together could solve the issue ?

Dear Community

Please advise. vCSA 6.7 constantly sends Error 21 Failed listing records from zone vsphere.local.... error 21

DNS Server works and nslookup allows to resolve names. DNS configured for 2 zones. And allows to resolve names.

Did not find any descriptions in vmware documentation or even Google. May be somebody encountered such errors and and could advice remediation procedure

Regards, AntesMv

https://communities.vmware.com/t5/vCenter-Server-Discussions/Failed-listing-records-from-zone-error-21-revmdnsd-vcenter/m-p/2813508#M91043

Regards, AntexMv

Re: VMware vCSA 6.7 too much noisy ! Need Warning syslog level only!

AntexMv — Tue, 31 Aug 2021 14:02:28 GMT

Dear Enthusiast

Long time since I we chat. Let me ask again your kind advise

Few days ago I reinstalled vCSA. Version 6_7_0_48 Everything seems OK. But again I have issue with logging

Seems everything is clear and algorithm straight forward. But every time I try to modify syslog.conf file, Syslog Server configuration disappearing from Syslog for vCenter Server Appliance Management menu. if only I change from *.* to anything (*.warn or *.error) for example - vCSA Manager loose Forwarding configuration. I tried different abbreviation or digits instead of name. I changed permissions and reloaded rsyslog or syslog daemons. Nothing help. Very strange. Version 6.7.0.48 does not accept any changes. Coud you please advise what it could be ?

Thank you much in advance, AntexMv

syslog.conf

*.* @192.168.X.Y:514;RSYSLOG_SyslogProtocol23Format
*.* @@192.168.X.V:514;RSYSLOG_SyslogProtocol23Format
*.* @192.168.X.Z:514;RSYSLOG_SyslogProtocol23Format

syslog_modified_full_name_levels_v_6_7_0_48000.conf

*.warning;*.error;*.crit;*.alert @192.168.X.Y:514;RSYSLOG_SyslogProtocol23Format
*.warning;*.error;*.crit;*.alert @192.168.X.V:514;RSYSLOG_SyslogProtocol23Format
*.warning;*.error;*.crit;*.alert @192.168.X.Z:514;RSYSLOG_SyslogProtocol23Format