ha logs location

[tdubb123]

where can i find the ha logs when i reconfigure a host for ha in a cluster and it hangs?

[AndreTheGiant]

The Automatic Availability Manager (AAM) logs are located at  /var/log/vmware/aam/vmware_<hostname>-xxx.log.

Andre

[NickEvans]

HA logs are located in /var/log/vmware/aam

In particular, check the add node log file  /var/log/vmware/aam/aam_config_util_addnode.log

This will give an indication on where its failing.

Cheers,

Nick.

[tdubb123]

cannot get ha to work

08/22/11 05:04:38 [issue_cmd           ]

08/22/11 05:04:38 [issue_cmd           ] --- 10.51.0.1 ping statistics ---

08/22/11 05:04:38 [issue_cmd           ] 1 packets transmitted, 1 packets received, 0% packet loss

08/22/11 05:04:38 [issue_cmd           ] round-trip min/avg/max = 0.512/0.512/0.512 ms

08/22/11 05:04:38 [issue_cmd           ]

08/22/11 05:04:38 [issue_cmd           ] CMD:    cp -f /var/lib/vmware/aam/ftbb.prm /var/lib/vmware/aam/ftbb.prm.bck

08/22/11 05:04:38 [issue_cmd           ] STATUS: 0

08/22/11 05:04:38 [issue_cmd           ] RESULT:

08/22/11 05:04:38 [issue_cmd           ]

08/22/11 05:04:38 [ft_startup_monitor  ] Waiting for /opt/vmware/aam/bin/ft_startup to complete

08/22/11 05:09:41 [issue_cli_cmd       ] command is '/opt/vmware/aam/bin/ftcli -domain vmware -cmd "stop VMap_eq-esx2 -wait=15"'

08/22/11 05:09:41 [issue_cmd           ] CMD:    /opt/vmware/aam/bin/ftcli -domain vmware -cmd "stop VMap_eq-esx2 -wait=15"

08/22/11 05:09:41 [issue_cmd           ] STATUS: 1

08/22/11 05:09:41 [issue_cmd           ] RESULT:

08/22/11 05:09:41 [issue_cmd           ] cli.c:doProxyConnect:1904: error:

08/22/11 05:09:41 [issue_cmd           ] [Err:13027] Cannot connect to a secondary agent

08/22/11 05:09:41 [issue_cmd           ]

08/22/11 05:09:41 [issue_cli_cmd       ] command is '/opt/vmware/aam/bin/ftcli -domain vmware -cmd "forceProcState VMap_eq-esx2 stopped"'

08/22/11 05:09:41 [issue_cmd           ] CMD:    /opt/vmware/aam/bin/ftcli -domain vmware -cmd "forceProcState VMap_eq-esx2 stopped"

08/22/11 05:09:41 [issue_cmd           ] STATUS: 1

08/22/11 05:09:41 [issue_cmd           ] RESULT:

08/22/11 05:09:41 [issue_cmd           ] cli.c:doProxyConnect:1904: error:

08/22/11 05:09:41 [issue_cmd           ] [Err:13027] Cannot connect to a secondary agent

08/22/11 05:09:41 [issue_cmd           ]

08/22/11 05:09:41 [shut_down_vmap_proce] attempt to stop VMap_eq-esx2 failed.

08/22/11 05:09:42 [issue_cmd           ] CMD:    /opt/vmware/aam/bin/ft_shutdown -b -ppid=189151

08/22/11 05:09:42 [issue_cmd           ] STATUS: 0

08/22/11 05:09:42 [issue_cmd           ] RESULT:

08/22/11 05:09:42 [issue_cmd           ] Shutting down the AAM agent for domain vmware

08/22/11 05:09:42 [issue_cmd           ]

08/22/11 05:09:42 [issue_cmd           ] Shutting down the Agent...

08/22/11 05:09:42 [issue_cmd           ]

08/22/11 05:09:42 [issue_cmd           ] cli.c:doProxyConnect:1904: error:

08/22/11 05:09:42 [issue_cmd           ]  [Err:13027] Cannot connect to a secondary agent

08/22/11 05:09:42 [issue_cmd           ]

08/22/11 05:09:42 [issue_cmd           ]

08/22/11 05:09:42 [issue_cmd           ]

08/22/11 05:09:42 [issue_cmd           ] Shutting down the Backbone...

08/22/11 05:09:42 [issue_cmd           ] Skipping /opt/vmware/aam/bin/ftPerl (189919) because it's this script.

08/22/11 05:09:42 [issue_cmd           ] Adding /opt/vmware/aam/bin/ftAgent (189283) to the kill list (AAM)

08/22/11 05:09:42 [issue_cmd           ] Adding /opt/vmware/aam/bin/ftPerl (189268) to the kill list (AAM)

08/22/11 05:09:42 [issue_cmd           ] Skipping /opt/vmware/aam/bin/ftPerl (189151) because it is a parent script.

08/22/11 05:09:42 [issue_cmd           ] Killing PIDs:  189283 189268

08/22/11 05:09:42 [issue_cmd           ] After kills issued:

08/22/11 05:09:42 [issue_cmd           ] PID:  PROC:

08/22/11 05:09:42 [issue_cmd           ] PID:  PROC:

08/22/11 05:09:42 [issue_cmd           ] PID: 189939 PROC: ps

08/22/11 05:09:42 [issue_cmd           ] PID: 189937 PROC: /bin/sh

08/22/11 05:09:42 [issue_cmd           ] PID: 189936 PROC: /bin/sh

08/22/11 05:09:42 [issue_cmd           ] PID: 189919 PROC: /opt/vmware/aam/bin/ftPerl

08/22/11 05:09:42 [issue_cmd           ] PID: 189671 PROC: /bin/sh

08/22/11 05:09:42 [issue_cmd           ] PID: 189407 PROC: tail

08/22/11 05:09:42 [issue_cmd           ] PID: 189151 PROC: /opt/vmware/aam/bin/ftPerl

08/22/11 05:09:42 [issue_cmd           ] PID: 179005 PROC: /usr/libexec/hostd/nssquery

08/22/11 05:09:42 [issue_cmd           ] PID: 110249 PROC: /usr/lib/ext/dell/srvadmin/sbin/dsm_sa_snmpd

08/22/11 05:09:42 [issue_cmd           ] PID: 110238 PROC: /usr/lib/ext/dell/srvadmin/sbin/dsm_sa_eventmgrd

08/22/11 05:09:42 [issue_cmd           ] PID: 108562 PROC: /usr/lib/ext/dell/srvadmin/sbin/dsm_sa_datamgrd

08/22/11 05:09:42 [issue_cmd           ] PID: 108546 PROC: /sbin/sfcbd

08/22/11 05:09:42 [issue_cmd           ] PID: 108545 PROC: /usr/lib/ext/dell/srvadmin/bin/om_launcher

08/22/11 05:09:42 [issue_cmd           ] PID: 108541 PROC: /sbin/sfcbd

08/22/11 05:09:42 [issue_cmd           ] PID: 108540 PROC: /sbin/sfcbd

08/22/11 05:09:42 [issue_cmd           ] PID: 108041 PROC: /sbin/sfcbd

08/22/11 05:09:42 [issue_cmd           ] PID: 108038 PROC: /sbin/sfcbd

08/22/11 05:09:42 [issue_cmd           ] PID: 108026 PROC: /sbin/sfcbd

08/22/11 05:09:42 [issue_cmd           ] PID: 108025 PROC: /sbin/sfcbd

08/22/11 05:09:42 [issue_cmd           ] PID: 108024 PROC: /sbin/sfcbd

08/22/11 05:09:42 [issue_cmd           ] PID: 108022 PROC: /bin/cimslp

08/22/11 05:09:42 [issue_cmd           ] PID: 108021 PROC: /sbin/sfcbd

08/22/11 05:09:42 [issue_cmd           ] PID: 32620 PROC: -ash

08/22/11 05:09:42 [issue_cmd           ] PID: 32619 PROC: dropbear

08/22/11 05:09:42 [issue_cmd           ] PID: 8775 PROC: /bin/login

08/22/11 05:09:42 [issue_cmd           ] PID: 8765 PROC: /sbin/dcui

08/22/11 05:09:42 [issue_cmd           ] PID: 8764 PROC: /bin/ash

08/22/11 05:09:42 [issue_cmd           ] PID: 8743 PROC: /usr/sbin/net-cdp

08/22/11 05:09:42 [issue_cmd           ] PID: 8731 PROC: crond

08/22/11 05:09:42 [issue_cmd           ] PID: 8729 PROC: /bin/sh

08/22/11 05:09:42 [issue_cmd           ] PID: 8724 PROC: /bin/sh

08/22/11 05:09:42 [issue_cmd           ] PID: 5498 PROC: /bin/sh

08/22/11 05:09:42 [issue_cmd           ] PID: 5497 PROC: /bin/sh

08/22/11 05:09:42 [issue_cmd           ] PID: 5487 PROC: /usr/lib/vmware/bin/vmware-usbarbitrator

08/22/11 05:09:42 [issue_cmd           ] PID: 5477 PROC: /bin/sh

08/22/11 05:09:42 [issue_cmd           ] PID: 5410 PROC: /opt/vmware/vpxa/vpx/vpxa

08/22/11 05:09:42 [issue_cmd           ] PID: 5401 PROC: /bin/sh

08/22/11 05:09:42 [issue_cmd           ] PID: 5371 PROC: /sbin/openwsmand

08/22/11 05:09:42 [issue_cmd           ] PID: 5361 PROC: /bin/sh

08/22/11 05:09:42 [issue_cmd           ] PID: 5260 PROC: /sbin/slpd

08/22/11 05:09:42 [issue_cmd           ] PID: 5149 PROC: vprobed

08/22/11 05:09:42 [issue_cmd           ] PID: 5139 PROC: /bin/sh

08/22/11 05:09:42 [issue_cmd           ] PID: 5122 PROC: vobd

08/22/11 05:09:42 [issue_cmd           ] PID: 5112 PROC: /bin/sh

08/22/11 05:09:42 [issue_cmd           ] PID: 5100 PROC: storageRM

08/22/11 05:09:42 [issue_cmd           ] PID: 5090 PROC: /bin/sh

08/22/11 05:09:42 [issue_cmd           ] PID: 5052 PROC: net-lbt

08/22/11 05:09:42 [issue_cmd           ] PID: 5042 PROC: /bin/sh

08/22/11 05:09:42 [issue_cmd           ] PID: 5030 PROC: hostd

08/22/11 05:09:42 [issue_cmd           ] PID: 5020 PROC: /bin/sh

08/22/11 05:09:42 [issue_cmd           ] PID: 4996 PROC: /sbin/ntpd

08/22/11 05:09:42 [issue_cmd           ] PID: 4928 PROC: inetd

08/22/11 05:09:42 [issue_cmd           ] PID: 4791 PROC: /usr/sbin/vmkiscsid

08/22/11 05:09:42 [issue_cmd           ] PID: 4418 PROC: vmklogger

08/22/11 05:09:42 [issue_cmd           ] PID: 4414 PROC: syslogd

08/22/11 05:09:42 [issue_cmd           ] PID: 4397 PROC: /usr/lib/vmware/bin/vmkeventd

08/22/11 05:09:42 [issue_cmd           ] PID: 4199 PROC: /sbin/init

08/22/11 05:09:42 [issue_cmd           ]

08/22/11 05:09:42 [issue_cmd           ] Backing up the AAM configuration to persistent storage

08/22/11 05:09:42 [issue_cmd           ]

08/22/11 05:09:42 [stop_aam            ] copying /var/lib/vmware/aam/vmware-sites to /var/log/vmware/aam/aam_config_util_addnode.log

FULLTIME_SITES_TID 00000012

+ 1:8042,8042,8043 eq-esx1    vmware #FT_Agent_Port=8045

- 2:8042,8042,8043 eq-esx3 vmware

+ 3:8042,8042,8043 eq-esx2 vmware

08/22/11 05:09:42 [vpxa_respond        ] VMwareerrortext=/opt/vmware/aam/bin/ft_startup failed to complete within 3 minutes

08/22/11 05:09:42 [vpxa_respond        ] VMwareerrorcat=internalerror

08/22/11 05:09:42 [myexit              ] copying /var/lib/vmware/aam/vmware-sites to /var/log/vmware/aam/aam_config_util_addnode.log

FULLTIME_SITES_TID 00000012

+ 1:8042,8042,8043 eq-esx1    vmware #FT_Agent_Port=8045

- 2:8042,8042,8043 eq-esx3 vmware

+ 3:8042,8042,8043 eq-esx2 vmware

08/22/11 05:09:42 [myexit              ] Failure location:

08/22/11 05:09:42 [myexit              ]        function main::myexit called from line 1957

08/22/11 05:09:42 [myexit              ]        function main::ft_startup_monitor called from line 2277

08/22/11 05:09:42 [myexit              ]        function main::start_agent called from line 1238

08/22/11 05:09:42 [myexit              ]        function main::add_aam_node called from line 210

08/22/11 05:09:42 [myexit              ] VMwareresult=failure

08/22/11 05:09:42 [elapsed_time        ] Total time for script to complete:  5 minute(s) and 7 second(s)

[john23]

08/22/11 05:09:42 [issue_cmd           ] cli.c:doProxyConnect:1904: error:

08/22/11 05:09:42 [issue_cmd           ]  [Err:13027] Cannot connect to a secondary agent

Seeing this logs, either host are in different subnet or not reachable to each other.

if different subnet try this option:

http://www.yellow-bricks.com/2008/08/08/dasallownetwork-where-and-when/

[rsaha]

Use same subnet and check physical NW connectivity with NW team.Hope no firewall is there btwn hosts.

[tdubb123]

all hosts are on the same subnet.

[DSTAVERT]

Make sure that you can ping or vmkping from and to each host and vCenter. http://kb.vmware.com/kb/1003728 Make sure DNS resolves and or use IP addresses or HOST files.

[john23]

what is both hosts network configuration?

[arturka]

Hi

take have a look at that link

http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&externalId=1018217&sl...

I have this same problem like you have it, now my network guys trying to figure out what is going on.

Cheers

Cheers
Artur

Please, don't forget the awarding points for "helpful" and/or "correct" answers.