ESX 3.02 error message in vemkernel file

[mdangel1]

Can someone please tell me if they have seen this error before in the vmkernel file. Please help

Nov 20 14:25:54 uspmitx0010014 vmkernel: VMWARE SCSI Id: Device id info for vmhba2:0:3: 0x1 0x3 0x0 0x10 0x60 0xa 0xb 0x80 0x0 0x26 0xa7 0x9e 0x0 0x0 0xed 0xd0 0x47 0x17 0x6c 0x6 0x1 0x93 0x0 0x8 0x20 0x15 0x0 0xa0 0xb8 0x26 0xa7 0x9e 0x1 0x94 0x0 0x4 0x0 0x0 0x0 0x1

[Texiwill]

Hello,

I have not but what are the error messages surrounding this one, that may give some clues.

Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education.

SearchVMware Blog: http://itknowledgeexchange.techtarget.com/virtualization-pro/

Blue Gears Blogs - http://www.itworld.com/ and http://www.networkworld.com/community/haletky

As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization

[SuryaVMware]

This dosen't look like an error msg. It's only a SCSI sense code. Perhaps i could tell you more if you post a bit more log around that message.

-Surya

[mdangel1]

Thank you for your reply, let me tell you the full story

1. i noticed that one of my ESX hosts was disconnected from VC, it was pingable but disconnected. the vm's on the host were down.

2. i believe HA did not kick in because the host did not completely go down.

3. the vmmessage file stated that a manual cutover was initiated from hba 1 to hba 2. I don't see any red lights on my hba so i don't think the HBA i bad.

[SuryaVMware]

There is a possibility that your LUNs have faildover to the standby processor. That still dont explain why the VM's were taken down though.

Can you give me more details like How many ESX servers? What storage? and the current status of the Server in HA cluster?

-Surya

[mdangel1]

Thank you for your replu,

4 esx 3.02 cluster

IBM SAN DS8300

the ESX server that had the issue is presently in maintenance mode. i am a bit owrried to bring it back to production.

[mdangel1]

this is the vmkwarning file output

# cd /var/log

# more vmkwarning

Nov 19 17:26:47 uspmitx0010014 vmkernel: 0:00:00:53.100 cpu12:1050)WARNING: SCSI

: 1785: Manual switchover to path vmhba2:0:1 begins.

Nov 19 17:26:47 uspmitx0010014 vmkernel: 0:00:00:53.100 cpu12:1050)WARNING: SCSI

: 1110: Did not switchover to vmhba2:0:1. Check Unit Ready Command returned READ

Y instead of NOT READY for standby controller .

Nov 19 17:26:47 uspmitx0010014 vmkernel: 0:00:00:53.100 cpu12:1050)WARNING: SCSI

: 1820: Manual switchover to vmhba2:0:1 completed successfully.

Nov 19 17:26:47 uspmitx0010014 vmkernel: 0:00:00:53.619 cpu12:1050)WARNING: SCSI

: 1785: Manual switchover to path vmhba2:0:2 begins.

Nov 19 17:26:47 uspmitx0010014 vmkernel: 0:00:00:53.619 cpu12:1050)WARNING: SCSI

: 1110: Did not switchover to vmhba2:0:2. Check Unit Ready Command returned READ

Y instead of NOT READY for standby controller .

Nov 19 17:26:47 uspmitx0010014 vmkernel: 0:00:00:53.619 cpu12:1050)WARNING: SCSI

: 1820: Manual switchover to vmhba2:0:2 completed successfully.

Nov 19 17:26:47 uspmitx0010014 vmkernel: 0:00:00:54.159 cpu12:1050)WARNING: SCSI

: 1785: Manual switchover to path vmhba2:0:5 begins.

Nov 19 17:26:47 uspmitx0010014 vmkernel: 0:00:00:54.160 cpu12:1050)WARNING: SCSI

: 1110: Did not switchover to vmhba2:0:5. Check Unit Ready Command returned READ

Y instead of NOT READY for standby controller .

Nov 19 17:26:47 uspmitx0010014 vmkernel: 0:00:00:54.160 cpu12:1050)WARNING: SCSI

: 1820: Manual switchover to vmhba2:0:5 completed successfully.

Nov 19 17:26:47 uspmitx0010014 vmkernel: 0:00:00:54.680 cpu12:1050)WARNING: SCSI

More(45%)

[mikepodoherty]

The log shows repeated messages of Check Unit Command returned READY instead of not ready for standby controller? Are you set to active/active or active passive? Do you have your HBAs set for fixed or most recently used?

I'd check with your SAN folks to make sure you have the ESX hosts configured to IBMs preferred settings.

[mdangel1]

hello,

it's active/active HBA's set to most recently used.

[mikepodoherty]

I'd doublecheck with the IBM SAN team as the error indicates that active/active is not expected. Otherwise, there should be no error reported when the SAN responds with the READY command. I konw on the DS4300 that I've got the SAN set to active/passive.

[SuryaVMware]

Active/Active should always use Fixed as the Multipathing policy. However, if there is a failover occuered it will still use the stand-by path to access the LUNs.

Having said that something has triggered a failover of SAN LUNs 1,2 and 5 in your environment on 19th. However, the message you have posted originally in the thread was not an error only a SCSI sense code when you re-scan a HBA or the LUNs are being detected as a result of failover.

Hope this information is useful. If you need a good review of your environment you can put some diagram of your environment and with all possible information like the Multipathing settings, vLANs and any thing else you can think of.

-Surya