VMware Cloud Community
TCP
Enthusiast
Enthusiast

Empty DataStore. Need Urgent help!

Hi All!

We have a problem with one of our Datastore.

When I try to browse DataStore from service console "ls" i recieve error message "Invalid argument".

When I try to browse DataStore from DataStore browser i see empty DataStore, but i know DataStore is not empty and several VM's working till now from this DataStore, but VM's inaccessible.

fdisk -l

esxcfg-vmhbadevs -m

lists all volumes correctly and problem DataStore too.

System config:

IBM BladeCenter H

HS21XM blades

VC 2.5 u2

12 - ESX 3.5 u2 hosts

IBM DS4400 SAN.

Please Help.

VCP3, VCP4
Reply
0 Kudos
9 Replies
NTurnbull
Expert
Expert

Hi, have you got any errors in your vmkernel logs? Anything changed on the SAN config/zoning?

Thanks, Neil
Reply
0 Kudos
TCP
Enthusiast
Enthusiast

Hi.

Yes I have errors like

Nov 4 08:19:15 esx-005-svr vmkernel: 7:21:23:29.380 cpu4:1043)WARNING: FS3: 4979: Reclaiming timed out heartbeat http://HB state abcdef02 offset 3766272 gen 102 stamp 681807788865 uuid 49058253-551cbe34-6067-001a64760058 jrnl <FB 88119> drv 4.31 failed: Invalid metadata

Nov 4 08:19:15 esx-005-svr vmkernel: 7:21:23:29.743 cpu1:1093)<6>Debug scsi underrun

Nov 4 08:19:15 esx-005-svr vmkernel: 7:21:23:29.743 cpu1:1093)<6>Debug scsi underrun

Nov 4 08:19:53 esx-005-svr vmkernel: 7:21:24:07.835 cpu1:1087)<6>Debug scsi underrun

Nov 4 08:20:15 esx-005-svr vmkernel: 7:21:24:29.749 cpu1:1168)<6>Debug scsi underrun

Nov 4 08:21:11 esx-005-svr vmkernel: 7:21:25:25.123 cpu1:1149)<6>Debug scsi underrun

Nov 4 08:21:11 esx-005-svr vmkernel: 7:21:25:25.123 cpu1:1149)<6>Debug scsi

Nov 4 08:46:34 dho-esx-005-svr vmkernel: 7:21:50:49.001 cpu7:1203)WARNING: Swap: vm 1204: 2268: Failed to open swap file '/volumes/48c8c0a1-2c706ad6-74e3-001a64760048/001-VSVR/001-VSVR-300dc447.vswp': Invalid metadata

Nov 4 08:46:34 dho-esx-005-svr vmkernel: 7:21:50:49.001 cpu7:1203)WARNING: Swap: vm 1204: 3586: Failed to initialize swap file '/volumes/48c8c0a1-2c706ad6-74e3-001a64760048/001-VSVR/001-VSVR-300dc447.vswp': Invalid metadata

Nov 4 08:46:34 dho-esx-005-svr vmkernel: 7:21:50:49.001 cpu7:1203)WARNING: World: vm 1204: 910: init fn swap failed with: No swap file!

Nov 4 08:46:34 dho-esx-005-svr vmkernel: 7:21:50:49.001 cpu7:1203)WARNING: World: vm 1204: 1775: WorldInit failed: trying to cleanup.

VCP3, VCP4
Reply
0 Kudos
polysulfide
Expert
Expert

Did you recently bring a new VCB proxy online? If so, hopefully you remembered to disable automount.

Reply
0 Kudos
TCP
Enthusiast
Enthusiast

No this is not vcb problem.

Powering off of all worked virtual machines solved the probIem.

I can't understand what happened.

VCP3, VCP4
Reply
0 Kudos
NTurnbull
Expert
Expert

TCp thats random as!! Got anything in the messages or kernel logs for anything to do with hba's or SCSI error/info (reservations maybe?) I'm confused

Thanks, Neil
Reply
0 Kudos
mikepodoherty
Expert
Expert

Do you use IBM Storage Manager? IF so, what do the logs in Storage Manger report for the time period when you couldn't see the datastore?

Reply
0 Kudos
Jasemccarty
Immortal
Immortal

Just a heads up... the DS4400 is only on the HCL through ESX 3.0.3. 3.5 isn't supported on DS4400's.

I'm not saying it doesn't work... I have one too. But it isn't supported with the latest builds.

Jase McCarty

http://www.jasemccarty.com

Co-Author of VMware ESX Essentials in the Virtual Data Center

(ISBN:1420070274) from Auerbach

Jase McCarty - @jasemccarty
Reply
0 Kudos
Jasemccarty
Immortal
Immortal

Alright... I'm going to say it.

Apparently DS4400's/FastT700's don't play well with ESX 3.5. VMware support will tell you to roll back to 3.0.3

Update: With a little help from Mike La Spina... 3.5 Update 2 will work, but Update 3, with the Storage Monitor addition will cause you some issues.

VMware won't support ESX 3.5 on a DS4400, but it will work.

Jase McCarty

http://www.jasemccarty.com

Co-Author of VMware ESX Essentials in the Virtual Data Center

(ISBN:1420070274) from Auerbach

Please consider awarding points if this post was helpful or correct

Jase McCarty - @jasemccarty
Reply
0 Kudos
GavinConway
Contributor
Contributor

Problem replicated here as well. Unless all the ESXi hosts within that cluster are running then you can't browse the datastores.

Anyone raising a bug report?

Reply
0 Kudos