Local storage Datastore is missing after a motherboard replacement

[Somafer_SI]

Hello, I’m so sorry because the following mistake concerns an old ESX 5.1

My configuration is 2 x ESX 5.1 HA and a SAN for VMs?

On one ESX, we replaced the motherboard after that ESX is booting well on his local SCSI raid (perhaps there is a raid reconstruction).

In vCenter, we have:

• an alert: “Configuration Issues: No vmkcore disk partition is available and no network coredump server has been configured. Host core dumps cannot be saved. System logs on host *_my_host_* are stored on non-persistent storage”

• In Configuration à Storage: the local disk datastore is missing (no effect, with Rescan-all). (But ESX is running on this raid).

• The partition scheme is anormal:

              # partedUtil getptbl /vmfs/devices/disks/naa.600507605804a8f8193f16c144f45061

               unknown

               17782 255 63 285671424

• In vmkernel.log (see attachement) we have have I/O errors (but I repeat ESXI boot with no problem on itself HDDs) : the raid adapter on vmhba1:C2:T0:L0 mounts well, but after we have I/O errors for reading mbr, partitions, etc.

I read that it’s possible to re-create these partitions, but I read also that this problem can appears when Intel VT-d is enabled in the bios. I read also it can be a problem of volume signature.

Could you help me please?

François

[a_p_]

Is this the datastore that's created on the boot volume, or do you have another local volume?
What does ls -lisa /vmfs/devices/disks return?

André

[scott28tt]

Moderators should move this thread to the area for ESXi now that I’ve reported it.

[Somafer_SI]

Hello André and thank you, (and thank you Scott for the moove)

Yes, the datastore "local-disk" is on the the boot volume (it seems to contain some logs or dumps in somes cases).

You will find the return of the ls command in the attachement.

-> Yellow : 146go raid local

-> green : 1,8 To on SAN.

[a_p_]

Can you please run the following command, and paste its text output to your next reply (please don't use .pdf attachments for this)

offset="128 2048"; for dev in `esxcfg-scsidevs -l | grep "Console Device:" | awk {'print $3'}`; do disk=$dev; echo $disk; partedUtil getptbl $disk; { for i in `echo $offset`; do echo "Checking offset found at $i:"; hexdump -n4 -s $((0x100000+(512*$i))) $disk; hexdump -n4 -s $((0x1300000+(512*$i))) $disk; hexdump -C -n 128 -s $((0x130001d + (512*$i))) $disk; done; } | grep -B 1 -A 5 d00d; echo "---------------------"; done

André

[Somafer_SI]

Hello André,

Here is the return of your test.

The volume naa.600507605804a8f8193f16c144f45061 is the local storage, integrated raid, which running ESX and where is the missing local datastore :

/vmfs/devices/cdrom/mpx.vmhba32:C0:T0:L0
Error: The device /dev/cdrom/mpx.vmhba32:C0:T0:L0 has zero length, and can't possibly store a file system or partition table. Perhaps you selected the wrong device?
Unable to get device /vmfs/devices/cdrom/mpx.vmhba32:C0:T0:L0

---------------------
/vmfs/devices/disks/naa.600507605804a8f8193f16c144f45061
unknown
17782 255 63 285671424
hexdump: /vmfs/devices/disks/naa.600507605804a8f8193f16c144f45061: Input/output error
hexdump: /vmfs/devices/disks/naa.600507605804a8f8193f16c144f45061: Input/output error
hexdump: /vmfs/devices/disks/naa.600507605804a8f8193f16c144f45061: Input/output error
hexdump: /vmfs/devices/disks/naa.600507605804a8f8193f16c144f45061: Input/output error
hexdump: /vmfs/devices/disks/naa.600507605804a8f8193f16c144f45061: Input/output error
hexdump: /vmfs/devices/disks/naa.600507605804a8f8193f16c144f45061: Input/output error
---------------------
/vmfs/devices/disks/naa.60080e5000371c32000001dc51ac1830
unknown
2 255 63 40960
hexdump: /vmfs/devices/disks/naa.60080e5000371c32000001dc51ac1830: Function not implemented
---------------------
/vmfs/devices/disks/naa.60080e5000371c320000025b51babe70
gpt
218445 255 63 3509329920
1 2048 3509329886 AA31E02A400F11DB9590000C2911D1B8 vmfs 0
Checking offset found at 2048:
0200000 d00d c001
0200004
1400000 f15e 2fab
1400004
0140001d 76 69 66 73 74 6f 72 61 67 65 00 00 00 00 00 00 |vifstorage......|
0140002d 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
---------------------

[continuum]

Using ESXi-buildin-tools against partitions that mention i/O errors is hopeless.
It is time to switch to a Linux-LiveCD and evaluate what is left.
Easiest option is probably the UFSexplorer-livecd .... try it
If necessary use the buildin Teamviewer and call me via skype.

Ulli

[Somafer_SI · Accepted solution]

Hello Ulli,

After several hours of google search, I think that this case feet exactly to my case :

https://forums.lenovo.com/t5/System-x-X6-M5-M4/x3650-M4-7915-m5110e-firmware-23-33-0033-incompatible...

It seems that I need to donwgrade the intégrated RAID contoler's firmware.

I analyse it ASAP tomorow.

Thx,

François

[continuum]

Your plan makes sense - if you later need assistance with Linux ... let me know. Good luck

Ulli