Datastore no longer available

[ylafont22]

Good afternoon. Apologies if this I am posting in an incorrect forum (was not really sure of the best place, please move is it incorrect).   I am in middle of a pickle which I need to troubleshoot as I am not an ESXI expert.   Today someone noticed  an issue where a datastore is report 0 size, this has been up for months.   The host now  reports machine are invalid.    

I figured I reboot, but do not think it would solve the issue,   can anyone provide any steps to take and resolve this issue? Find what the problem is?   

The volumen is list with the command esxcli storage filesystem list

and this is how there appear on the host

Just  update - i did rescan of the datastore, and new "grey out" storage has appeared with a warning on kernel.log

21-05-27T00:23:35.908Z cpu21:2097708)WARNING: NMP: vmk_NmpSatpIssueTUR:1074: Device naa.6141877033256b002644af641f601b56 path vmhba2:C2:T0:L0 has been unmapped from the array

is my datastore gone?????   can i create a new on with UID and find my files?

thank you in advance - assistance is vey much appreciated. 

[ylafont22]

How can two drive fail at the same time without warning!  I have never seen that in all my time.   Crossing  me fingers  on recovery.   I am attaching log for review and will order drives as soon as you say the word.

[e_espinel · Accepted solution]

Hello.
The failure of two disks in a RAID 5 is more common than you think.
Attached is the event log analysis and recovery procedure.

I have done my good deed for the day, best of luck.

[ylafont22]

Frist, thank you for all the assistance in demystifying what has occurred, I would have never found the files to read the logs for SCSI card. As well as having t dedication and professionalism in providing guidance!  much a apreciaste. 

Drives ordered, however I am doubtful of any success since the reports says the drive failed on two different days.   Had the dates been the same i would be more optimistic.  

We never saw a light indicating the drives had failed or seen a warning form esxi, is there something that needs to be installed or configured for this to in place?  how are the devices monitored form the ESXI server?

lets us pray!

[e_espinel]

Hello
Personally I prefer the physical server to send alerts about problems.
This function of sending alerts via SMTP is very common in the service devices (IMM, ILO, IDRAC, etc.) of most computers.

I would understand that your equipment is DELL, so I attach the following link.
https://techstat.net/how-to-setup-simple-smtp-alerts-for-dells-idrac/

Another option is from vCenter Server, I attach link on this
https://digitalthoughtdisruption.com/2020/03/19/setup-email-alert-for-alarms-in-vcenter-6-7-u3/

Finally it is always advisable to have a management network with the service devices (IMM, ILO, IDARC ... etc.) of the equipment. This network allows the verification and monitoring of the equipment, including remote access in many cases and of course the sending of alerts.

.

[ylafont22]

Actually, it is supermicro motherboard with the dell SCSI 

let me get through this ordeal,  Drives wont be in for another 4 days or so with the holidays and I may have some time to investigarte your recomendatorios. 

I will post back with  with results!

again i want to you for you knowledge and oversight on this. Really, really, appreciated.

[ylafont22]

Just want to double check there is nothing left to try, even though the drive comes back up.   no data though

 Thank you again for the assistance and guidance. 

[ylafont22]

!SUCCESS! - I don't believe it!  

The report provided showed the first drive failed over an year ago with a recent secondary drive failure,  I fully expected to rebuild the raid after both drives were reported as failed and no data returned when the second drive as re-enabled.    I had also read somewhere that a drive should only be swapped while the machine is on. 

The drive what swapped out while the machine was on.

as soon I accessed the  RAID utility noticed it was rebuilding, since i had already lost all hope, i figured there was no hard in letting it complete the process.   14 hours later, I rebooted  and found the data had been restored.   I can't believe it. 

e_espinel, I could not have done this without you!   if you would not have alerted me to enable the second drive failure,  this would have not turned out this way. Thank you again!

[e_espinel]

Hello.
For a technician it is always nice to receive good news.
You always have to have faith and keep hope.

Do not forget to activate the alerts, either from the server or in the vcenter server.