VMware

This Question is Possibly Answered

1 "correct" answer available (10 pts) 2 "helpful" answers available (6 pts)
7 Replies Last post: Mar 25, 2009 3:20 AM by Flan5ter  

Kernel Panic: VFS: Unable to mount root fs posted: Jul 17, 2007 3:14 AM

Click to view dclark's profile Hot Shot
dclark
109 posts since
Oct 29, 2004
Hello

Running ESX3.0.1 patched with 1006511 and 2066306. Patches have been installed for approx 3 months. Rebooted server on Sunday and got error;

Kernel Panic: VFS: Unable to mount root fs

In the end I booted into troubleshooting mode and ran esxcfg-boot -b which fixed the problem.

The worry I have is that these servers are at a remote site, and if we need to reboot remotely we won't know if the problem is going to re-occur until it is too late, and then I have a 150 mile drive to the remote site to fix.

It looks as if a few others have had the problem, but does anyone know why it occurs and is there a proper fix/patch available?

Many thanks

Re: Kernel Panic: VFS: Unable to mount root fs

1. Jul 17, 2007 3:46 PM in response to: dclark
Click to view acmcnick's profile Hot Shot
acmcnick
94 posts since
Aug 1, 2005
I had a similar issue but it didn't happen on a reboot, it happened during production.

It was a bug in the PERC5/i firmware and it cause the logical disk to become "un-attached" for lack of a better word. We updated the firmware and all was well.

A couple of things:

1. Make sure to review all patches for possible bugs
2. Make sure you actually need to apply the patches
3. Make sure your company lets you buy iLO for at least your remote servers if not all of your servers to avoid a drive. (we have sites between 300-1000 miles away)
4. Don't just look for VMWare Patches, Hardware Vendor patches for Disk Controllers have been know to fix major problems and cause major problems.

Hope this helps.

Re: Kernel Panic: VFS: Unable to mount root fs

2. Jul 17, 2007 8:16 PM in response to: acmcnick
Click to view Schorschi's profile Expert
Schorschi
743 posts since
Jul 6, 2005
Can you list what firmware version the 5/i was at? And what version you updated to?

We had almost the EXACT same issue with PERC 4e DI controller running 5.22A, and we had to back-rev to 5.21X to get stable again. We lost 2 servers out of about of about 30 or so. They came from 5.22A or where flashed to 5.22A before ESX 3.0.1 was installed (all patches up to 051807).

Both VMware and Dell are STILL trying to figure out what happened! We have yet to see any issue with PERC 5/i on our newer servers, but this thread scares the you-know-what out of you-know-where, it sounds so close to our PERC 4 issues, it is scary!

Re: Kernel Panic: VFS: Unable to mount root fs

3. Jul 18, 2007 9:36 AM in response to: Schorschi
Click to view acmcnick's profile Hot Shot
acmcnick
94 posts since
Aug 1, 2005
http://support.dell.com/support/downloads/download.aspx?c=us&l=en&s=gen&releaseid=R149666&SystemID=PWE_2950&servicetag=CD7SLB1&os=WNET&osl=en&deviceid=9182&devlib=0&typecnt=0&vercnt=3&catid=-1&impid=-1&formatcnt=6&libid=46&fileid=199452

2. If all physical disks making up a virtual disk are removed from the system or
enclosure, the RAID controller will delete the virtual disk. If the disks are
later re-inserted, they will be marked foreign and can be imported using a
management application.

I believe we upgraded from firmware 1.02.10 to 1.03.10-0216 which is in the link above.

Re: Kernel Panic: VFS: Unable to mount root fs

4. Jul 18, 2007 9:45 AM in response to: acmcnick
Click to view acmcnick's profile Hot Shot
acmcnick
94 posts since
Aug 1, 2005
R149666

Description:
Dell PowerEdge RAID Controller 5/I Integrated Firmware Release

Package Version: 5.1.1-0040

Re: Kernel Panic: VFS: Unable to mount root fs

5. Jul 18, 2007 10:09 AM in response to: acmcnick
Click to view Texiwill's profile Guru
Texiwill
10,205 posts since
Jan 13, 2004
Hello,

This implies that the boot volume of the SC is no longer accessible. You only have a few solutions.

One is to use 'rescue' media or boot from the TroubleShooting mode. If the trouble shooting mode works then there is a problem with the initrd for the ESX Server and you will need to recreate using esxcfg-boot.

If you can not boot into Trouble shooting mode then you need the rescue media and understand how to mount linux partitions, use fsck, and chroot, and rebuild the master boot record.

I would try troubleshooting mode first.

Best regards,
Edward

Re: Kernel Panic: VFS: Unable to mount root fs

6. Aug 11, 2007 6:58 AM in response to: Texiwill
Click to view Schorschi's profile Expert
Schorschi
743 posts since
Jul 6, 2005
Dell and VMware have not figured out the original issue, but after some research it appears to be related to an older Linux boot issue, what is mucked up when kernel changes are made or grub gets confused. However, neither of these related issues makes sense per se in our situation since we did no kernel change nor any grub configuration changes when we experienced the issue. The symptom issue of the back-rev or forward-rev which I referenced before, which now appears unrelated to the original problem, Dell has documented and VMware has published at the following link, be careful, this technote applies to ESX 2.5.x and 3.0.x even though the title and products affected notes state ESX 3.0.x only... Dell said they are correcting this soon.

http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&externalId=1001577&sliceId=2&docTypeID=DT_KB_1_1&dialogID=16561809&stateId=0%200%2016563204

Message was edited by:
Schorschi

Re: Kernel Panic: VFS: Unable to mount root fs

7. Mar 25, 2009 3:21 AM in response to: Texiwill
Click to view Flan5ter's profile Novice
Flan5ter
5 posts since
Jul 30, 2007
Edward,

You Star. I had the following message on boot and it would go no further. "Kernel Panic: VFS: Unable to mount root fs on 00:00"

I tried troubleshooting mode that told me I had an unclean shutdown and offered a repair of volumes which I did. This did not solve the problem. But led me to believe that the boot was corrupt.

I then went into Debug mode and ran the following command

esxcfg-boot -g -b -p -r

This fixed it.

Regards

Andy

Actions

VMware Developer

SDKs, APIs, Videos, Learn and much more in the Developer community.

Learn More

Developer Sample Code

Increase your developer productivity with VMware API sample code.

Learn More

VMworld Sessions & Labs

Online access to the latest VMworld Sessions & Labs and online services.

Learn more

Purchase PSO Credits Online

Purchase credits to redeem training and consulting services online.

Buy Now

Community Hardware Software

View reported configurations or report your own.

Learn More

VMware vSphere

Come witness the next giant leap in virtualization.

Register Today

Communities