VMware Cloud Community
fcturner
Contributor
Contributor

Warning Event: VFAT: 2600: Possible FAT corruption!

Hi All,

On Monday I installed ESX Server 3i 3.5.0 build 103909 on a Dell PowerEdge 2950. I've loaded several VMs and stressed the system over the last couple of days. I just noticed two warnings in the event log that occurred yesterday afternoon and just now:

VFAT: 2600: Possible FAT corruption!: offset not found: 0x400, length 8704, seekOffset 0x400. File length: 9508, status: Limit excee (2:19:08:01.420 cpu6:833625)

VFAT: 2600: Possible FAT corruption!: offset not found: 0x400, length 8704, seekOffset 0x400. File length: 9506, status: Limit excee (1:23:08:01.512 cpu1:584272)

Any idea what these mean or how to diagnose them further? I searched KB and communities.

I have two raid volumes one raid 0 for boot/swap and one raid 5 for VMs. The datastore on volume 0 is not being used, just the raid 5.

Thanks,

Reply
0 Kudos
68 Replies
great_scandinav
Contributor
Contributor

I think VMware has a bug that they need to go fix and that you therefore would just waste your time following the suggested procedure.

Reply
0 Kudos
3sh
Contributor
Contributor

This error just showed up on one of my ESXi systems. 3.5 U3. Dell r805, installed on the internal SD card. Other 2 hosts are not showing this (yet).

Reply
0 Kudos
Hover8
Contributor
Contributor

Same error here (3 times) with ESX3.5 u3 143129 on HP ML110 G5 at time 06:01:01, 21:01:01 and 01:01:01.

Reply
0 Kudos
ottoman4566
Contributor
Contributor

Hi,

does anyone know what it is that kicks off the configuration backup at 1 min past the hour when this error appears ? Can it be changed and what's the risk of not running this ?

Thanks

Reply
0 Kudos
Dave_Mishchenko
Immortal
Immortal

The file is located here - /var/spool/cron/crontabs/root. If you don't let the backup run then any configuration changes made won't be available after you reboot. For example, if you add a VM you'll be able to run it but after a reboot the VM won't be in inventory (although the VM's files would still be on disk).

Reply
0 Kudos
zaphod777
Contributor
Contributor

Hi,

currently got no error on our Dell PE 2950 since two weeks.

Neither changed anything on the ESXi host nor removed any disks.

Any news on that topic regarding the cause of that warning in the events?

Reply
0 Kudos
mkmorris
Contributor
Contributor

zaphod: it will keep happening.

Recently for me 23/02/2009 2:01:01pm, 13/02/2009 10:01:01am, 11/02/2009 1:01:01PM, 09/02/2009 4:01:03pm, 25/01/2009 9:01:01pm, 18/01/2009 11:01:01pm, then all the way back to 04/01/2009 12:01:01am.

it can be the same day, days between, or weeks between without error. then you'll get one. don't worry, you will see more.

Reply
0 Kudos
Maddison1975
Contributor
Contributor

Just for the record i am also getting this error at the same time as everyone else.

Maddy

Reply
0 Kudos
Wessa
Contributor
Contributor

Hi all

Suddenly after 2009-01-31 this error disappeared and i´ve not changed anything,and not rebooted.

can someone explain that?

has anybody else seen this behavior ?

Reply
0 Kudos
zaphod777
Contributor
Contributor

Don't want to disappoint you - the error will pop up again. Yesterday I got it after 6 weeks of normal operation... Smiley Sad

Can anyone tell the reason for this issue?

Reply
0 Kudos
mkmorris
Contributor
Contributor

Reply
0 Kudos
great_scandinav
Contributor
Contributor

As said I have a ticket #1153427441 open on this subject and VMware just got back to me today and said the following:

"I checked again and no updates on a timetable for a fix to the issue you reported."

So ESXi users are kind of stuck on the issue.

Reply
0 Kudos
zaphod777
Contributor
Contributor

VMWare shouldn't wait to long to solve this issue - we'll check out Citrix XenServer which will be free starting in April.

Citrix Link

Reply
0 Kudos
admin
Immortal
Immortal

Gentlemen:

This error message is benign.

I've been running ESXi without problems since it was beta code.

Don't panic.

/mark

Reply
0 Kudos
HectorH
Contributor
Contributor

I believe the error is quite dangerous myself. mbrunstad, you should try rebooting your ESXi server within an hour of the error happening and then tell us what happens to your server configuration .... Myself, after I make any changes to the server configuration (including adding/deleting VMs) I execute the backup script and then I make an offline copy of the backup file set. Then I can sleep well at night.

If making a backup of those files was not important, then why would VMWare script it to happen every hour? ... I'm pretty sure it's something to worry about.

If VMWare does not seem to care about an issue reported by someone with a support contract, then would somebody please tell me why I would want to pay for a support contract?

Reply
0 Kudos
admin
Immortal
Immortal

Hi HectorH:

I did as you suggested and rebooted each of my ESXi servers within an hour of the error message, actually within about 10 minutes.

I've done this repeatedly and have not lost my ESXi configuration settings, VM configs, access to VMs, or anything else.

I did notice that the reboot process took longer than usual after an error of this type.

I'm not saying this shouldn't be addressed and fixed! I'm just observing that for me, in my context it's been benign.

/mark

Reply
0 Kudos
mazday
Expert
Expert

Hi all, I am poor in English... sorry.

I have this error in my rsyslog server from one of the ESXi host (build130775 on intel S3000AH and SATA drives). Error occure once (yestеrday), after running auto-backup.sh (every hour running, by rsyslog ststistics).

I understand, the script saves the current settings of ESXi for load them, then boot next time?

Question! Therefore I can lose the settings (or more), only if the last launch of auto-backup.sh generate error: "ALERT: VFAT: 2583: Possible FAT corruption!......."?

If next-hour running of script auto-backup.sh was successful (and no occur error: "ALERT: VFAT: 2583: Possible FAT corruption!.......") - no threat of data loss?

Reply
0 Kudos
Dave_Mishchenko
Immortal
Immortal

When you shutdown / reboot the host ESXi will also take a system backup then, so that save would need to be successful.

Have you tried running dosfsck on your system - http://www.vm-help.com/esx/esx3i/check_system_partitions.php.

Reply
0 Kudos
kfoley
Contributor
Contributor

I have a Dell M1000e with 12 of 16 blade slots populated.

Each blade is running ESX Server 3i, 3.5.0, 123529

All 12 blades show this sporatic event:

VFAT: 2583: Possible FAT corruption!: offset not found: 0x400, length 7680, seekOffset 0x400. File length: 8573: status:.

The "File length" value varies by bytes and does not appear to be getting larger or smaller with each error.

Pattern is that it does occur at 1 minute 1 second after "the hour" . Where "the hour" can be any hour, seems to be on even numbered hours.

All blades were installed from CD built from a the product download on the VMWare site.

Reply
0 Kudos
Picche
Contributor
Contributor

I have the same problem on a Fujitsu server with local storage,

after upgrade to ESXi 3.5 Build 163429, it seems that now everything is ok.

Reply
0 Kudos