xxxxx.core files filling up /var/core

[mike-p]

our esx3.5u1 server showed a 100% full partition for "/".

in /var/core we found a bunch of 27M sized files like xxxxx.core

1. i want to find out which vm is responsible for this

2. is this a correct behaviour ?

3. do i have to change my partitioning scheme or is there way to redirect the files ?

[petedr]

I believe they are vmkernel dump files. You may need to end one to VMware support to determine why they were created/

I did find one post saying to try the file command against one to see

http://communities.vmware.com/message/450027

[Texiwill]

Hello,

our esx3.5u1 server showed a 100% full partition for "/".

in /var/core we found a bunch of 27M sized files like xxxxx.core

1. i want to find out which vm is responsible for this

These are not necessarily VM files but SC services that are core dumping. To find out to which programs these core files belong run the command....

file xxxxx.core

2. is this a correct behaviour ?

Yes, however you should track down what is causing the coredumps.

3. do i have to change my partitioning scheme or is there way to redirect the files ?

I would change your partitioning scheme. You can redirect but mainly you would want to redirect using a hardlink to another partition. I use the following scheme:

/boot -> 200MB

/ -> 5GB

/var -> 5GB

/tmp -> 5GB

/var/log -> 5GB

Yes /var/log is its own beast. Sometimes I add /home depending on how the admins will use the system. Generally it is not needed.

Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education.

CIO Virtualization Blog: http://www.cio.com/blog/index/topic/168354

As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization

[mike-p]

file xxxx.core shows that the files are created from 'vmware-hostd'

[madda]

Have you checked the log files for error messages regarding hostd?

[Texiwill]

Hello,

file xxxx.core shows that the files are created from 'vmware-hostd'

This implies that vmware-hostd is consistently crashing. You should look at the /var/log/vmware/hostd.log files to determine the cause of this failure. Generally I have found that an XML configuration file is corrupt when this happens. If you run 'service mgmt-vmware restart' while running 'tail -f /var/log/vmware/hostd.log' in another window/screen you should be able to see if there are any XML inconsistencies and fix them.

It is safe to remove the xxxxx.core files.

Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education.

CIO Virtualization Blog: http://www.cio.com/blog/index/topic/168354

As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization

[kcollo]

I have had a few problems with hostd filling up the /var filesystem as well. Now I have started breaking /var/core out into its own filesystem as not to lock up any other services dependent on /var being read write. Below is a link to some more details.

[jmcdonald1]

I have seen a number of cases with hostd cores filling up the drive. If you check the hostd.log file on the system and you manage to catch the initial crash (ie a crash that does not indicate that the drive is full) you will will see the real error that caused the crash to occur. If you see the following error in the logs:

Caught signal 11

Exception: Assert Failed: "!_mutex.IsLocked()" @/build/mts/release/bora-84374/bora/vim/lib/vmacore/system/MutexImpl.cpp:84

Backtrace:

eip 0x11ac8de

eip 0x1070139

Upgrading to ESX 3.5 U3 will stop the repeated cores from occuring.

Cheers,

/Jon