4 Replies Latest reply on Aug 12, 2019 2:35 AM by GeertCRW

    vpxd service crashes due to memory usage and restarts (VCSA 6.7)

    lulu62 Novice

      Hello!

      Our vCenter Server Appliance crashes regularly. Something in the vpxd service is eating up the memory quite fast and at some point vpxd decides to restart:

      vcsa_crash_issue_1.png

      In vpxd-xxx.log file:

      2019-06-19T11:35:30.764+02:00 error vpxd[15037] [Originator@6876 sub=Memory checker] Current value 7868284 exceeds hard limit 7854080. Shutting down process.

      2019-06-19T11:35:30.827+02:00 panic vpxd[15037] [Originator@6876 sub=Default]

      -->

      --> Panic: Memory exceeds hard limit. Panic

      --> Backtrace:

      --> [backtrace begin] product: VMware VirtualCenter, version: 6.7.0, build: build-13639324, tag: vpxd, cpu: x86_64, os: linux, buildType: release

      --> backtrace[00] libvmacore.so[0x002AF3B8]: Vmacore::System::Stacktrace::CaptureFullWork(unsigned int)

      --> backtrace[01] libvmacore.so[0x001B2154]: Vmacore::System::SystemFactory::CreateBacktrace(Vmacore::Ref<Vmacore::System::Backtrace>&)

      --> backtrace[02] libvmacore.so[0x002AD58E]

      --> backtrace[03] libvmacore.so[0x002AD66E]: Vmacore::PanicExit(char const*)

      --> backtrace[04] libvmacore.so[0x001B5D72]: Vmacore::System::ResourceChecker::DoCheck()

      --> backtrace[05] libvmacore.so[0x0027E355]

      --> backtrace[06] libvmacore.so[0x00236231]

      --> backtrace[07] libvmacore.so[0x002364FA]

      --> backtrace[08] libvmacore.so[0x002B495D]

      --> backtrace[09] libpthread.so.0[0x000073D4]

      --> backtrace[10] libc.so.6[0x000E8EBD]

      --> [backtrace end]

       

      I opened a case at VMware Support but it's been more than 1.5 month and we can't find the root cause...

       

      Another thing I noticed which might be linked:

      'Last day' space usage statistics are missing for both VMs and datastores:

      vcsa_crash_issue_2.png

      vcsa_crash_issue_3.png

       

      These missing stats will eventually get loaded and displayed when vpxd crashes or when I restart vpxd manually...

      I'm still following up my support case at VMware but I'm losing motivation.

      Can someone here help me?