About a year ago I upgraded my ESXi 5.5 U1 installation to VMware ESXi 5.5.0 build-1881737 to fix problems with Intermittent NFS APDs as per VMware KB: Intermittent NFS APDs on VMware ESXi 5.5 U1
The system has been running ever since without any problems. Last weekend I upgraded to latest ESXi 5.5.0 build-2718055 and today the Intermittent NFS APDs errors were back again. However, this time, the vms continued to work when on ESXi 5.5 U1 they would immediately crash.
In their KB article VMware states that the issue was fixed as of build 1881737. I am curious why am I still seeing this issue on later builds?
I can provide detailed logs on request.
/edit: I just analyzed our firewall logs from the same period and pfSense started to drop fragmented packets shortly befor the Intermittent NFS APDs started. Not sure whether ESXi or the NFS server is at fault here. For now I will try to mitigate the issue by configuring pfSense so that it clears invalid DF bits instead of dropping the packets.
Please lunch the command: esxcli software vib list and look if the kb is installed. Or watch it on the GUI of the vSphere Client.
thanks for the reply. It does not seem to be installed! I thought that the latest patch level would always include critical patch history. Can I install just this patch without reverting back the security patches etc. that are part of the current build?
Name Version Vendor Acceptance Level Install Date
----------------------------- ------------------------------------- ------ ---------------- ------------
ata-pata-amd 0.3.10-3vmw.5184.108.40.2061820 VMware VMwareCertified 2013-10-04
ata-pata-atiixp 0.4.6-4vmw.5220.127.116.111820 VMware VMwareCertified 2013-10-04
ata-pata-cmd64x 0.2.5-3vmw.518.104.22.1681820 VMware VMwareCertified 2013-10-04
ata-pata-hpt3x2n 0.3.4-3vmw.522.214.171.1241820 VMware VMwareCertified 2013-10-04
ata-pata-pdc2027x 1.0-3vmw.5126.96.36.1991820 VMware VMwareCertified 2013-10-04
ata-pata-serverworks 0.4.3-3vmw.5188.8.131.521820 VMware VMwareCertified 2013-10-04
ata-pata-sil680 0.4.8-3vmw.5184.108.40.2061820 VMware VMwareCertified 2013-10-04
ata-pata-via 0.3.3-2vmw.5220.127.116.111820 VMware VMwareCertified 2013-10-04
block-cciss 3.6.14-10vmw.518.104.22.1681820 VMware VMwareCertified 2013-10-04
ehci-ehci-hcd 1.0-3vmw.522.214.171.1241820 VMware VMwareCertified 2013-10-04
elxnet 10.0.100.0v-1vmw.5126.96.36.1991820 VMware VMwareCertified 2013-10-04
esx-base 5.5.0-2.62.2718055 VMware VMwareCertified 2015-06-28
esx-dvfilter-generic-fastpath 5.5.0-0.0.1331820 VMware VMwareCertified 2013-10-04
esx-tboot 5.5.0-2.33.2068190 VMware VMwareCertified 2015-02-25
esx-xlibs 5.5.0-0.0.1331820 VMware VMwareCertified 2013-10-04
esx-xserver 5.5.0-0.0.1331820 VMware VMwareCertified 2013-10-04
ima-qla4xxx 2.01.31-1vmw.5188.8.131.521820 VMware VMwareCertified 2013-10-04
ipmi-ipmi-devintf 39.1-4vmw.5184.108.40.2061820 VMware VMwareCertified 2013-10-04
ipmi-ipmi-msghandler 39.1-4vmw.5220.127.116.111820 VMware VMwareCertified 2013-10-04
ipmi-ipmi-si-drv 39.1-4vmw.518.104.22.1681820 VMware VMwareCertified 2013-10-04
lpfc 10.0.100.1-1vmw.522.214.171.1241820 VMware VMwareCertified 2013-10-04
lsi-mr3 0.255.03.01-2vmw.5126.96.36.1996018 VMware VMwareCertified 2014-10-03
lsi-msgpt3 00.255.03.03-1vmw.5188.8.131.523387 VMware VMwareCertified 2014-10-03
misc-cnic-register 1.72.1.v50.1i-1vmw.5184.108.40.2061820 VMware VMwareCertified 2013-10-04
misc-drivers 5.5.0-2.62.2718055 VMware VMwareCertified 2015-06-28
mtip32xx-native 3.3.4-1vmw.5220.127.116.113387 VMware VMwareCertified 2014-10-03
net-be2net 18.104.22.168v-1vmw.522.214.171.1241820 VMware VMwareCertified 2013-10-04
net-bnx2 2.2.3d.v55.2-1vmw.5126.96.36.1991820 VMware VMwareCertified 2013-10-04
net-bnx2x 1.72.56.v55.2-1vmw.5188.8.131.521820 VMware VMwareCertified 2013-10-04
net-cnic 1.72.52.v55.1-1vmw.5184.108.40.2061820 VMware VMwareCertified 2013-10-04
net-e1000 220.127.116.11-3vmw.518.104.22.1681820 VMware VMwareCertified 2013-10-04
net-e1000e 1.1.2-4vmw.522.214.171.1243387 VMware VMwareCertified 2014-10-03
net-enic 126.96.36.199a-1vmw.5188.8.131.521820 VMware VMwareCertified 2013-10-04
net-forcedeth 0.61-2vmw.5184.108.40.2061820 VMware VMwareCertified 2013-10-04
net-igb 220.127.116.11.1-1vmw.518.104.22.1683361 VMware VMwareCertified 2015-06-28
net-ixgbe 22.214.171.124.14iov-12vmw.5126.96.36.19918055 VMware VMwareCertified 2015-06-28
net-mlx4-core 188.8.131.52-1vmw.5184.108.40.2061820 VMware VMwareCertified 2013-10-04
net-mlx4-en 220.127.116.11-1vmw.518.104.22.1681820 VMware VMwareCertified 2013-10-04
net-nx-nic 5.0.621-1vmw.522.214.171.1241820 VMware VMwareCertified 2013-10-04
net-r8168 8.013.00-3vmw.510.0.0.799733 VMware VMwareCertified 2013-10-04
net-r8169 6.011.00-2vmw.510.0.0.799733 VMware VMwareCertified 2013-10-04
net-s2io 126.96.36.19927-3vmw.510.0.0.799733 VMware VMwareCertified 2013-10-04
net-sky2 1.20-2vmw.510.0.0.799733 VMware VMwareCertified 2013-10-04
net-tg3 3.123c.v55.5-1vmw.5188.8.131.528190 VMware VMwareCertified 2015-02-25
net-vmxnet3 184.108.40.206-3vmw.5220.127.116.113827 VMware VMwareCertified 2015-06-28
ohci-usb-ohci 1.0-3vmw.518.104.22.1681820 VMware VMwareCertified 2013-10-04
qlnativefc 22.214.171.124-1vmw.5126.96.36.1991820 VMware VMwareCertified 2013-10-04
rste 2.0.2.0088-4vmw.5188.8.131.523387 VMware VMwareCertified 2014-10-03
sata-ahci 3.0-21vmw.5184.108.40.2063361 VMware VMwareCertified 2015-06-28
sata-ata-piix 2.12-10vmw.5220.127.116.118190 VMware VMwareCertified 2015-02-25
sata-sata-nv 3.5-4vmw.518.104.22.1681820 VMware VMwareCertified 2013-10-04
sata-sata-promise 2.12-3vmw.522.214.171.1241820 VMware VMwareCertified 2013-10-04
sata-sata-sil24 1.1-1vmw.5126.96.36.1991820 VMware VMwareCertified 2013-10-04
sata-sata-sil 2.3-4vmw.5188.8.131.521820 VMware VMwareCertified 2013-10-04
sata-sata-svw 2.3-3vmw.5184.108.40.2061820 VMware VMwareCertified 2013-10-04
scsi-aacraid 220.127.116.11-9vmw.518.104.22.1681820 VMware VMwareCertified 2013-10-04
scsi-adp94xx 22.214.171.124-6vmw.5126.96.36.1991820 VMware VMwareCertified 2013-10-04
scsi-aic79xx 3.1-5vmw.5188.8.131.521820 VMware VMwareCertified 2013-10-04
scsi-bnx2fc 1.72.53.v55.1-1vmw.5184.108.40.2061820 VMware VMwareCertified 2013-10-04
scsi-bnx2i 2.72.11.v55.4-1vmw.5220.127.116.111820 VMware VMwareCertified 2013-10-04
scsi-fnic 18.104.22.168-1vmw.522.214.171.1241820 VMware VMwareCertified 2013-10-04
scsi-hpsa 5.5.0-44vmw.5126.96.36.1991820 VMware VMwareCertified 2013-10-04
scsi-ips 7.12.05-4vmw.5188.8.131.521820 VMware VMwareCertified 2013-10-04
scsi-lpfc820 184.108.40.206-129vmw.5220.127.116.111820 VMware VMwareCertified 2013-10-04
scsi-megaraid-mbox 18.104.22.168-6vmw.522.214.171.1241820 VMware VMwareCertified 2013-10-04
scsi-megaraid-sas 5.34-9vmw.5126.96.36.1998190 VMware VMwareCertified 2015-02-25
scsi-megaraid2 2.00.4-9vmw.5188.8.131.521820 VMware VMwareCertified 2013-10-04
scsi-mpt2sas 14.00.00.00-3vmw.5184.108.40.2063387 VMware VMwareCertified 2014-10-03
scsi-mptsas 4.23.01.00-9vmw.5220.127.116.111820 VMware VMwareCertified 2013-10-04
scsi-mptspi 4.23.01.00-9vmw.518.104.22.1681820 VMware VMwareCertified 2013-10-04
scsi-qla2xxx 902.k1.1-9vmw.522.214.171.1241820 VMware VMwareCertified 2013-10-04
scsi-qla4xxx 5.01.03.2-6vmw.5126.96.36.1991820 VMware VMwareCertified 2013-10-04
uhci-usb-uhci 1.0-3vmw.5188.8.131.521820 VMware VMwareCertified 2013-10-04
xhci-xhci 1.0-2vmw.5184.108.40.2063827 VMware VMwareCertified 2015-06-28
tools-light 5.5.0-2.62.2718055 VMware VMwareCertified 2015-06-28
Yes you can do from the console of ESXi, or also through the vSphere Update Manager.
From command line:
Search and Download the patch: https://my.vmware.com/group/vmware/patch#search
Copy the patch in your ESXi, for example in the datastorage local or in another datastorage.
Enter Maintenence Mode
Navigate to the directory on the datastore:
Install the Patch:
esxcli software vib install -d "/vmfs/volumes/datastore1/patch-directory/ESXi550-201406001.zip"
Verify that the VIBs are installed on your ESXi host:
esxcli software vib list
After the patch has been installed, reboot the ESX host:
Exit maintenance mode:
If I follow this procedure it will downgrade my build level to ESX build number 1881737. At least this is what it tells me after the reboot.
~ # esxcli software sources profile list -d /vmfs/volumes/esxi_coreos0/ESXi550-201406001.zip
Name Vendor Acceptance Level
------------------------------- ------------ ----------------
ESXi-5.5.0-20140604001-no-tools VMware, Inc. PartnerSupported
ESXi-5.5.0-20140604001-standard VMware, Inc. PartnerSupported
~ # vmware --version
VMware ESXi 5.5.0 build-1881737
So what I am looking for I guess is a way to just get this patch properly installed (VMware KB: VMware ESXi 5.5, Patch ESXi-5.5.0-20140604001-standard) without reverting back to the old build number.
OK, but after downgrade, you're back has passed the VUM to upgrade the last level ??
You have performed the procedure by command line when has downgrade the version, but why not use the VUM to patch it? Because using VUM should not be downgraded.
Please try it with the VUM!
I am running a single ESXi host, no vCenter server and no VUM. That's why.
But after giving this a second thought - I am pretty sure that the patch must be included in the esx-base 5.5.0-2.62.2718055 package. Otherwise this doesn't make any sense. So the reason for it not appearing in the esxcli "software vib list" output above is likely that it is included in the higher build level anyways.
The patch is included in the latest version of ESXi image 5.5U2 but in the ISO.
If the ESXi only, if you had to date with the latest ISO ESXi you have gotten the patch failure NFS and also all other patches until leaving the last level.
And in the case of the VUM upgrade it had been gotten so this patch like everyone else.
so basically what you are saying is that in order for the patch to be included I must upgrade my host with the *.iso rather than using the "
esxcli software vib update" command? Or use VUM with a vCenter service that would automatically upgrade my ESXi host? Can you point me to any official VMware resource where this is documented?