1 Reply Latest reply on Feb 13, 2020 7:31 AM by marcelfn

    ScsiDeviceIO: 3449 and HppResetdeviceLogThrottling:631

    marcelfn Lurker

      HI!


      I was using ESXI 6.5 (and after my problem, was upgraded to 6.7U2) with Intel NVME SSD DC P3520 Series, and when I started a Virtual Machine with Windows Server (with vmware tools and paravirtual scsi controller), in vmkernel.log I received alot of messages like this:

      2020-02-13T12:23:18.666Z cpu3:2097600)ScsiDeviceIO: 3449: Cmd(0x459a40d29440) 0x93, CmdSN 0x56f from world 2097305 to dev "t10.NVMe____INTEL_SSDPE2MX450G7_____________________CVPF6334002A450RGN__00000001" failed H:0x0 D:0x2 P:0x0 Valid sense data:

      2020-02-13T12:23:18.666Z cpu3:2097600)0x5 0x0 0x0.

      2020-02-13T12:23:18.673Z cpu3:2097600)ScsiDeviceIO: 3449: Cmd(0x459a40d2a700) 0x93, CmdSN 0x571 from world 2097305 to dev "t10.NVMe____INTEL_SSDPE2MX450G7_____________________CVPF6334002A450RGN__00000001" failed H:0x0 D:0x2 P:0x0 Valid sense data:

      2020-02-13T12:23:18.673Z cpu3:2097600)0x5 0x0 0x0.

      2020-02-13T12:23:18.677Z cpu3:2097600)ScsiDeviceIO: 3449: Cmd(0x459a40cbbd00) 0x93, CmdSN 0x573 from world 2097305 to dev "t10.NVMe____INTEL_SSDPE2MX450G7_____________________CVPF6334002A450RGN__00000001" failed H:0x0 D:0x2 P:0x0 Valid sense data:

      2020-02-13T12:23:18.677Z cpu3:2097600)0x5 0x0 0x0.

      2020-02-13T12:24:03.442Z cpu0:2097643)HPP: HppResetDeviceLogThrottling:631: last error status from device t10.NVMe____INTEL_SSDPE2MX450G7_____________________CVPF6334002A450RGN__00000001 repeated4 times

       

       

      I was upgrade firmware and install intel specifc vib (https://www.intel.com/content/dam/support/us/en/documents/memory-and-storage/ssd-software/Intel_VMD_NVMe_VMWare_User_Gui… ), I was check the Sense codes (https://www.virten.net/vmware/esxi-scsi-sense-code-decoder/?host=0&device=2&plugin=0&sensekey=5&asc=0&ascq=0&opcode=93 ), and I was follow the HPP best pratice (VMware High Performance Plug-In ), but I can´t solve this problem.

       

      And after some time, start this error in same log:

       

      2020-02-13T13:09:09.893Z cpu4:2097183)HPP: HppThrottleLogForDevice:507: Error status H:0xc D:0x0 P:0x0 Invalid sense data: 0x0 0x0 0x0. from device t10.NVMe____INTEL_SSDPE2MX450G7_____________________CVPF6334002A450RGN__00000001 repeated 1280 times

      2020-02-13T13:09:09.896Z cpu4:2097690)0000:03:00.0 nvme_ScsiCommand Failed Dsm request

      2020-02-13T13:09:09.896Z cpu4:2097690)0000:03:00.0 nvme_ScsiCommand Failed Dsm request

      2020-02-13T13:09:09.899Z cpu4:2097690)0000:03:00.0 nvme_ScsiCommand Failed Dsm request

      2020-02-13T13:09:09.899Z cpu4:2097690)0000:03:00.0 nvme_ScsiCommand Failed Dsm request

      2020-02-13T13:09:09.902Z cpu4:2097690)0000:03:00.0 nvme_ScsiCommand Failed Dsm request

      2020-02-13T13:09:09.902Z cpu4:2097690)0000:03:00.0 nvme_ScsiCommand Failed Dsm request

      2020-02-13T13:09:09.907Z cpu4:2097690)0000:03:00.0 nvme_ScsiCommand Failed Dsm request

      2020-02-13T13:09:09.907Z cpu4:2097690)0000:03:00.0 nvme_ScsiCommand Failed Dsm request

      and, after start this error, I stop all vmware, put in maintence mode and this error don´t stop. I need restart esxi to stop.

       

      can Somebody help?

       

      Tks for everthing!

        • 1. Re: ScsiDeviceIO: 3449 and HppResetdeviceLogThrottling:631
          marcelfn Lurker

          I don´t understand if I found solution, but, I found this articles:

           

          VMware Knowledge Base

          VMware Knowledge Base

          VMware Knowledge Base

          VMware Knowledge Base

           

          And, after run this commands:

           

          List the support to vaai:

          esxcli storage core device vaai status get

             VAAI Plugin Name:

             ATS Status: unsupported

             Clone Status: unsupported

             Zero Status: supported

             Delete Status: supported

           

          Disable if unsupported:

          ATS Status:

          esxcli system settings advanced set --int-value 0 --option /VMFS3/HardwareAcceleratedLocking

             Clone Status:

          esxcli system settings advanced set --int-value 0 --option /DataMover/HardwareAcceleratedMove 

             Zero Status:

          esxcli system settings advanced set --int-value 0 --option /DataMover/HardwareAcceleratedInit

           

          Stop that messages in vmkernel.log, but I don´t know if this is the best solution.

           

          If somebody can me help, I´ll gratefull