VMware Cloud Community
Scissor
Virtuoso
Virtuoso

DQLEN values changing in esxtop while system running?

Should the DQLEN esxtop parameter change during runtime?

I was under the impression that the Device Queue Length (DQLEN) parameter was fixed at boot time. However, I have observed on my test whitebox ESXi 4 server that the DQLEN parameter seems to switch between 32 and 128 when viewed using esxtop.

My server has a Dell Perc5/E card set up with 2 local SATA disks in RAID0. I have only one large 690 GB VMFS volume on this RAID.

viewing /var/log/messages shows that the card is being configured with a "cmd_per_lun" depth setting of 128 (0x80 hex) during boot. (I assume that "cmd_per_lun" is the same as DQLEN)

... <7>megasas: max_sectors : 0x280, cmd_per_lun : 0x80

viewing esxtop and selecting "u" for device statistics shows the DQLEN parameter switching between 32 and 128, seemingly depending on how busy the disks are -- at least it seems I can cause the value to jump to 128 if I run chkdsk inside one of my Windows Guests. See attached screenshots.

I also notice the LQLEN parameter changing between 32 and 128 under the same conditions (under the esxtop "d" screen)

Shouldn't the DQLEN parameter be fixed at 128? Or am I misunderstanding what this parameter actually is?

On a side note, I see that my Adapter Queue length (AQLEN) is set to 975 in esxtop. Since I only have one VMFS volume attached to this adapter, should I try configuring my DQLEN upper-limit to be higher?

0 Kudos
3 Replies
Scissor
Virtuoso
Virtuoso

I have also noticed this happening on my production ESX3.5 servers as well, but only for the adapters connected to local disks. Perhaps this is "normal" behavior for direct attached storage?

0 Kudos
Scissor
Virtuoso
Virtuoso

Anyone have input on this? Do you see the same behavior on locally attached disks?

0 Kudos
za_mkh
Contributor
Contributor

I've noticed the same thing on our vSphere Servers. I thought this was supposed to be fixed to the QD setting.

On our servers, we worked out that the QD should be 16, so did the following:

In the QLOGIC BIOS - Set Execution Throttle to 16 (which is default - previously ours was 64)

On ESX server advanced settings, set disk.SchedNumReqOutstanding to 16 (Default 32)

running ESXTop on this servers shows the DQLEN changing from 16 to 32 on various FC SAN volumes depedning on load. In my case, QUED remains 0 which is good as far as I understand. Once it DQLEN changes to 32, it seems to stick ...

I wonder where the 32 is coming from? I wonder if the QLOGIC driver in vsphere also has a default queue depth of 32?? One thing I did not do is set the ql2xmaxqdepth value for the qla2xxx drivers to 16 ...

0 Kudos