Where are you getting "physical disk write latency" from? Maybe a Perfmon counter? Definitely do not trust Perfmon performance counters. In VC and esxtop, we provide kernel and device latencies. The kernel latency is the time VMkernel takes to process an IO request. The device latency is the time it takes the hardware to handle the request. In esxtop, these to are added together and presented as guest latency.
You gave me a chuckle. These are VMware VI API performance counters. I've dumped the VC 2.5 counters available (via the API) into an XML table so that you can see the VMware identification and description. Here's the 3 counters the I have questions regarding:
"Physical Deivce Write Latency" - http://nworks.com/vmware/counters_vc2.5.xml#disk.deviceWriteLatency.average
"Kernel Disk Write Latency" - http://nworks.com/vmware/counters_vc2.5.xml#disk.kernelWriteLatency.average
"Disk Write Latency" - http://nworks.com/vmware/counters_vc2.5.xml#disk.totalWriteLatency.average
The one-liner description for "Kernel Disk Write Latency" (disk.kernelWriteLatency.average) is "The average time spent in ESX Server VMKernel per write." Probably true but pretty vague. ?:| Does this mean the time from the Guest initiating the I/O until its on the HBA? Maybe one of your buddies on the API team knows? TIA.