PVSCSI adapter performance.

[Gintonic]

Hello!

I have held testing and find out that PVSCSI adapter performance is not fine like I want.

Platform: ESX 4.0 evaluate on Intel SR2500 with Xeon 5420, 32GB RAM and SRCSASRB RAID controller with 3 SATA (Seagate 500GB) drives in 5RAID.

There is single VM on this host: Win 2003 x64 Std with 8Gb RAM and 2 CPU.

I used SQLIO tool for my tests:

sqlio -kR -s120 -frandom -o8 -b8 -LS -Fparam.txt

param.txt is one string - e:\testfile.dat 2 0x0 512

First test:

One boot hard disk on LSI Parallel SCSI controller (SCSI 0:0) and one tested drive(NO independent) on PVSCSI adapter (SCSI 1:0).

Results:

IOs/sec: 503.25

MBs/sec: 3.93

latency metrics:

Min_Latency(ms): 0

Avg_Latency(ms): 31

Max_Latency(ms): 924

Second test:

One boot hard disk on LSI Parallel SCSI controller(SCSI 0:0) and one tested drive(independent and persistent as here http://kb.vmware.com/selfservice/viewContent.do?externalId=1010398&sliceId=1) on PVSCSI adapter(SCSI 1:0)

Results:

IOs/sec: 499.71

MBs/sec: 3.90

latency metrics:

Min_Latency(ms): 0

Avg_Latency(ms): 31

Max_Latency(ms): 763

Third test:

One boot hard disk on LSI Parallel SCSI controller(SCSI 0:0) and one tested drive on another LSI Parallel SCSI controller (SCSI 1:0).

IOs/sec: 614.19

MBs/sec: 4.79

latency metrics:

Min_Latency(ms): 0

Avg_Latency(ms): 25

Max_Latency(ms): 704

Performance of LSI Parallel SCSI controller is greater then PVSCSI! Somebody knows why so?

[ankaiser]

Similar experience here. Almost no difference, with the LSI being a tad faster.

Running a real Oracle (10.1 and 10.2) application with heavy transaction load and lots of small writes.

[drummonds]

The short answer for this is that the PVSCSI adapter has been tuned for high-performance SAN environments. The drivers used for direct-attached storage and those for SAN HBAs are very different. In the current version of the PVSCSI driver we chose to optimize for the latter case. Future versions may be optimal for both.

As of today, don't use PVSCSI with direct attached storage.

Scott

More information on my communities blog and on Twitter:

http://communities.vmware.com/blogs/drummonds

[Gintonic]

Scott, Is the EMC AX4-5i (iSCSi) with SAS drives high performance storage or not?

[drummonds]

It's difficult to generalize the value of the PVSCSI driver, but I'll try. Remember that PVSCSI was created to save CPU cycles in environments where storage activity caused significant CPU utilization.

We estimate "significant IO" at 5000 IOPS or so. For 4K blocks this would be about 160 Mb/s of storage traffic. Below this PVSCSI won't provide much gain. Far below this, with trivial storage utilization and with direct-attached SCSI disks, it's possible for the LSI Logic adapter to outperform PVSCSI. Trivial in this context means 500 IOPS or less.

So, the decision is not array specific. It's workload-specific.

Scott

More information on my communities blog and on Twitter:

http://communities.vmware.com/blogs/drummonds

[VirtualKenneth]

Hi,

And how does the LSI SAS stands inside this performance story? Or isn't this any performance related option and does it only affect the support on hardware level? (i.e. SAS only supported on HW 7)

Thanks,

Kenneth