First off, I am not having any problems right now, just deploying a new SAN and taking baselines. I'm noticing a bit of a discrepancy between what my SAN says and what Vmware says, primarily around latency. I will see somewhere in the neighborhood of 1-4ms of latency using my SAN monitoring tools for each LUN. However, on the Vcenter side, I have occasional spikes into the 15-30ms range that do not appear on the SAN tools. I can only assume this latency is on the Vmware side. Also IOPS are often not equal, but that's probably not a big deal.
Is this normal? And again, it's not really a problem as my average latency is about 1ms, it is just the occasional spikes I see from Vcenter that do not appear on the SAN that concern me.
Have in mind the latency you see in vCenter is VM or LUN based, where you need to consider that the vmkernel, the HBA, the transport layer and the storage target have their own latencies. Don't know what latency you are checking on your SAN. Check this great KB to monitor latancy on ESX server: VMware KB: Using esxtop to identify storage performance issues for ESX / ESXi (multiple versions)