Lacking FC infrastructure will always stop you using SAN 
Are you finding any performance issues with using iSCSI at all?
Have you dedicated switches etc. to this or do you share with your prod network?
Just curious as I haven't had anything to do with iSCSI yet and was wondering how it would go.
Hi, everyone. Since I was looking for some info on the TP Mode option and found this, I figured I'd try to help out a bit.
Having read this thread, I see there are about 3 separate topics getting confused. I know some of you know this, and some might not, so I'm not "talking down" or anything, just clarifying from my point of view. I say all that so no one gets upset, it's my standard disclaimer from the start. If I get something wrong, or you have a different opinion, please just let me know, don't yell, ok?
Topics:
1) iSCSI vs. Fibre Channel for SAN infrastructure - connections to Hosts basically
2) TCP Offloading for NICs at the Host level (TOE)
3) TCP/IP Offload in the
Guest
VMBeing as though I do get to work with all this stuff, the first topic is practically religion for some people. That said, iSCSI is poised to take SMB by storm in 2007. Stay tuned. I won't get into the debate, but FC SANs aren't going away, iSCSI SANs aren't taking over. It's like saying there'll never be another hard drive internal to a server or no one will ever buy another FC SAN because they're expensive. Let's not kid ourselves, people still buy mainframes, don't they?
The second topic is whether or not the Host, read: Host OS supports TOE. That's where TOE would have to occur. The idea of doing TOE at the Guest OS level is silly. Think about it, it's purpose is to do a physical thing to improve performance. Enabling TOE at the Guest OS level would be like enabling a virtual co-processor. The best it could do is virtualize the instruction set and still have to go back down through VMKernel to do the "work". That said, TOE at the Host level is Guest-OS-independent. It just takes the packets coming through and adds/strips the TCP headers so that the CPU doesn't have to do it. A note on this, Intel's IOAT is the TOE alternative and you'll see does very little to help CPU performance - imagine that, Intel wants to sell more CPUs. All of that being said, TOE is only supported on one major OS as of today - Windows Server 2003 R2 SP1 with the Scalable Networking Pack installed. So, TOE for NICs for the Host OS of VMware can't work as of today. It'd be fantastic if it did. Stay tuned. BTW - TOE has immediate benefits for CPU offloading in Windows with only one gigabit card. It does not necessarily improve throughput all that much until you reach 2 gigabit interfaces driven to 100% utilization or more.
The third topic refers to TCP Checksum offloading, actually. It's poor-man's TOE done with a little bit of driver help. I've seen it have some benefits in hardware-based Windows. In a Guest, again, all you're doing is virtualizing the hardware, so enabling a hardware/driver-based feature isn't going to help you, it'll only probably increase processor utilization. Which is exactly what it did when I just ran a test on a Windows Server 2003 Guest Virtual machine just now. My network throughput dropped a couple percent, nothing big, and proc utilization went up about 5-10%, nothing big, but as expected.
Net: Leave TCP Offload (OFF) in your Guest VM's.
Use iSCSI (TOE-based) hardware initiators for SAN/Disk access
TOE for NICs, network-based access isn't there yet; but would help a lot.
I hope I've helped to clarify here.
I swear I used paragraphs, but it's not taking.
-Bill
