Hardware vs Software iSCSI initiator

Version 4

    For latest version see: http://vinfrastructure.it/vdesign/vstorage-software-vs-hardware-iscsi/

     

    ESX/ESXi 4 supports both hardware and software iSCSI initiators. Both supports multipathing for higher avaiability.

    A best practice is to never mix both type of initators in the same host.

     

    Software iSCSI initiators are appropriate mostly where there are limited number of host PCI slots. A software initiator can use a port of an existing NIC for iSCSI traffic (but it's still strongly recomened to isolate iSCSI traffic for performance and security reasons).

     

    -


    Technical difference

    VMware ESX/ESXi support both type of iSCSI and in vSphere the performance difference is quite minimal.

    The configuration is quite different, cause in the hardware version (with iSCSI HBA) the configuration is done only under storage adapters, with a software initiator the configuration must be done also on networking and on firewall (in the full ESX version).

    For more info about the iSCSI configuration see the official documentation: http://www.vmware.com/pdf/vsphere4/r40_u1/vsp_40_u1_iscsi_san_cfg.pdf

     

    The big technical difference is that an hardware initiator is a way to offload the complete iSCSI and TCP/IP stack on hardware and save host resources (primary CPU and memory resources). But on new CPU this overhead (for software initiator) could be a minimal problem.

     

    -


    Hardware initiator without HBA

    A lot of new NICs have new hardware feature to offload part of the iSCSI and/or network stack. For TCP/IP offload this feature is usually called TOE (TCP/IP offload Engine).

    Intel NICs use a quite different approach rather than TOE, that is called Intel® I/O Acceleration Technology:

    http://download.intel.com/support/network/sb/19127_lad_competeguide_r04.pdf

    http://features.techworld.com/storage/1236/intels-i-o-at-is-not-a-toe-killer/

     

    Note that vSphere 4.1 introduces a support for some kind of TOE/iSCSI optiomization, but only on a few NIC cards:

    http://www.vmwareinfo.com/2009/01/iscsi-hardware-or-software-how-many.html

    http://vbl0g.blogspot.com/2010/07/vsphere-41-with-iscsi-offloading.html

    http://www.sanstor.info/5iSCSI%20software%20initiators%20vs.pdf

    http://www.dell.com/downloads/global/products/pedge/en/BroadcomDellEnhancedVirualizationFunctionalityWhitePaper.pdf

     

    The problem of this approach is that this kind of optimization will not support jumbo frames...

    Software or Hardware iSCSI?  No Jumbo Frame on HBA?

     

    -


    Performance difference

    What's New in VMware vSphere™ 4: Performance Enhancements - http://www.vmware.com/files/pdf/vsphere_performance_wp.pdf

    iSCSI Performance with ESX 4 - http://www.halfbytetechnologies.com/content/iscsi-performance-esx-4

     

    -


    More info

    http://www.yellow-bricks.com/2009/03/18/iscsi-multipathing-with-esxcliexploring-the-next-version-of-esx/

    http://virtualgeek.typepad.com/virtual_geek/2009/01/a-multivendor-post-to-help-our-mutual-iscsi-customers-using-vmware.html

    http://vmworld2010.com/docs/DOC-3844

    iSCSI HBA vs. Std NIC

    Re: General Question on hardware vs software iSCSI initiators