All other responses have been appreciated.

On to your second comment about rack servers vs. blades. Both options are feesible. In the next year you will see every blade vendor (IBM, HP, Dell) coming out with advanced function blade switches for the server chassis. Brocade and Cisco already produce 10G and Fiber Channel blades for the Dell m1000e chassis. This will allow those enterprises who have already invested in blades to extend their efficiency. Due to fabric and backplane requirements I wouldn't be surprised if this required some vendors to introduce a new chassis to support the added functionality. FCoE may introduce some great improvements, but at what cost? You don't want to have spend the savings from hardware consolodation on management of the technology. FCoE shouldn't have any impact on your server platform other than the usualy driver requirements. Both Emulex and Qlogic has drivers for every major platform - Windows, Linux, Vmware and Soalris.

Design will always be dependent on your networks requirements. For those with a blade environement you may not need to uplink into a top of Rack switch like a Nexus 5000, but rather directly to your core switch such as a Cat 6500 or Nexus 7000. In my personal opinion I'm not yet a fan of the Nexus 7000. It's a very expensive core switch with limited fuctionality (at this point). Unlike the Cat 6500 there are no service cards (Firewall, Content Switching etc) for the 7000. It's built to be a very fast core switch. It doesn't even have Fiber Channel functionality yet. This is going to keep the 6500 as most enterprises core switch-of-choice for a while longer.

As for security Texiwill, what I can offer to address your concerns is what Cisco has come out with called TrustSec. TrustSec will allow for encryption of data between switches or between hosts & switches preventing MiTM attacks. I'm not an expert on TrustSec, but do know one of its advantages will be to address security concerns that are being identified as part of virtualization's maturing. I do hope this will grow into a standard protocol that can be used in environements with hardware from multiple vendors similar to IPSec.

Texiwill - any additional insight on this?

Cheers.

Truth be told - they're both great options. iSCSI is a great low-er cost option, but does not match the performance of FC. When I first took at look at both protocols my first impression was "wow, they're encapsulating ANOTHER protocol...big surprise". Sometimes I just prefer pumpkin pie over a 7-layer cake. Most performance hits come from the IP vs. FC protocol stack. Another very important factor is install base. Show me any enterprise network running critical HPC or databases on solely on an iSCSI SAN and I'll be shocked. Due solely to FC's presence in the market place networking giants will keep pushing this technology to be the higher performer of the two, of course with a heft price tag attached. It will also keep the more advanced management at in the hands that have come to be accustomed to it since FCoE is simply encapsulated FC which has been around far longer than iSCSI. It's a fact that it's tough to teach old dogs new tricks. Since iSCSI utilizes TCP for reliable delivery we know that it comes at the price of collisions, dropped packets and re-transmits. On the other hand, FC requires a lossless transport as you mentioned so rather than dropping packets EVERY packet must arrive in order without dropping them. FC accomplishes this through buffer-to-buffer (b2b) credits. At the end of the day, having to re-transmit packets over a lossy network will not be as fast as utilizing pause frames to control congestion. Re-transmitting packets also require the sending initiator to reprocess and send the information again, whereas b2b credits will utilize the buffer of the closest switch to the storage target and work backwards. This means that congestion would only ever reach the host on a worst case scenario. Personally I'd rather let my switches share the congestion load than force my hosts to retransmit from the furthest point away from of the storage target. Another benefit of FCoE is Priortiy Flow Control (PFC) which basically is QoS for FCoE. This allows you to keep your storage and IPC traffic at high priority & low latency and you LAN or other non-critical traffic using up any left over bandwidth. QoS can be used in addition to PFC in an FCoE infrastructure to give an unpresidented about of traffic control to the network admins.

For me the selling point of FCoE over iSCSI is consolodation. There's a huge cost reduction in cable management, powerusage and administration (both physical and logical) with multiple NICs and networks. iSCSI will not be able to unify storage, LAN, IPC, Management, and HPC networks as FCoE will (at least not anytime soon) - all without changing your existing storage infrastructure. The important thing to realize is the development of the FCoE technology will in no way force iSCSI to become obsolete. I believe every technology has its place.