We are currently going through a design build with HP and have come across the same problem.
HP documentation is at fault here as it is not adequately explaining how the Flex-Nics work.
In the following document: HP Flex-10 Technology.pdf brief on Page 14 it says:
SmartLink | SmartLink not supported on individual FlexNICs but will continue to be supported from the physical port point of view. In this Flex-10 release, VC is not capable of dropping the link to a single FlexNIC. Therefore, the physical link will drop only when all networks assigned to all FlexNICs on a given physical port have “SmartLink” option checked, and all of their uplinks are broken. Conversely, as soon as the first vNet has a single port restored, the physical port, and therefore all FlexNICs, will have the link restored. This functions in a similar fashion to non-Flex-10 ports with multiple networks assigned. SmartLink continues to operate normally for traditional 1Gb NICs in a VC environment and will be fully functional for Flex-10 operation in a future release of VC firmware. |
That indicates that you cannot use SmartLink for individual Flex-Nics so if the uplinks do fail it will not pass that information onto the blades and so VMware will not fail over and either the LAN or NAS traffic will be blocked.
However back to HP BladeSystem Reference Architecture Virtual Connect Flex-10 and VMware vSphere 4.0.pdf on page 24 it says:
Virtual Connect firmware v2.30 introduced Dynamic Control Channel (DCC) support to enable SmartLink with FlexNICs. This allows for individual FlexNIC
state change if the uplink ports for a defined vNet are no longer available to force NIC teaming software failover.
So, which is it?
Our testing shows that Smart Link is not working on a FlexNic even though we are on firmware 2.31
We have 2 x Virtual Connect Flex-10 modules per chassis, 3 chassis in a rack, so 6 Flex-10s per rack.
All Internal and external stacking links are active.
We tunnel all VLANS through the Flex-10 so we don't have the extra administration of specifying vlans in the Flex-10 modules so they pass through all vlans.
We have created the following Virtual Connect Ethernet Networks for production use.
esx_switch_a
esx_switch_b
esx_int_vmotion_a
esx_int_vmotion_b
We also create 4 more networks with no uplinks which we called unused nics. This is so we can apply the server profile with pre-created networks so we shouldn't have to power down any blades once deployed to amend the server profiles.
These unused networks may be used in the future to add additional networking for example for within VM iSCSI storage(ie. not the disk files but
remote databases etc.)
The server profiles are then applied which add the 8 networks which correspond to this:
Virtual Connect Port | Flexible Nic | VMware assigned Nic | Ethernet Network |
Port1 | LOM1a | vmnic0 | esx_switch_a |
Port2 | LOM2a | vmnic1 | esx_switch_b |
Port3 | LOM1b | vmnic2 | esx_int_vmotion_a |
Port4 | LOM2b | vmnic3 | esx_int_vmotion_b |
Port5 | LOM1c | vmnic4 | unused_nic_3a |
Port6 | LOM2c | vmnic5 | unused_nic_3b |
Port7 | LOM1d | vmnic6 | unused_nic_4a |
Port8 | LOM2d | vmnic7 | unused_nic_4b |
The 2 esx networks, esx_switch_a and esx_switch_b is for all external traffic which includes LAN traffic for Virtual Machine and Service Console and NAS traffic for NFS storage. All traffic is logically separated by vlans.
The 2 vmotion networks esx_int_vmotion_a and esx_int_vmotion_b have no uplinks assigned which use the internal stacking network per rack so vmotion traffic is all internal to the rack.
We have also tried to force SmartLInk to think that all links are down by allocating uplinks to all the internal networks but without them connecting to anything.
When we disconnect the real uplink it still didn't mark the vmnic as down.
We can solve it with Beacon Probing but I don't know whether this is a better way or having redundent uplinks at the switch level, basically having 2 uplinks from the Ethernet Network but from different Flex-10 modules so if a switch fails there is another route. One uplink would be Active and the other Standby
We have a call later today with HP to discuss and I'll update here with any more information.
What has frustrated me is running ESX on blades through Flex-10 is not a different scenario and HP's own documentation is confusing and at times wrong. the VC cookbook is out of date.
If anyone has ever read the technical reports from NetApp on how to set up their equipment with various different vendor products, they know how a solution document should be written....come on HP.
http://WoodITWork.com
