Nexus 1000v System VLAN Ports Blocked

[NetJnkie]

I'm trying to migrate the Nexus 1000v VSM from vSwitch port groups to those I created on the dvSwitch for Packet, Control, and Management. Management seems fine, but Packet and Control are not. When I move the network connections to the dvSwitch it appears the ports are blocked. A screenshot below shows the X icons. Why is it doing this?

[lwatta]

I can't recall what my configuration shows. One thing I do remember is that the GUI is not always a good representation of whats going on.

After you move the VSM so that it's network ports are on the VEM module, you should still see the VEM modules with a "show module" command on the VSM command line. If you still see all the VEM modules show up then your control network is working.

There is no L3 connectivity between VSM and VEM on the control network. It could be that VC doesn't see L3 traffic so it does not turn the port green. All control traffic is L2 only.

As long as your VSM can still see all the VEM modules you are good to go. I'll duplicate in my lab and if the ports don't go green I'll file a bug.

One last thing. Make sure your port-profiles for control and packet have a "system vlan" directive in the configuration.

louis

[NetJnkie]

Thank you Louis. I had forgotten the "system vlan" lines in the Control and Packet port-profiles. So now it's working when I move it over but I'm getting:

2009 Aug 7 15:43:27 CRMCN1KvMcK %KERN-1-SYSTEM_MSG: Dropping received frames from duplicate VSM - kernel

I know this means it's seeing a loop, but why? I'm using vPC-HM and it appears to be correct. Below is config and output:

show int bri

-

---

Port VRF Status IP Address Speed MTU

-

---

mgmt0 -- up 10.180.0.10 1000 1500

-

---

Ethernet VLAN Type Mode Status Reason Speed Port

Interface Ch #

-

---

Eth3/5 1 eth trunk up none 10G(D) 1

Eth3/6 1 eth trunk up none 10G(D) 1

Eth4/5 1 eth trunk up none 10G(D) 2

Eth4/6 1 eth trunk up none 10G(D) 2

Eth5/5 1 eth trunk up none 10G(D) 3

Eth5/6 1 eth trunk up none 10G(D) 3

Eth6/5 1 eth trunk up none 10G(D) 4

Eth6/6 1 eth trunk up none 10G(D) 4

Eth7/5 1 eth trunk up none 10G(D) 5

Eth7/6 1 eth trunk up none 10G(D) 5

Eth8/5 1 eth trunk up none 10G(D) 6

Eth8/6 1 eth trunk up none 10G(D) 6

Eth9/5 1 eth trunk up none 10G(D) 7

Eth9/6 1 eth trunk up none 10G(D) 7

-

---

Port-channel VLAN Type Mode Status Reason Speed Protocol

Interface

-

---

Po1 1 eth trunk up none a-10G(D) none

Po2 1 eth trunk up none a-10G(D) none

Po3 1 eth trunk up none a-10G(D) none

Po4 1 eth trunk up none a-10G(D) none

Po5 1 eth trunk up none a-10G(D) none

Po6 1 eth trunk up none a-10G(D) none

Po7 1 eth trunk up none a-10G(D) none

-

---

Interface VLAN Type Mode Status Reason MTU

-

---

Veth1 404 virt access up none 1500

Veth2 403 virt access up none 1500

Veth3 403 virt access up none 1500

Veth4 404 virt access up none 1500

Veth5 401 virt access up none 1500

Veth6 401 virt access up none 1500

Veth7 401 virt access nonPcpt nonParticipating 1500

Veth8 403 virt access up none 1500

Veth9 403 virt access nonPcpt nonParticipating 1500

Veth10 404 virt access nonPcpt nonParticipating 1500

Veth11 404 virt access up none 1500

CRMCN1KvMcK(config-port-prof)#

port-profile system-uplink

capability uplink

vmware port-group

switchport mode trunk

switchport trunk allowed vlan all

channel-group auto mode on sub-group cdp

no shutdown

system vlan 403-404

state enabled

port-profile Management

vmware port-group

switchport mode access

switchport access vlan 401

no shutdown

state enabled

port-profile Control

vmware port-group

switchport mode access

switchport access vlan 403

channel-group auto mode on sub-group cdp

no shutdown

system vlan 403

state enabled

port-profile Packet

vmware port-group

switchport mode access

switchport access vlan 404

channel-group auto mode on sub-group cdp

no shutdown

system vlan 404

state enabled

[lwatta]

You should not be getting duplicate frames if you are using vPC-HM.

The config below looks like 7 hosts with 2 nics each. Are the connections of the nics to different switches or the same switch? vPC-HM is meant to channel connections to two different switches. You'll get the duplicate frame error if you configure vPC-HM with multiple connections to the same switch.

The only other thing is that you have specified "channel-group auto mode on sub-group cdp" under the Control and Packet port-profiles.You only need to configure channel groups on the uplink port-profiles.

Take the channel-group command out of the control and packet port-profiles and see if the error goes away.

louis

[rob_ellison]

i'd seriously suggest not moving the VSM inside the dvSwitch. if for some reason you have a problem, the whole of the network will die. i tried this initially but after a lot of testing decided on keeping the VSM on a separeate standard vSwitch. since then i have had no issues loosing the VSM.

[ViFXStu]

Can you attach standard VM's to this port profile too? Can the vCenter server be on the same port profile when it is defined as a system VLAN?

Also, out of curiosity what's vswif3 for in the screenshot?

[lwatta]

Yes you can have attach standard VMs to the port-profile. I do this all the time. I create on port-profile and stick my packet, control, and mgmt interfaces for my VSM as well as my vCenter server on the same port-profile.

louis