NSX Edge and L2 (MAC-Set) Rule not working

[psullatibm]

Goal:

Use an NSX Edge appliance to apply L2 (MAC based) filtering to VLAN based clients.

Version:

NSX 6.2.3

Steps Followed:

a) Create MAC-set which contains addresses of physical/virtual clients on the VLAN

b) Create the L2 rule in Networking & Security > Firewall to deny access from clients whose MAC addresses are in the MAC-Set

c) In the 'Applies to' field of the rule we select Edge > the Edge want it to apply on.

We can then see if we browse to Networking & Security > Edges > Edge01 that the rule has been applied on the Edge in question. No problem there.

However, if we attempt to generate any traffic i.e. from a client in the VLAN which traverses the Edge it passes without issue. Not good.

Seems like the NSX Edge is not applying the L2 rule correctly..

Questions:

1) Is L2 filtering supported on the NSX Edge in 6.2.3?

2) If not why is it possible to create the rule in N&S > Firewall then apply it to the Edge?

3) Where is this documented, i.e. whether or not NSX Edge supports L2 rules. I can't find it anywhere.

Thanks in advance.

Patrick

Further Info:
1) Rule configuration

2) If we expand the contents of the MAC-set within the Edge view, we get a spinning hourglass and it never seems able to query the membership of the MAC-set. This has happened on three different instances of NSX where we have tested the same configuration. Seems like a bug/feature?

NOTE - In the above screenshots, what I've actually done is add one of the hosts we will allow only to the MAC set. Then instead of inputting the default deny, I've simply changed the action to Block on the rule that would be used to allow those hosts, to see if the functionality works. The machine whose MAC is included in the MAC set for the above rule can still communicate after publishing the rule, therefore I know this does not work.

[cnrz]

NSX Edge Gateways are automatically excluded from the Dstributed Firewall Rules. (Networking and Security > NSX Manager > Manage > Exclusion list ). So if this rule is logged does the traffic match this MAC Rule or other Rule? Most probably it does not match because if left default Edge should be on the Exclusion list.

Troubleshooting NSX for vSphere 6.x Distributed Firewall (DFW) (2125437) | VMware KB

DFW is activated as soon as the host preparation process is completed. If a virtual machine needs no DFW service at all, it can be added in the exclusion list functionality (by default, NSX Manager, NSX Controllers and Edge Services Gateways are automatically excluded from DFW function).

The Ethernet  MAC Firewall Rule Applies to Edge, but this Firewall is the Distributed Firewall that functions at the Kernel Module on ESX host at VNIC Level, not the Firewall of the Edge. The Edge Firewall that exists on the DLR I think is for traffic coming to the Bridged DLR itself, and not for the passing traffic.

If the MAC-List contains the Physical Host Macs, then Applying this Rule to the input direction of the VM NICs may work.  These VMs be at the Logical Switch, or a more general object that will at the end be converted to an IP Address of that VM, so I think don't need to add each VM on a MAC List but may be different for MAC based rules.

Some points that may be helpful to understand:

• Is the NSX Edge that meets the Vlan Based Physical hosts a Distributed Logical Router or an Edge Gateway?
• If bothVM and Physical host  are on the same IP Subnet, then a DLR in the bridged mode should exist for Physical-VM Vlan-Vxlan Conversion.

[cnrz]

There may be a confusion because in the second picture it indicates as the Applied To field is an" Edge Services Gateway" and not a DLR (Logical Distributed Router), so this Firewall rule seems to match the Routed Bare Metal Traffic. So for routed traffic IP SET may be better, about MAC SET it may be different because since Edge Firewall  not a VNIC based, if the Firewall Rule is looked after the packet is routed the source MAC may be changed to Edge Gateway's Internal MAC. But if the Firewall rule is looked at before the routing decision is made, then detailed debugging or troubleshooting may be needed. Since the rule is accepted without an error I think it should work as in your question.

The confusion is I thought there are 2 different Firewall Rule sets independent of each other, but as in this article same ruleset may apply as a dFW rule as well as all or specific Edge Gateways.

http://networkinferno.net/enforcing-a-firewall-rule-on-all-edgeshttp://networkinferno.net/enforcing-...

[sullivanepatric]

Thanks for the responses,

I'm not sure if there is a language barrier here, but the responses are not clear enough to be marked as answers.

1) The particular rules applied at the DFW, appear within the firewall component of the particular Edge in question, therefore I don't think poster #1's response is accurate that this is applying at the DFW layer, and because Edges are exluded from DFW dvFilter that it doesn't apply.

2) As I mentioned, the testing was done on NSX 6.2.3 and the screenshot referenced in #2's response is no longer relevant.

I've had confirmation from the NSX Product team that apply L2 rules to the NSX Edge is not supported, and they are following up internally as to how it was possible to push the rule to the Edge, as their suspicion is it should not be possible.

Looks like I'm the first person in the world to try this...

[sullivanepatric]

Also, this has nothing to do with the DLR, so I'm not sure why that is mentioned.

[cnrz]

I think the DLR  confusion occurred because Physical and Virtual Machines and MAC Filters were mentioned in the post. Since both ESG (Edge Service Gateway) and DLR (Distributed Logical Router) called Edges and Ethernet Firewall Rules are generally used for same L2 Segment communication.

If the Physical Machines are upstream the ESG, then in most cases they are not in the Port Group (Vlan) of the Uplink Interface. They can be in a seperate Vlan behind a Physical Router or L3 Switch that routes the traffic. So in most cases  the ARP table of the ESG does not even see the MAC addresses of Physical Machines. But if in a specific design that  Firewall Rules should explicitly refer to MAC address instead of IP, and the Physical Machines are placed in the same broadcast domain with the Uplink Interface of ESG then this MAC Set based rule be needed.

So if not supported, then getting an error message not accepting the Rule would be better, thanks for your clarification of Product team

[VCDX159]

I am little confused as to why you are using a "Mac-Set" rule on a routing appliance? The routing appliance will only see the mac values of the router that sent it to the Edge (or the MAC that the Edge will place in the Ethernet header as a destination next hop) and its own as either the source or destination depending upon egress or ingress respectively. Therefore, using a MAC-Set rule here is most likely the wrong rule to use when blocking MAC values from respective clients as the MAC addresses are interchanged per every routed hop. MAC-SET rules in NSX are only useful (in most cases) on VNICs for the DFW when attempting to stop or allow traffic by Ethernet within a logical switch segment, with NO DLR or EDGE between the two VMs.

Paul A. Mancuso