NSX-T 3.1 - DFW, "applied to" field - correct usage?

[Czernobog]

I received a simple request, where a user wants to allow multiple employees to access a webserver over https.

I have created a distributed firewall policy with a rule. Two security groups are used:

- Group_source: contains a list of IP addresses, is added as source in the firewall rule

- Group_destination: contains a fabric vm, where the web service is hosts, used as the destination

I have entered both of the security groups in the "applied to" field, as is best practice, so that the created rule only applies to them.

After this the user did some tests, but always got a connection rejected error. The newly created rule did not register any hits, which I could confirm because I receive reject messages from the default deny rule.

After some time I have removed the groups from the "applied to" field and the connection works!

The target virtual machine is placed in a nsx vlan segment. NSX-T version is 3.1.3.

Is there any in-depth documentation on the "applied to" field? I want to use it, to reduce unnecessary rule sprawl, but have doubts on how to apply it correctly.

Edit: upgraded to 3.1.3.6

[CyberNils]

Groups consisting of only IP addresses, MAC Addresses, or Active Directory groups cannot be used in the Applied To text box.

https://docs.vmware.com/en/VMware-NSX-T-Data-Center/3.2/administration/GUID-41CC06DF-1CD4-4233-B43E-...

Examples for usage can be found here:

https://nsx.techzone.vmware.com/resource/nsx-t-reference-design-guide-3-0#_25b2l0r

[Czernobog]

Thank you!

I don't fully understand it yet - if there is a mixed group, containing for example IP addresses and fabric VMs, and this group is added to "applied to" -will the rule be applied to the network interfaces of the fabric vms, or will it not be applied at all?

And in the scenario I have mentioned above: I have Group_sources with IPs only, and Group_destinations with VMs, would a valid solution be to use only Group_destinations in "applied to"?

Edit:

I just did another test. I have a DFW rule where there are 4 security groups as sources: 3 contain vms, 1 a few IP addresses. On destination side there is one security group containing a fabric vm. So in total there are 5 security groups, initially all were inserted into "applied to". A coworker complained that the connection from one of the sources to the destination failed.

Based on the previous experience, I have removed the group containing IPs from the "applied to" field, but this did not help. Only removing all of the groups, that are members of the rule, allowed for the communication to work again.

[Sreec]

It is recommended to configure Applied To at the Rule level or policy level  ( Source&Destination) if both source and destination are on NSX, that being said groups with IPs will be not be applicable in applied to field as it is not mapped to any vnic.

[Czernobog]

I get that using applied_to with a group containing only ip addresses would make no sense in regard to enforcing the rule, but there are some cases where when using applied_to a rule should be applied, but isn't and what's frustrating is, that I cannot find a good explanation for the behaviour. 
In following scenarios the rule is not applied.
Group-A - IPs only
Group-B - a few VMs placed on a NSX segment

Rule 1: Source: Group-A, Destination: Group-B, applied_to: Group-A, Group-B >> rule not applied, as expected, although the documentation only states that this should not be done, without pointing out the consequences
Rule 2: Source: Group-A, Destination: Group-B, applied_to: Group-B >> rule not applied, why? works when applied_to is set to DFW
Rule 3: Source: "any", Destination: Group-B, applied_to: Group-B >> rule not applied, why? works when applied_to is set to DFW

I could not find an explanation for the behaviour of Rule 2 and Rule 3 anywhere. Intuitively I would say they should be applied to the target systems, but this is not the case. Setting applied_to to DFW seems to be a workaround, but then eventually I would face the problem of uncontrolled rule sprawl - which should be prevented by using applied_to.

[animesh41]

Please provide specifics on your testing with Rule2 and Rule3 scenario. Can you check on the ESXi host cli on the VM dvfilter if the rules are getting applied with Rule2 and Rule3 for VM which are in Group B . 

[Czernobog]

The problem is hard to troubleshoot, because the manifestation is not consistent.

I had a client again today who said the communication from "any" to his server is not working. I've first tried reaching his server from my system, which worked fine, given the "any" source in the policy rule. I've let him try, the response was immedietaly a "connection refused", in my syslog I could also see that the nsx default deny rule was reached.

I have checked the applied rules on the host when the vm ist running, and regardless if the rule is applied to the destination gorup or if applied to is "DFW", the rule gets applied to the vnic:

rule 9755 at 588 inout protocol tcp strict from any to addrset 45055f28-8a46-4845-ad78-a1235e1d8f6c port 8082 accept;

After I have removed the destination group from applied_to and set it to "DFW", the rule works. However this sould not be a solution, I have already >800 rules that are applied to the DFW instead of NSX groups.

[guilbar]

Have you ever found a solution for this?

I'm getting similar results on my testing but no explanation of why when the "Applied to" is used the rule using IP-Based Sec Group doesn't apply to the VMs that are part of the Group in the "Applied to".

Even more strange I created a rule:

Source:Group1(vm-based), Destination:Group-02(ip-based-rfc1918), service all, Applied to: Group1(vm-based)

Group1 VMs can reach other VMs that are part of NSX segment and have an IP on RFC1918.
Group1 VMs cannot access other RFC1918 IPs that are not part of NSX segment.

If I remove the "Applied to" Group1 VMs can access IPs on RFC1918 that are not part of NSX segment.
If I disable the rule (to validate that there's no other rule allowing it), Group1 cannot access other servers that are part of a NSX segment.