VMware Networking Community
Nane76
Contributor
Contributor
Jump to solution

PKS Kubernetes Network Policy and NSX-T Firewall rules sequence problem

PKS version 1.6.1, NSX-T version 2.5.0

Problem:

Policy API Categories and Section gerated by NCP by Manager API don't fit the right, recommended, needed rulebase strategy and the order is wrong.

NSX-T Distributed Firewall – Policy API versus Manager API

Implementation PKS with NSX-T

I had already written many distributed firewall policies and rules in Simplified UI (Policy API) as Infrastructure rules, Environment rules,... .

And I started to apply K8s Network policies and vice versa the PKS applies ("translate") them over NCP as FW sections and rules into Advance UI (Manager API).

The desired state of rulebase is following:

2020-02-06 12_38_50-PKS, Kubernetes Network Policy and NSX-T Firewalls - OneNote.png

Ref. Kubernetes Network Policy in Kubernetes and VMware Enterprise PKS Networking & Security Operations with NSX-T Data Center post

Problem details:

The problem is with the order (sequence) of rules installed into resultant firewall rulebase consisting of FW configuration of both APIs. All is seen in the linear FW rulebase in advanced UI.

The problem is actually with the order of the Categories and Policies of Simplified UI and Sections of Manager API (Advance UI).

Found out order is:

PolicyAPI.Emergency Category.Policies with Rules

ManagerAPI.Sections with Inter+Intra Application Rules (generated by PKS-K8s-NetworkPolicies)

PolicyAPI.Infrastructure Category.Policies with Rules

PolicyAPI.Environment Category.Policies with Rules

PolicyAPI.Application Category.Policies with Rules

( PolicyAPI.Default Layer3 Policy Section – If Distributed Firewall Strategy <> None(my case) )

ManagerAPI.Sections with Cleanup (Default) Inter+Intra Application Rules (generated by PKS-K8s-NetworkPolicies)

ManagerAPI.Default Layer3 Section

But the desired rulebase design assumed and needs that all ManagerAPI.Sections are bellow at least Environment PolicyAPI Policies which contains:

- Mainly : Shared services and infrastructure rules, Dev-Test-Prod environment isolation block rules

- Optionally: PKS environments (DEV, TEST, PROD) infrastructure matrix

And now I see several ways or workarrounds how to use NSX-T with PKS and also manage Standard virtualized datacenter (VMs in vSphere).

But which is right and which is dead end?

A) Configure and manage all objects and firewalls in „legacy“ Manager Policy (Advanced UI)

  • This way is forced by PKS configs into NSX-T
  • It means rewrite hundreds groups and rules in to Advanced Networking
  • But Advanced UI should have been "depreciated and it is frozen" and "all the new features are implemented only on Simplified UI/API"
  • = Dead end?

B) Reinstall PKS into Simplify UI

C) Use mix of Policy and Manager API

  • Place all needed and preferred Policies into Emergency category
  • And wait until PKS would support PolicyAPI

D) Some how force Section sequence or Category priority

Moreover.

Now we use NSX-T SDN only under the PKS platform. But soon we must migrate current virtualized datacenter (VMs in vSphere) networked by NSX-V into NSX-T.

I cannot imagine that I will still have to stay in Advanced UI (Manager (API)), and besides other things, for example write automation by legacy imperative Manager API.

What is the right way or solution?

and many sub-questions and hopefully useful answers …

1 Solution

Accepted Solutions
mauricioamorim
VMware Employee
VMware Employee
Jump to solution

I would go with letter "c". Since Policy API is where VMware is going with since NSX-T 2.4, this way you use it with everything else except PKS, which still uses MP API. After PKS releases a version that supports Policy API you could then migrate so your whole environment is in Policy API.

View solution in original post

6 Replies
daphnissov
Immortal
Immortal
Jump to solution

Ok, so here's an important piece of information which is likely going to rain on your parade and thereby potentially alleviate your questions.

NSX-T as part of Enterprise PKS does not support you consuming it from other things outside of PKS itself and the workloads deployed by it. That means you're not technically allowed to migrate existing VMs into that environment and join them to NSX-T resources. Here's the source: https://www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/product/vmware-product-guide.pdf  and important extract:

pastedImage_0.png

Reply
0 Kudos
mauricioamorim
VMware Employee
VMware Employee
Jump to solution

I would go with letter "c". Since Policy API is where VMware is going with since NSX-T 2.4, this way you use it with everything else except PKS, which still uses MP API. After PKS releases a version that supports Policy API you could then migrate so your whole environment is in Policy API.

Nane76
Contributor
Contributor
Jump to solution

Thank you for help. I guess everything fits.

We have purchased VMware Enterprise PKS with NSX-T Datacenter Advanced Edition.

Now the systems and servers are dedicated only for that.

And the issue is about priority(order) issue of Distributed Firewall Policy API Policies and Manager API Sections.

Note: The recommendation and I hope it make sense is to have only one SDN NSX-T in the datacenter (maybe over more datacenters).

And I am sure we will solve everything to be compliant we VMware licensing model. But it is not question now.

Reply
0 Kudos
Nane76
Contributor
Contributor
Jump to solution

Thank you for your understanding and advise. This make sense.

But two related sub-questions:

- Do you think PKS will ever know Policy API?

- What is actually technical reason, differences of Simplified DFW Categories? Are they only cosmetic folders or do they have some useful features?

Reply
0 Kudos
mauricioamorim
VMware Employee
VMware Employee
Jump to solution

I can't say for sure, but since policy API is where work is concentrated I would say PKS should support it at some point.

The DFW categories are more of way of organizing the rules for an optimized and best practice way of using the DFW. This is part of the microseg strategy explained in VMware® NSX-T Reference Design,​ in section 5.4.

Reply
0 Kudos
Nane76
Contributor
Contributor
Jump to solution

The C way has been selected.

And I wait for PKS and Polici API compatibility and then we will have to use rebuild all the NSX-T environment for PKS infrastructure, reconfigure PKS and redeploy clusters and all apps.

Reply
0 Kudos