PKS version 1.6.1, NSX-T version 2.5.0
Policy API Categories and Section gerated by NCP by Manager API don't fit the right, recommended, needed rulebase strategy and the order is wrong.
NSX-T Distributed Firewall – Policy API versus Manager API
Implementation PKS with NSX-T
I had already written many distributed firewall policies and rules in Simplified UI (Policy API) as Infrastructure rules, Environment rules,... .
And I started to apply K8s Network policies and vice versa the PKS applies ("translate") them over NCP as FW sections and rules into Advance UI (Manager API).
The desired state of rulebase is following:
Ref. Kubernetes Network Policy in Kubernetes and VMware Enterprise PKS Networking & Security Operations with NSX-T Data Center post
The problem is with the order (sequence) of rules installed into resultant firewall rulebase consisting of FW configuration of both APIs. All is seen in the linear FW rulebase in advanced UI.
The problem is actually with the order of the Categories and Policies of Simplified UI and Sections of Manager API (Advance UI).
Found out order is:
PolicyAPI.Emergency Category.Policies with Rules
ManagerAPI.Sections with Inter+Intra Application Rules (generated by PKS-K8s-NetworkPolicies)
PolicyAPI.Infrastructure Category.Policies with Rules
PolicyAPI.Environment Category.Policies with Rules
PolicyAPI.Application Category.Policies with Rules
( PolicyAPI.Default Layer3 Policy Section – If Distributed Firewall Strategy <> None(my case) )
ManagerAPI.Sections with Cleanup (Default) Inter+Intra Application Rules (generated by PKS-K8s-NetworkPolicies)
ManagerAPI.Default Layer3 Section
But the desired rulebase design assumed and needs that all ManagerAPI.Sections are bellow at least Environment PolicyAPI Policies which contains:
- Mainly : Shared services and infrastructure rules, Dev-Test-Prod environment isolation block rules
- Optionally: PKS environments (DEV, TEST, PROD) infrastructure matrix
And now I see several ways or workarrounds how to use NSX-T with PKS and also manage Standard virtualized datacenter (VMs in vSphere).
But which is right and which is dead end?
A) Configure and manage all objects and firewalls in „legacy“ Manager Policy (Advanced UI)
- This way is forced by PKS configs into NSX-T
- It means rewrite hundreds groups and rules in to Advanced Networking
- But Advanced UI should have been "depreciated and it is frozen" and "all the new features are implemented only on Simplified UI/API"
- = Dead end?
B) Reinstall PKS into Simplify UI
- Is it possible? https://communities.vmware.com/message/2924093#2924093
- And manage all in the Simplified UI. By VMware this is generally recommended way (but what about PKS)
C) Use mix of Policy and Manager API
- Place all needed and preferred Policies into Emergency category
- And wait until PKS would support PolicyAPI
D) Some how force Section sequence or Category priority
Now we use NSX-T SDN only under the PKS platform. But soon we must migrate current virtualized datacenter (VMs in vSphere) networked by NSX-V into NSX-T.
I cannot imagine that I will still have to stay in Advanced UI (Manager (API)), and besides other things, for example write automation by legacy imperative Manager API.
What is the right way or solution?
and many sub-questions and hopefully useful answers …