14 Replies Latest reply on Nov 11, 2016 7:58 AM by SwenEvers

    vCloud: vShield Edge FW Rules

    red_davelee Novice

      Hi all

       

      I'm looking for a way to get and set firewall rules on a vShield Edge firewall in a vCloud environment.  My final target is to write two scripts; one that will export the firewall rules from a given vShield Edge firewall to CSV and another script that will import them from CSV into another vShield Edge firewall.  This is to help with a DR scenario where we'll shift a public IP block from a public facing vShield Edge in one vCloud environment to one in another environment.

       

      I need to do this through the vCloud API or PowerCLI because, if I do it directly at the vShield Manager, vCloud won't know about the changes that have been made.  I'm not a Powershell expert by any means but I'm picking things up as I have a need for them.  I've looked through several blog posts people have written and it looks as though I need to dig down into ExtensionData as there aren't get and sets for the info I'm after.  I've found some info which is really close to what I'm after in this post:

      Deepdive: vCloud vApp Networks | Geek after Five

       

      This covers pulling the information from the GetNetworkConfigSection method in the Extensiondata of a vApp.  The issue I have is that the vShield Edge / network I'm after information from, isn't actually in a vApp.  It's a bit of a funny setup but I've got a vShield Edge firewall connected to the Internet and to an Org Network.  No VMs or vApps are connected to the Org Network.  Instead, I have about a dozen vApps, each with a vApp network and a vShield Edge connecting the vApp network to the Org Network.  This was a strategy recommended by VMware to overcome the limitation of 10 networks on the public facing vShield Edge and works brilliantly in that respect.  However... the public facing vShield Edge and Org Network are not in a vApp I can't use $vapp.ExtensionData.GetNetworkConfigSection()

      Thinking about it while writing this, I guess one option would be to create another vApp and add the Org Network to it, then I might be able to get the info using GetNetworkConfigSection() but I wonder if there is a better/proper/prettier way to do it.

       

      thanks in advance!

       

      Dave

        • 1. Re: vCloud: vShield Edge FW Rules
          rusteh Lurker

          Did you, or anyone else ever resolve this issue? I'm having the same problem in that my vSEs aren't tied to a specific vApp and instead sit under a vDC and service all vApps within the Org. I need to add 20-30 firewall rules to each of them which is insane to do via the GUI.

          • 2. Re: vCloud: vShield Edge FW Rules
            ITSnoesberger Novice

            Hi

             

            I'm not sure if this helps. But to configure the rules on an edge gateway, you should be able to use something similar to this:

             

            #Search EdgeGW
            $edges = Search-Cloud -QueryType edgegateway
            $edge = $edges | Where-Object {$_.Name -eq "XXXXX"}
            #Generate CIView
            $edgeview = $edge | get-ciview
            #Item to Configure Services
            $edgeview.Configuration.EdgeGatewayServiceConfiguration

            #Create new firewallService
            $fw = New-Object VMware.VimAutomation.Cloud.Views.firewallservice
            #Protocol to use
            $protocols = New-Object VMware.VimAutomation.Cloud.Views.FirewallRuleTypeProtocols
            $protocols.Any = $true
            #Create Rule
            $rules = New-Object VMware.VimAutomation.Cloud.Views.firewallrule
            $rules.Protocols = $protocols
            $rules.SourceIp = "x.x.x.x"
            $rules.DestinationIp = "x.x.x.x"
            $fw.FirewallRule = $rules
            #configure Edge
            $edgeview.ConfigureServices($fw)

             

            this replaces all the configured rules, so you have to pay attention when your using it.

            The only way I've seen to read out rules is to go over the api of the vShield Manager.

             

            I hope this helps to solve your problem.

            • 3. Re: vCloud: vShield Edge FW Rules
              rusteh Lurker

              Awesome, that put me in the right direction, thanks a lot. I've modified the script a bit as follows. Is there anyway to do this without overwriting existing rules? I suppose if the rules can't be read from vCloud it would involve 1st reading the rules via the vSM API and then loading these into the same array as the new rules.

               

              #Connect to vCloud

              Connect-CIServer -Server serverName

               

              #Search EdgeGW

              $edges = Search-Cloud -QueryType edgegateway

              $edge = $edges | Where-Object {$_.Name -eq "vseName"}

               

              #Generate CIView

              $edgeview = $edge | get-ciview

               

              #Item to Configure Services

              $edgeview.Configuration.EdgeGatewayServiceConfiguration

               

              $fwService = New-Object vmware.vimautomation.cloud.views.firewallservice

              $fwService.DefaultAction = "drop"

              $fwService.LogDefaultAction = $false

              $fwService.IsEnabled = $true

              $fwService.FirewallRule = New-Object vmware.vimautomation.cloud.views.firewallrule

              $fwService.FirewallRule += New-Object vmware.vimautomation.cloud.views.firewallrule

               

              #First Rule

              $fwService.FirewallRule[0].isenabled = $true

              $fwService.FirewallRule[0].description = "Allow all outgoing traffic"

              $fwService.FirewallRule[0].protocols = New-Object vmware.vimautomation.cloud.views.firewallRuleTypeProtocols

              $fwService.FirewallRule[0].protocols.ANY = $true

              $fwService.FirewallRule[0].policy = "allow"

              $fwService.FirewallRule[0].destinationIp = "external"

              $fwService.FirewallRule[0].sourceip = "internal"

               

              #Second Rule

              $fwService.FirewallRule[1].isenabled = $true

              $fwService.FirewallRule[1].description = "RDP"

              $fwService.FirewallRule[1].protocols = New-Object vmware.vimautomation.cloud.views.firewallRuleTypeProtocols

              $fwService.FirewallRule[1].protocols.tcp = $true

              $fwService.FirewallRule[1].policy = "allow"

              $fwService.FirewallRule[1].port = "3389"

              $fwService.FirewallRule[1].destinationIp = "Any"

              $fwService.FirewallRule[1].sourceip = "10.0.0.0/8"

               

              #configure Edge

              $edgeview.ConfigureServices($fwservice)

              • 4. Re: vCloud: vShield Edge FW Rules
                monderick Enthusiast

                awesome work, going to add this to my arsenal.

                is there a way to export the existing firewall rules from a vShield edge?

                 

                thanks

                • 5. Re: vCloud: vShield Edge FW Rules
                  red_davelee Novice

                  Really appreciate the continued discussion on this one - I was beginning to think I was the only person who wanted to do this!  If anyone can come up with how to export the rules from another vShield Edge that would make my day

                   

                  Dave

                  • 6. Re: vCloud: vShield Edge FW Rules
                    rusteh Lurker

                    Should be easy enough using the vshield manager api. I haven't found a documented way to do it via the vcloud api but that said the above method for putting the rules in doesn't seem to be documented either, and its hard to believe you can add rules without being able to display existing ones.

                    • 7. Re: vCloud: vShield Edge FW Rules
                      ITSnoesberger Novice

                      Here a tipp how to read Data over the vShield Manager API:

                      $vShieldmanagerip = "x.x.x.x"
                      [System.Net.ServicePointManager]::ServerCertificateValidationCallback = {$true}
                      $auth = 'Basic ' + [System.Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes("admin:default"))

                      #GET all edges
                      $url = "https://"+$vShieldmanagerip+"/api/3.0/edges/"
                      $req = [System.Net.WebRequest]::Create($url)
                      $req.Method ="GET"
                      $req.Headers.add("AUTHORIZATION", $auth);

                      $resp = $req.GetResponse()
                      $reader = new-object System.IO.StreamReader($resp.GetResponseStream())
                      [xml]$xmloutput = $reader.ReadToEnd()

                       

                      #here you find all edges:

                      $xmloutput.pagedEdgeList.edgePage.edgeSummary

                       

                      #With the objectid from an edge you can then read out more details from this edge.

                      $url = "https://"+$vShieldmanagerip+"/api/3.0/edges/"+$($edge.objectID)

                      $req = [System.Net.WebRequest]::Create($url)

                      $req.Method ="GET"

                      $req.Headers.add("AUTHORIZATION", $auth);

                      $resp = $req.GetResponse()

                      $reader = new-object System.IO.StreamReader($resp.GetResponseStream())

                      [xml]$xmloutputdetails = $reader.ReadToEnd()

                       

                      #For example NAT and Firewall Rules:

                      $xmloutputdetails.edge.features.firewall.firewallRules.firewallRule

                      $xmloutputdetails.edge.features.nat.natRules.natRule

                       

                      I hope this helps and give some ideas.

                      • 8. Re: vCloud: vShield Edge FW Rules
                        monderick Enthusiast

                        thanks ITSNoesberger!

                        Only had a small issue parsing the line "$url = "https://"+$vShieldmanagerip+"/api/3.0/edges/"+$($edge.objectID)"

                        It would only work if i hardcoded the ObjectID like "https://"+$vShieldmanagerip+"/api/3.0/edges/edge-6" but i could be doing something wrong.

                         

                        now to figure out how to dig deeper and get the actual source/destination IPs and such

                        • 9. Re: vCloud: vShield Edge FW Rules
                          ITSnoesberger Novice

                          Hi

                           

                          Sorry, I had a for-each before to export all edges one-by-one.

                          foreach($edge in $xmloutput.pagedEdgeList.edgePage.edgeSummary){xxx}

                           

                          That's why I had the variable $edge.

                          Instead of looping all edges you can also use a where-object statement to find a single one and assign it to the variable $edge.

                          • 10. Re: vCloud: vShield Edge FW Rules
                            rusteh Lurker

                            So one of my more skilled colleagues matured this script and incorporated forcing you to specify both the OrgName and the VSE name to reduce errors as well as input from a CSV. I'll have a go at incorporating existing rules via VSM api soon.

                             

                            # Replaces all rules for a given vshield with the ones from a CSV file.

                            # CSV header is: Num,Descr,Proto,SrcIP,SrcPort,DstIP,DstPortRange,Policy,Direction,isEnabled,EnableLogging

                            # http://pubs.vmware.com/vcd-51/index.jsp?topic=%2Fcom.vmware.vcloud.api.reference.doc_51%2Fdoc%2Ftypes%2FFirewallRuleType.html

                            # Note: SrcPort can be -1 (for any), any or a port number. DstPortRange can be any or a port number range (ex: 22-26)

                            param (

                            [parameter(Mandatory = $true, HelpMessage="vCD Server")][alias("-server","s")][ValidateNotNullOrEmpty()][string[]]$CIServer,

                            [parameter(Mandatory = $true, HelpMessage="Org")][alias("-vOrg","o")][ValidateNotNullOrEmpty()][string[]]$orgName,

                            [parameter(Mandatory = $true, HelpMessage="OrgNet")][alias("-orgNet","n")][ValidateNotNullOrEmpty()][string[]]$orgNet,

                            [parameter(Mandatory = $true, HelpMessage="CSV Path")][alias("-file","f")][ValidateNotNullOrEmpty()][string[]]$csvFile

                            )

                             

                            # Add in the VI Toolkit

                            if ( (Get-PSSnapin -Name VMware.VimAutomation.Core -ErrorAction SilentlyContinue) -eq $null ) {

                            Add-PSsnapin VMware.VimAutomation.Core

                            }

                            if ( (Get-PSSnapin -Name VMware.VimAutomation.Cloud -ErrorAction SilentlyContinue) -eq $null ) {

                            Add-PSsnapin VMware.VimAutomation.Cloud

                            }

                             

                            try {

                            Connect-CIServer -Server $CIServer 2>&1 | out-null

                            } catch {

                            Exit

                            }

                             

                            #Search EdgeGW

                            try {

                              $myOrgNet = Get-Org -Name $orgName | Get-OrgNetwork -Name $orgNet

                              $edgeHREF = $myOrgNet.ExtensionData.EdgeGateway.Href

                              $edgeView = Search-Cloud -QueryType EdgeGateway -ErrorAction Stop | Get-CIView | where {$_.href -eq $edgeHREF}

                            } catch {

                            [System.Windows.Forms.MessageBox]::Show("Exception: " + $_.Exception.Message + " - Failed item:" + $_.Exception.ItemName ,"Error.",0,[System.Windows.Forms.MessageBoxIcon]::Exclamation)

                              Exit

                            }

                             

                            #Item to Configure Services

                            $edgeView.Configuration.EdgeGatewayServiceConfiguration

                            $fwService = New-Object vmware.vimautomation.cloud.views.firewallservice

                            $fwService.DefaultAction = "drop"

                            $fwService.LogDefaultAction = $false

                            $fwService.IsEnabled = $true

                            $fwService.FirewallRule = @()

                             

                            Ipcsv -path $csvFile |

                            foreach-object `

                            {

                            $fwService.FirewallRule += New-Object vmware.vimautomation.cloud.views.firewallrule

                             

                            $rowNum = $_.Num -as [int]

                             

                            $fwService.FirewallRule[$rowNum].description = $_.Descr

                            $fwService.FirewallRule[$rowNum].protocols = New-Object vmware.vimautomation.cloud.views.firewallRuleTypeProtocols

                            switch ($_.Proto)

                            {

                            "tcp" { $fwService.FirewallRule[$rowNum].protocols.tcp = $true }

                            "udp" { $fwService.FirewallRule[$rowNum].protocols.udp = $true }

                            "any" { $fwService.FirewallRule[$rowNum].protocols.any = $true }

                            default { $fwService.FirewallRule[$rowNum].protocols.any = $true }

                            }

                            $fwService.FirewallRule[$rowNum].sourceip = $_.SrcIP

                             

                            if ($_.SrcPort -eq "any" ) { $srcPort = "-1" } else { $srcPort = $_.SrcPort }

                            $fwService.FirewallRule[$rowNum].sourceport = $srcPort

                             

                            $fwService.FirewallRule[$rowNum].destinationip = $_.DstIP

                            $fwService.FirewallRule[$rowNum].destinationportrange = $_.DstPortRange

                            $fwService.FirewallRule[$rowNum].policy = $_.Policy

                            $fwService.FirewallRule[$rowNum].direction = $_.Direction

                            $fwService.FirewallRule[$rowNum].MatchOnTranslate = [System.Convert]::ToBoolean($_.MatchOnTranslate)

                            $fwService.FirewallRule[$rowNum].isenabled = [System.Convert]::ToBoolean($_.isEnabled)

                            $fwService.FirewallRule[$rowNum].enablelogging = [System.Convert]::ToBoolean($_.EnableLogging)

                            }

                             

                            #configure Edge

                            $edgeView.ConfigureServices($fwService)

                            Example of the csv file:

                            Num,Descr,Proto,SrcIP,SrcPort,DstIP,DstPortRange,Policy,Direction,MatchOnTranslate,isEnabled,EnableLogging
                            0,Allow incoming 80 to webS,tcp,any,any,172.31.31.100,80,allow,in,true,true,false
                            1,Allow incoming 22 to webS,tcp,any,any,172.31.31.100,22,allow,in,true,true,false
                            2,Allow all outgoing,any,any,any,any,any,allow,out,true,true,false

                            Example of invocation:

                            .\load_firewall_rules.ps1 -s 10.16.1.229 -o "Org" -n "DmzNet" -f .\test_csv.csv

                            • 11. Re: vCloud: vShield Edge FW Rules
                              poltraf Novice
                              vExpert

                              Hi Rusteh,

                              I'm a bit low (no, VERY low) on script, as far as I understood the script you posted last is connecting and acting in vShield Manager, am I right? And, if so, how do you manage the vCloud Director database interaction? I mean, if you modify the vse via VSM, when reloading from vCD all the rules would be lost...

                              I apologize for any mistake I wrote...

                              Thanks a lot

                              Raffaello

                              • 12. Re: vCloud: vShield Edge FW Rules
                                Antony13 Novice

                                I've been trying to get this to work for a while, and this script and others have really helped and I now have something working. There is a difference in that I'm running this into the API provided by our cloud provider, so using PowerCLI for Tenants. However, it works, except if I try and import more than 315 rules, I get the below error:

                                 

                                Exception calling "ConfigureServices" with "1" argument(s): "The server

                                returned 'Bad Request' with the status code 400 - BadRequest."

                                At add-firewall-rules-from-CSV-v4.ps1:52 char:1

                                + $vse.ConfigureServices($fws)

                                + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~

                                    + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException

                                    + FullyQualifiedErrorId : CloudException

                                 

                                From everything I've seen on this, the only way to add rules is to import the existing ones from CSV and add the extra ones you want after it, meaning if you have more than 315 rules on the vShield, you can't automate this. Has anyone else had the same issue, and did you find any way round it?

                                 

                                I have a current requirement where the ruleset will be about 500+, and also changing a fair amount, and I can't stand the prospect of having to manually update 200 odd rules via the GUI!

                                 

                                Many thanks!

                                 

                                Antony

                                • 13. Re: vCloud: vShield Edge FW Rules
                                  dmcdave212 Novice

                                  Almost 3 years since I originally asked the question and I had given up on making this work.  I recently had another project that would really benefit from being able to export and import vShield Edge rules so decided to revisit it.  Starting with what rusteh had contributed (and a lot of reading and experimenting!) I was able to put together scripts to import/export NAT and FW rules from vShield Edges, via the vCloud Director API.  This has allowed me to setup automated rule base exports which I can use to import into standby vShield Edges for DR purposes.  I also put together one for doing import/export of vApp FW rules, although I've not needed to do anything with NAT rules as yet.

                                   

                                  Links to the scripts and explanations here.  Thanks to @rusteh for the script that got me started with these!

                                   

                                  vCloud API and PowerCLI – Import/Export vShield Edge NAT rules
                                  http://www.vscratchpad.com/export-vshield-edge-nat-rules-from-vcloud-director/

                                   

                                  vCloud API and PowerCLI – Import/Export vShield Edge FW Rules
                                  http://www.vscratchpad.com/vcloud-api-and-powercli-importexport-vshield-edge-fw-rules/

                                   

                                  Importing and Exporting vApp FW Rules
                                  http://www.vscratchpad.com/importexport-of-vapp-fw-rules/

                                   

                                  Dave

                                  • 14. Re: vCloud: vShield Edge FW Rules
                                    SwenEvers Lurker

                                    Hi Dave

                                    Links seems not to work at the moment.

                                    Kind regards

                                    Swen