Skip navigation

Another time we are hearing about a worldwide attack using a ransomware to stole money (bitcoins) and create services unavailability or data losses. One question in many company is asked to IT department :- Are we protect against this attack?-

 

It’s know that this attack, like the major part of the attacks, is permitted due a security bugs in Windows systems. For this reason, last month, Microsoft made available a patch to avoid this attack. But how many people already have updated their system?

 

You know, sometimes is difficult for many IT department to find the right time to apply updates and patches, because in 99% of cases it requires a reboot, with consequent unavailability or dependency problems. Answering the question for protection, if you’re in a VMware farm with many windows VMs in a domain controller, you could take the advantage of the powercli to getting the state of windows update for every window VM.

 

A note from Microsoft is released with its KBID: https://support.microsoft.com/en-us/help/4013389/title

 

Requirements

  • an vCenter account with enough privileges to execute a script using Invoke-VM script
  • an active directory administrator account (or with enough privileges) to execute Get-HotFix command
  • powercli (of course!)

The Script

Here the example:

$vCenter = "vcenter-ip-fqdn"

$winUser = "DOMAIN\Administrator"

$winPwd = "password"

$checkHotFix = @("KB4019215","KB4012598", "KB4012216", "KB4012213", "KB4012217", "KB4012214", "KB4012215" , "KB4012212", "KB4013429" , "KB4012606", "KB4013198")

 

Foreach ($vm in (Get-VM | Get-VMGuest | Select-String "windows")){

$vm = ($vm -split ":")[0]

## Test credentials

Try {

      $result = Invoke-VMScript -VM $vm -GuestUser $winUser -GuestPassword $winPwd -ScriptText "[System.Environment]::OSVersion.Version" -ErrorAction Stop

}Catch{

      Write-Host "Invalid Credentials for $vm"

      continue

}  

Write-Host "----------------------------------"

Write-Host "VM Name $vm"

Write-Host "----------------------------------"

 

$result = Invoke-VMScript -VM $vm -GuestUser $winUser -GuestPassword $winPwd -ScriptText "Get-HotFix | Select 'HotFixID'"  

Foreach ($fix in $checkHotFix){ 

      if ($result.Contains($fix)){

           Write-Host " $fix  Found"

      }

}

Write-Host "----------------------------------"

}

For newbie in powercli, before launching the script you must connect to vCenter using the cmdlet Connect-VIServer. Hope this could be useful!

Don’t ask to your Network Administrator: could you handle all my VLAN and security starting from applications? He never say yes! But there are some easy ways that could be taken to gain flexibility deploying and operating network connections between VMs and the external world:

  1. Use a Business Process software with REST Api: depending on your environment this process could take hour and days to being deployed and many prayers to make “secure” your environment
  2. Use CLI and gain the access to every datacenter switches
  3. Use NSX

The last is what every virtualization administrators are looking for but the major part of networking admin are not safe to approve! And this is why I’m writing this post.

All starts from application… but ends soon with Network administrator

One of the most important element that every system administrator must consider during infrastructure design is the application perspective: the only contact point with the development staff! Depending on the experiences of your dev engineer, you should be able to get some important application characteristics  like:

  • capacities
  • functional diagrams
  • performance metrics

But there is a forever missed argument on your interview that often cause a security problem:

  • network connections (LAN)
  • ports and sessions
  • service availability

They always say: networks and security are up to you and the network team! Before NSX this was the exasperation of the project interviews, and you, the poor infrastructure administrator, are acting as “the man in middle” to try to fit what was missing with the environment you are working on!

 

image

Giving a security perspective, the top from application perspective is:

  • segment every connection
  • handle every traffic under the magnifying glass
  • change the network topology and provide the isolation as provided by gap air
  • don’t change network topology in test and DR environment

From systems perspective working in datacenter means:

  • simplify where is possible
  • introduce security on every network segment
  • centralize the management

There are some challenges near some worries that must be covered using the technology and this is the case where software defined paradigm aid to build a flexible and secure virtual datacenter.

Hardware (not)commodity?

Traditional datacenter net deployment is composed by:

  • firewall/router
  • switch (2n per every rack)
  • server and storage

 

image

 

Firewall/router and L3 switch on top of the scheme represent the single point of failure of the entire network (for some manager is the way to spend a lot of budget!). Talking about connection flows: cps (connections per second) and throughput are the two main critical factors to consider to avoid networking congestions. For every connection starting for VM and ending to another VM, quite all physical network devices are involved in linking and controlling every packet, even if the traffic doesn’t go outside the vDC. There is a “waste” of resources!

 

Let NSX exceeds the limits with VXLAN

From more and more literature about NSX there is a scheme that shows every component from management plane to data plane. Let’s take a look here: https://communities.vmware.com/docs/DOC-30645 There are 3 key elements that are involved in data flow optimization:

  • VXLAN
  • DLR Distributed Logical Router
  • DFW Distributed Firewall

VXLAN could be defined as a L2 in a L3 connection, build to virtualize network traffic inside physical or logical connection (https://tools.ietf.org/html/rfc7348). During NSX host preparation one vib enables virtual switches through a vmkKernel interface to use this “service” and start deploying logical connection starting from index 5000 (it’s a convention used to separate VXLAN index from VLAN: a value greater than  4096). The prerequisites are:

  • all host in the cluster must be connected to a common distributed switch
  • MTU for physical and virtual network elements must be set > 1550 (1600 is the default)
  • NIC teaming policy design must be the same for every portgroup in the distributed switch

Using a transport zone, a global connection between every hosts involved in one or more VXLAN connections, every hosts through vmkernel service, called VTEP, could establish a network connection like happens in physical world. In other words every packet generated by a single VM follows this sequence:

  1. Through virtual nic the packet arrives to the VXLAN port group (generated after VXLAN segment deployment aka logical switch deployment) which is handled by the VTEP
  2. VTEP relative to the Host which handle MAC address tables and MAC address resolution.
  3. If destination VM is in the same host, the “link” will be established with no traffic outside the host, otherwise, through Transport zone, packet is encapsulated and transmitted to the other ESXi host, which is handling the VM
  4. VTEP of the other host de-capsulate the packet and bring it to destination VM.

 

image

 

Source: https://communities.vmware.com/docs/DOC-27683

 

Apparently this could be a complication of the “standard” way to make networking in a virtual environment… but let’s see the benefits:

  • No physical intervention: all traffic flows through a single VLAN!
  • Bypass the limit of 4095 VLAN
  • Take the control of the entire networking through a unique interface, because every logical switch has more than a simple connection mechanism… let’s see in the next paragraph!

 

Let NSX routes your packets with DLR

Suppose you’re going to handle connections between two VM connected to two separated VXLAN segments:

image

Source: https://communities.vmware.com/docs/DOC-27683

 

Near ARP table, there is a vSwitch vib called DLR that handles (and shares) a routing table. The new sequence from VM1 to VM2 will be

  1. The packet flows through VXLAN portgroup (aka logical switch portgroup) and arrives to the default gateway located to local DLR
  2. A routing lookup is performed at local Distributed Local Router which indicates the destination logical interface (LIF) logically connected to destination VXLAN segment, and a lookup ARP table is performed to determine the VM destination MAC address (ARP requests could be done is ARP table doesn’t contain the resolved IP address)
  3. If the VM is placed outside Host the packed is encapsulated and sent to destination Host, otherwise the packet is sent to destination VM without flowing out the host.
  4. The destination VTEP de-capsulate the packet and send it to destination VM

 

Do you remember the same situation the In physical world? Single or multiple router, with a single connections per second limit, must be deployed… and large amount of traffic will be flowed across every switches places between hosts and router. In the current case no traffic goes outside the transport network!

Let NSX secure your traffic with DFW

Last but not the least is the firewalling check in every segment to protect north-south and east-west traffics. With a similar mechanism of DLR, every packet is checked by Distributed Firewall (L2-L4 stateful), using a Rule Table and a Connection Tracker Table to allow or deny connections before the packet is going out of logical switch.

For every packet sent and/or received by a VM:

  1. a connection tracker lookup is done to check if an entry of the flow already exists
  2. if no  entry has found, a rule table lookup is done to verify which rule is applicable
  3. if action is Allow the packet flows outside the DFW

 

image

Source: https://communities.vmware.com/docs/DOC-27683

 

Note that no traffic is generated in every direction in order to analyze and permit/deny the traffic. In old physical deployment this is could not be happened because firewall connection is after switch connection. The result is that every flow generates a lot of connection and the core firewall must be well sized in order to guarantee security and availability.

 

Physical World connections at the “Edge” and more

VXLAN, DFL and DFW are involved in east-west traffic to realize a resilient and efficient networking cluster. Using NSX EDGE it could be possible to handle traffic and services outside the vDatacenter: the north-south connection. Let’s see an example:

 

http://blog.linoproject.net/wp-content/uploads/2017/04/image-17.png

 

image

Source: https://communities.vmware.com/docs/DOC-27683

 

The combination of DLR and VXLAN could be used to realize east-west traffic and the EDGE is used to route packets in a portgroup which is physically connect to a switch. Out of the box EDGE is a virtual Router/Firewall (a VM) that brings many services like firewall rule and routing table but also NAT, load balancing and VPN. Be in a virtual environment gains the advantage to interconnect VM with a “software link” that could be used to make a bridge for the physical world.

 

A new network design

Well! The question is: with these elements is it mandatory a physical firewall? I could say NO only if the north traffic is clean… in a other words: only if you are not directly connected to Internet. Watching what is going on about the newest the security attacks, I suggest a L7 protection using:

  1. A 3rd party plugin like F5, CheckPoint, TrendMicro
  2. Use a physical IDS/IPS

 

Thinking all virtual this could be a great solution:

 

 

image

 

Physically is really simple: a top of the rack switch (min 2 stacked) and a cluster of physical server with only 3 VLAN running across the entire host with a single dVS. Just few notes:

  • consider the using of 10Gbe switch with 10 Gbe SFP+ NICs;
  • in order to use VSAN consider a correct throughput (the best is a separate network infrastructure for this purpose);
  • in you’re expanding this cluster across multiple racks, consider the “leaf-spine” network topology for production network.

 

And now you’re the King!

 

Notes and Sources

There’s a lot of documentation and integration, with some success cases. For study I really suggest the design guide available here: https://communities.vmware.com/docs/DOC-27683 (the major part of the images are taken from this document)

For newbie there is a classic “For dummies” guide that I found really well written for who is approaching to network virtualization with an eye to practical uses… Here the link to register and download a free copy

Near the literature, I suggest to practice with Hands on Lab to preview all these functionalities without wasting time for setting up a test environment:  http://www.vmware.com/go/try-nsx-en

Developing through containers is not only a fashion wave for coding in your PC/MAC (aka devbox) using library limitless, but a real opportunity to operate from dev to production environemnts without system administration knowledge. With vcloud air and docker, any developers will get all application controls in your devbox.

Sometimes people things: << vcloud director and vcloud air are only cloud infrastruscture as a Service... Platform as a Service is another thing>>. Well! It's time di think different stating with a little vcloud vdatacenter and docker-machine (Suggest to use docker-toolbox which contains  docker environment, docker-machine, kitematik...)

 

 

Setup vCloud Air (vchs)!

The first thing is: start pay vdatacenter and wait for activation: VMware vcloud air personell could help you to acheive it without blocks or fears!

One of the important thing to do is to write down available ip, because there is a 1:1 relationship between docker host and public ip and 1:many between public ip and containers. From behind docker machine will works by ip: in some scenarios the use of routed ip's could be enough to start using containers without the need of public ip.

In summary these parameters are mandatory to deliver a docker host in cloud:

  1. vcloud id (or tenant id)
  2. public ip (be sure that edge gateway has a vaild dns)
  3. vcloud tenant with vm creation permissions:

...and after user confirmation don't forget to assign vDC resource: http://pubs.vmware.com/vca/index.jsp?topic=%2Fcom.vmware.vca.ug.doc%2FGUID-A6EDBE95-24CA-4AA6-9C1E-AF0A3D8CE57A.html

 

Install docker toolbox (with kitemitik)

This is the devbox side, and I don't wont to tell how to double click the install file and follow the clear installation instruction provided in the official website. I leave your play with https://www.docker.com/products/docker-toolbox.

Another things to do is the creation of a docker-hub and a git-hub accounts (if you don't have one of these):

  1. https://hub.docker.com/
  2. https://github.com/

 

Deploy and use docker-host

Finally this is the coolest part of the show: run your first container host.

To acheive this simply run this command from your cmd (or shell) in the devbox machine:

docker-machine create --driver vmwarevcloudair --vmwarevcloudair-username=xxxxxxxxxx \--vmwarevcloudair-password=xxxxxxxx --vmwarevcloudair-vdcid=mxxxxxxx-xxxx \--vmwarevcloudair-publicip=aa.bb.cc.dd docker-vchs-01

After few minutes system must go on (you could check vm operations simply opening vchs console via vcloud director portal)

In some cases default catalog could be return many errors generated during docker installation problem... this is due a ubuntu catalog problem.

To solve this you shoud ssh into machine using docker-machine ssh command... then remove (or better move out) vchs repository under /etc/apt/sources.list.d/

then launch the provision command:

... well system is up and running with this effort:

  • 1 user creation and or login in vcloud air
  • a piece of notepad
  • 1 command
  • 1 problem that hope could be solved by waiting vchs team or changing catalog.
  • no sysadmin intervention

 

Simplify application delivery from dev to prod

Now it's time to devel your first application in devbox and in production simply using docker-machine command and moving between these 2 environment:

devbox

& "C:\Program Files\Docker Toolbox\docker-machine.exe" env default | Invoke-Expression

then deliver your first application like drupal:

docker run --name druapl-dev -p 8080:80 -d drupal

production

& "C:\Program Files\Docker Toolbox\docker-machine.exe" env docker-vchs-01 | Invoke-Expression

then deploy drupal in production

docker run --name druapl-prod -p 80:80 -d drupal

 

Now it's time to enjoy these 2 apps and start developing components, modify something and working with containers without limits, without vpn, and from your dev machine!

...and wow!!! No sysadmin has aided you during these operations! Dear devops welcome to your new working environment.

 

 

With Workstation 11 we have seen the integration with vCloud Air: uploading, running, and viewing virtual machines from the interface, to make a kind of hybrid cloud between your workstation and vCloud Air.

With this new release VMWare want to give up a gear:

 

VMware vCloud® Air™ Incentive Program – To help users get familiar with the benefits of cloud computing, VMware Workstation 12 Pro customers are eligible to receive $600 in vCloud Air service credit to use for up to 6 months – twice the standard VMware vCloud Air sign-up offer.


Other cool things are the ability to run Windows 10 and the last release of important OS (CentoOS, Ubuntu, ...) and the support for DirectX 10 and OpenGL 3.3.

 

For further informations read VMware Workstation 12 Pro Release Notes

 

Because, I'm trying this product, this post could be updated... Stay tuned!

In many projects my colleagues often ask me what VM templates I must create, and how many resources I must assign to it.

The first thing should to do is identify the workload or the business logic that platform must sustain during its lifetime. In the case of a service provider, templates, follow the simple list of products; but in the case of a dedicated project or private cloud consolidation, the analysis of the workload is mandatory to identify which and how many templates must be created:

 

 

VMOSNetwork/EnvironmentResources
VM1Windows 2008Prod12vCPU, 8GB RAM, 150 GB vmdk
VM2Windows 2012Prod12vCPU, 8GB RAM, 50 GB vmdk
VM3Windows 2012Prod11vCPU, 4GB RAM, 50 GB vmdk
VM4Windows 2012Prod21vCPU, 4GB RAM, 50 GB vmdk
VM5Windows 2012Prod21vCPU, 4GB RAM, 50 GB vmdk
VM6Windows 2012Prod24vCPU, 16GB RAM, 200 GB vmdk
VM7Windows 2012Prod12vCPU, 8GB RAM, 50GB vmdk

 

First of all, identify the Operative Systems:

  • Windows 2008
  • Windows 2012

In this Windows 2008 could be excluded from the templating project because is the one in the project, and assume that the future instances should be Windows 2012 only. Then VM template list could be:

 

TemplateOSNetworkResources
TPL1Windows 2012

 

The next consideration is the network and/or the environment, because time to deploy of a VM could be decreased in a environment with dhcp and active directory: during deploy you could automatically join active directory, without next manual intervention. <<Less manual intervention for better life of the sysadmin>>.

 

TemplateOSNetworkResources
TPL1Windows 2012Prod1
TPL2Windows 2012Prod2

 

 

Last but not the least is identifying the minimum resources the template should have. for vCPU and Memory you could follow the minimum requirements for the OS or identify the minimum parameter in the list. In our case 1 vCPU and 4GB vRAM. But for vmdk the rule could be different: you could extend your disk increasing vmdk capacity or binding another vmdk, but reducing could be dangerous, complex and insecure. For this reason, it must consider the minimum capacity for OS or the minimum parameter in the list.

 

Template list could be this:

 

TemplateOSNetworkResources
TPL1Windows 2012Prod11vCPU, 4GB RAM, 50GB vmdk
TPL2Windows 2012Prod21vCPU, 4GB RAM, 50GB vmdk

 

Enjoy your deploy.

linotelera Hot Shot
vExpert

ESXi 6.0b patch released

Posted by linotelera Jul 13, 2015

And vsphere 6 goes ahead with this update called 6.0.0b. For vcenter and esxi patch include fixes and improvement like

  • cim and api issues correct  "Error getting provider context from provider manager: 11", cim indications and vcenter hardware monitoring
  • esxi host not usable until reboot, in absence of network connectivity
  • NTP package
  • windows 8 and windows server 2012 vm reboot fails
  • create more than 16TB of VMFS5 datastore on storage fails
  • boot on esxi 6 host on iSCSI SAN might fails
  • slow NFS storage performance in VM running in VSA provisioned NFS Storage
  • Many upgrade and installation issues
  • Many VM Management issues
  • VM Ware tools issues
  • PSOD in esxi host might happen with 40 or more VM running in a vsan cluster

 

For a full list refer to:

https://www.vmware.com/support/vsphere6/doc/vsphere-esxi-600b-release-notes.html?vmw_so_vex_mande_12

Have you ever found this issue running suspended VM in VMWare workstation?

 

 

 

 

After a VMWare workstation crash when you try to run or resume a  suspended VM could be a apper the above message.

This happens because process VMWare workstation can release the authorization lock to run VM; in fact this problem affect VM which are running before VMWare workstation crash.

 

To solve this issue simply:

  • Close VMWare workstation
  • run Task Manager and find vmware-vmx.exe process

 

  • Kill this process and start VMWare workstation again
  • Resume or start VM
 

 

In some cases it would be necessary to restart VMWare authorization service, simply doing this steps:
  • Close VMWare workstation

  • run as Administrator services.msc

  • Find process "VMware Authorization Service" and restart it
  • Start VMWare workstation again
  • Resume or start VM