VMware Cloud Community
Czernobog
Expert
Expert

vRA 8.4 - add variable number of network interfaces to a VM

I want to create a vSphere VM blueprint for the following simple scenario:
A user can submit a request for a vSphere VM, with a variable number (1 to 2) of network interfaces. The interfaces can be connected to different NSX segments.

input:
selectedNetworks:
type: array
title: networks
items:
title: network
type: object
properties:
networkPlacement:
type: string
enum:
- title: Zone 1
- title: Zone 2
- title: Zone 3
default: Zone 1
maxItems: 2

As for now I have found no way to successfully create a blueprint that would fit the requirement.
One workaround is to pre-define 2 interfaces, attach all of them to the VM and later change the port group assignment, like here:

resources:
Cloud_vSphere_Machine_1:
type: Cloud.vSphere.Machine
properties:
(...)
networks:
- network: '${resource.Cloud_NSX_Network_1.id}'
- network: '${resource.Cloud_NSX_Network_2.id}'
Cloud_NSX_Network_1:
type: Cloud.NSX.Network
properties:
constraints:
- tag: '${input.selectedNetworks[count.index].networkPlacement}'
Cloud_NSX_Network_2:
type: Cloud.NSX.Network
properties:
constraints:
- tag: '${input.selectedNetworks[count.index].networkPlacement}'

But this way the VM gets deployed with more interfaces than needed, in most cases. I want to attach exactly the number of interfaces that the user provides a configration for.
I have also tried to do a similar construct as with attaching multiple disks, that is, defining an input of type Array of Objects and conditionaly, based on the length of the array, and add single networks to the VM:

resources:
Cloud_NSX_Network:
type: Cloud.NSX.Network
properties:
constraints:
- tag: '${input.selectedNetworks[count.index].networkPlacement}'
count: '${length(input.selectedNetworks)}'
Cloud_vSphere_Machine:
type: Cloud.vSphere.Machine
properties:
(...)
networks:
- network: '${length(input.selectedNetworks) >= 1 ? resource.Cloud_NSX_Network[0].id : null}'
- network: '${length(input.selectedNetworks) >= 2 ? resource.Cloud_NSX_Network[1].id : null}' # error in editor on this line

Here I am receiving an error in the template editor: "Index 1 out of bounds for length 1". I assume this means that Cloud_NSX_network is somehow wrong defined, as a single entity, it should be an array though.

I've also tried it with:

- network: '${resource.Cloud_NSX_Network[count.index].id}'

But here I get an error during test execution: Error while validating count property: Value 2 is greater than allowed maximum value 1

Can someone please demonstrate a valid way of dynamically adding multiple network interfaces during deployment?

Labels (1)
Reply
0 Kudos
9 Replies
emacintosh
Hot Shot
Hot Shot

Not sure if there is a better way yet, but we do something similar today.  I posted out here a bit ago about it, if you want to read that post.

Adding/Removing NICs Before Provisioning 

We have 3 nics on our blueprint.  Depending on the scenario, we may only need 1 or 2.  So on our form, we have a hidden input that we use to populate a json string representing our nic configuration (and constraints too) based on user input - that way we can ensure the right device index ourselves based on our logic.

We then use that input in the cloud template to define whether the nic should be included or not - specifically using the count property.  When using the map_by function, if the count=0, the nic will not be included when built.

So here is an example of what that nic json might look like and the relevant cloud template sections.  In this case, Nic2 would not be included.  Hope this helps a little.

emacintosh_3-1620160054369.png

 

emacintosh_2-1620159984000.png

 

 

 

Czernobog
Expert
Expert

Thanks for taking the time to answer!

So in the provided example 2 NICs will get attached, on index 0 and 2, right? Did you to the background calculation for the hidden parameter in the custom form? Does it still work when updating the deployment, when you want to change the NIC configuration?

As for the input - you give your users the option to select a network for the attached interfaces, from a predefined list (3 nics) and where they do not add a network - the array element is ignored (set count to 0)? Or is it an expandable array?

I need some time to wrap my head around it and adapt your solution to my needs.

Reply
0 Kudos
emacintosh
Hot Shot
Hot Shot

Ah, so in my example Nic2 would not have been created.  So we would have Nic1 and index0 and Nic3 at index 1 (those aren't there real names, just lazy names for this example)

Yes, we use a vRO action to determine the nic configuration (as well as machine constraints) and populate the hidden field on the form.  But we don't let users choose how many nics, instead that is determined by business logic.  In our case, each of those nics has specific purpose and whether they are needed can be determined by other things the user selects on the form.

For day-2 activities, we have custom workflows to allow the users to add/remove/change nics.  Those resource actions are attached to the vsphere machine resource.  We do not let users interact with the network resources directly in the deployment.  It's a somewhat complicated approach, but a lot of the oob functionality in vRA does work well in our environment.

If it's helpful, here's a watered down version of the nic part of that action.  Not shown here, but at the end we use JSON.stringify() to send a json string back to that hidden input.  Which is then used in our cloud template with all of those from_json() calls to get at the actual properties.  

There is probably a better approach, but this is where we landed in 8.1 and haven't had a reason to revisit just yet.

var nic1 = {"type":"nic1","count":1};
var nic2 = {"type":"nic2"};
var nic3 = {"type":"nic3"};

// array to store nics
var nics = [];

// update nics based on business logic
if (someBusinessLogic) {
    nic2.count = 1
}
else {
    nic2.count = 0;
}

if(someBusinessLogic) {
    nic3.count = 1;
}
else {
    nic3.count = 0;
}

// add nics to array based on some business logic
if(someBusinessLogic) {
    nics.push(nic1,nic3);
}
else {
    nics.push(nic1,nic2,nic3);
}

// Set device indexes appropriately, ignoring count=0 nics 
deviceIndex = 0;
for(var i = 0;i < nics.length;i++) {
    if(nics[i].count == 1) {
        nics[i].index = deviceIndex;
        deviceIndex++;
    }
}

 

Czernobog
Expert
Expert

I've tried a few other approaches, including one that leverages vRO for the network configuration like in your example, but I stillwanted to keep the logic in the template, as much as possible. Here's the result:

 

input:
  (...)
  nsxSegmentArray: # hidden! # selects nsx segment tag based on other net* inputs; value computed by vRO action
  type: array
  items:
    type: string
    title: nsxSegmentTag

resources:
  Cloud_vSphere_Machine:
    type: Cloud.vSphere.Machine
    properties:
      (..)
      networks: '${map_by(resource.NIC_0[*] + resource.NIC_1[*],nic => {"network":nic.id})}' # left out deviceIndex
  NIC_0:
    type: Cloud.NSX.Network
    properties:
      count: '${length(input.nsxSegmentArray) >= 1 ? 1 : 0}'
      constraints:
        - tag: '${input.nsxSegmentArray[count.index]}'
  NIC_1:
    type: Cloud.NSX.Network
    properties:
      count: '${length(input.nsxSegmentArray) >= 2 ? 1 : 0}'
      constraints:
        - tag: '${input.nsxSegmentArray[count.index]}'   

 

This attaches a variable number of network interfaces, based on the lenght of the input array.

Previously I wanted to force creating a clustered resource of the Cloud.NSX.Network type, and failed at the count value:

 

  Cloud_NSX_Network:
    type: Cloud.NSX.Network
    properties:
      constraints:
        - tag: '${input.nsxSegmentArray[count.index]}'
      count: '${length(input.nsxSegmentArray)}'

 

This solution will work, but only for a maximum of 1 network interface (input array lenght = max 1). Otherwise it will throw a vague error like "Value 2 is greater than allowed maximum value of 1". This references the network resource and tells you that it basically cannot be clustered, in contrast to the Disk reosurce, which can be clustered.

 

I still need to check, if the reosurce can be updates correctly since one issue I have noticed in vRA 8.4, which was present in past releases, is, that when you do a value calculation in the user form, based on an external source like a vRO action, this calculation is not done when you update the resource after deployment.

Czernobog
Expert
Expert

Here's the "final" version of the template, I have no more patience to put further work into this, at least for some time (see reasons below).
The idea is to allow a user to select a "network zone" for the adapters attached to the requested vm. So adapter 1 could be attached to a private network, another one to a public network. At some point the user should be able to remove or change the network assignment, without redeplyoing the wohle system. Since network resource itself (here: Cloud.NSX.Network) cannot be edited by itself post deplyoment (actions are greyed out), the only way to change the configuration is to use the Update action on the deployment.

Adding new adapters with custom network assignments works now during the initial deployment, except for setting the device index - no matter how I define the value, the index is not set to what I want and instead vRA just increments it, depending on the number of adapters. So for example when I add only nic0 and nic2, I'd expect for the device indexes to be 0 and 2, but it is 0 and 1.
Here are the inputs, I'm showing only one of four to keep the post short:

 

inputs:
nic0:
type: string
enum:
- Zone 1 # 4 options: Zone 1-3 or not_connected
- Zone 2
- Zone 3
- not_connected # 'null' value cannot be used!
default: Zone 1
(nic1-nic3 below...)

# below are option specific for my environment, probably not universally applicable
nicSelectArray:
type: array # hidden! build string array (,) based on net assignment of nic0 to nic3. passed on to vRO action to create array (no built in custom form field allows this)
items:
type: string
nsxSegmentArray:
type: array # hidden! computed by vRO action, based on nicSelectArray value(s); returns nsx network (segment) tags, so the nics get correctly attached
items:
type: string
title: nsxSegmentTag

 


Ande here are the resources:

 

resources:
NIC_0:
  type: Cloud.NSX.Network
    properties:
      count: '${input.nsxSegmentArray[0] != "not_connected" ? 1 : 0}' # if value is 'not_connected', set count to 0 with results in not provisioning this resource
      deviceIndex: 0
      recreateOnUpdate: false
    constraints:
      - tag: '${input.nsxSegmentArray[0]}' # nsx segment tag the vm network adapter is attached to
(NIC_1-NIC3 below...)

 

Network resource assignment to the Cloud.vSphere.Machine resource:

 

networks: '${map_by(resource.NIC_0[*] + resource.NIC_1[*] + resource.NIC_2[*] + resource.NIC_3[*],nic => {"network":nic.id, "deviceIndex":nic.deviceIndex})}'

 

This works (except for deviceIndex) during the initial deplyoment ONLY. Some of the required field values are computed by external vRO actions, but the custom form seems to be bugged when using the Update action. Or maybe this is how it is intended to work? That would make no sense and makes the whole construct really pointless anyway.


I would have to try to switch from using dynamic values in the custom form to just a plain input array or object, like in your example, do all calculations in vRO and assign them durign one of the allocation or configuration phases.

Reply
0 Kudos
emacintosh
Hot Shot
Hot Shot

Ah interesting, it seems like maybe you want to hardcode device indexes for nics where we want them to be dynamically determined in a certain order.  Assuming staggered indexes are legal, pretty disappointing that vRA just ignores your values.

Also, we have only been using Update as a poor-man's versions of previously available (and deeply missed) resubmit from vra 7.x  For changes to NICs, Disks, Memory, CPU, etc. we write our own day-2 actions (resource actions).  Maybe with future versions of vRA that won't be as necessary, but pretty much everything oob really doesn't fit our needs/env.

Reply
0 Kudos
Czernobog
Expert
Expert


Also, we have only been using Update as a poor-man's versions of previously available (and deeply missed) resubmit from vra 7.x

It makes me happy to see, that I am not the only one. I've even asked about reintroducing this feature in the past.

Using Update to Reprovision might be ok if you could reconfigure other resources than the VM and Disks directly. I don't really get why this is made impossible - why even provide an action menu, when its content is empty? That option would save the trouble with updating the whole deployment - you could just assign a new tag or some other property to the network resource. As an alternative, you can set "recreateOnUpdate: false" and create a custom "Reprovisioning" action.

Reply
0 Kudos
Altkeymist
Contributor
Contributor

Spoiler
Did you ever get this sort it out? This was my solution to the issue regarding deviceIndex. However this solution limits the number of dynamic nics that you can add to vm. 

networks: |-
${
map_by(resource.network_1[*].id, id => { "deviceIndex": 0, "network": id, "assignment": "static" }) +
map_by(resource.network_2[*].id, id => { "deviceIndex": 1, "network": id, "assignment": "static" })
}

Reply
0 Kudos
Czernobog
Expert
Expert

We did not go in production with vRA yet, due to the shortcomings. Instead of the "fully" dynamic network adapter attachment we went with attaching 4 adapters at deployment time and later only changing the network association. This allows updating the configuration without rebuilding the whole deployment, it works only with a vSphere vm though.

The workaround introduced another challenge though - since we have four adapters now, they have to be assigned to a vRA network profile, or the deployment would not finish. For this I have created a dummy network, which in my case is a simple NSX segment with only layer 2 connectivity. A vRA network profile with a configured ipv4 range is backed by this segment and whenever one of the vms' network adapters is not in use, the user can select the "not connected" profile when editing the deployment. This allows us to use the "assignment: static" property on all adapters, which the IPAM solution also requires, otherwise we would have to change the property value from static to dynamic or back again, when the adapter configuration changes - that would result is a rebuild of the deployment, which we want to avoid.

One other minor problem I have found here is, that when assigning both an ipv6 and ipv4 range to the network and network profile in vRA, addresses from the ipv6 network are not assigned; I have generated myself a RFC 4193-conform network range and selected a /64 range from it. When you try to assign a ipv6 range only, the deployment fails with the error "No placement exists that satisfies all of the request requirements."

Anyway here are the current inputs and resource configuration from my blueprint:

INPUTS:

nic0:
    type: string
    title: adapter 0
    oneOf:
      - title: Zone1
        const: z1
      - title: Zone2
        const: z2
      - title: not in use
        const: dummynetwork
    default: z1
  nic1:
    type: string
    title: adapter 1
    oneOf:
      - title: Zone1
        const: z1
      - title: Zone2
        const: z2
      - title: not in use
        const: dummynetwork
    default: dummynetwork
  nic2:
    type: string
    title: adapter 2
    oneOf:
      - title: Zone1
        const: z1
      - title: Zone2
        const: z2
      - title: not in use
        const: dummynetwork
    default: dummynetwork
  nic3:
    type: string
    title: adapter 3
    oneOf:
      - title: Zone1
        const: z1
      - title: Zone2
        const: z2
      - title: not in use
        const: dummynetwork
    default: dummynetwork

RESOURCES:

  Cloud_vSphere_Machine:
    type: Cloud.vSphere.Machine
    properties:
      networks:
        - network: ${resource.Network0.id}
          deviceIndex: 0
          assignment: static
        - network: ${resource.Network1.id}
          deviceIndex: 1
          assignment: static
        - network: ${resource.Network2.id}
          deviceIndex: 2
          assignment: static
        - network: ${resource.Network3.id}
          deviceIndex: 3
          assignment: static

  Network0:
    type: Cloud.NSX.Network
    properties:
      networkType: existing
      constraints:
        - tag: netscope:private
        - tag: netzone:${input.nic0}
  Network1:
    type: Cloud.NSX.Network
    properties:
      networkType: existing
      constraints:
        - tag: netscope:private
        - tag: netzone:${input.nic1}
  Network2:
    type: Cloud.NSX.Network
    properties:
      networkType: existing
      constraints:
        - tag: netscope:private
        - tag: netzone:${input.nic2}
  Network3:
    type: Cloud.NSX.Network
    properties:
      networkType: existing
      constraints:
        - tag: netscope:private
        - tag: netzone:${input.nic3}

 

Reply
0 Kudos