VMware Cloud Community
SeanKohler
Expert
Expert

Single Resource Inventory Update (Data Collection)

It has been a while since I looked at this, but has anyone discovered a way to do targeted data collection?  (e.g.  vm updated cpu/mem/disk in vsphere reflected in VRA without Reconfigure D2 operation)

daphnissov

Thanks,

~Sean

Reply
0 Kudos
8 Replies
daphnissov
Immortal
Immortal

Not sure what you're asking here, can you clarify?

Reply
0 Kudos
SeanKohler
Expert
Expert

Yeah thanks...

Here is the premise:

1. We cannot use the "Reconfigure" Day 2 Operation. (long story)

2. We created memory, cpu, and disk Day 2 operations in XaaS that hot add on the vcenter side. (Add/Remove Memory, Add/Remove CPU, Add/Remove/Change Disk)

3. Backend VRO workflow performs modifications to VM without issue.

4. The change isn't reflected in VRA until after daily (hourly whatever) data collection is run in VRA.

5. People are confused because VRA Catalog Resource Item shows a different value than the XaaS form (e.g. we can query the memory, cpu, and disk and provide the right value instantly) and a different value than the VM actually has ( e.g. VRA says 2GiB memory and VM has 4GiB)

6. We have a large number of VMs and growing.  In the beginning we could just trigger a FULL data collection to update VRA with correct stats.  That is no longer viable.

Figure if anybody knows anything, you all do...  or if you have target strategy for memory/cpu/etc update.  I was thinking of taking the values in a XaaS form and then calling a Reconfigure with the values using a service account that has entitlement to reconfigure in every business group.  That works for CPU and Memory... but it is a bit a problem for Disk as we manage SCSI controller in our existing XaaS form.  All along, we have merely wanted the ability to make VRA reflect recent change for a single item.  Because of the legacy nature of the IaaS portion of this product, we haven't had much luck in that sense.

Refreshing VM CPU/Memory values in GUI as part of ASD action

Manual "data collect" specific objects

Hi GrantOrchardVMware​.  How you been? Smiley Happy

Reply
0 Kudos
daphnissov
Immortal
Immortal

I get what you're saying but don't know of a way to run a selective data collection process on the target VM. That's just not how the system was built to work. I'm willing to bet a regular reconfigure action doesn't even run a selective data collect but probably updates the database(s) with the values specified in the reconfig parameters. That is, of course, just a guess, but if the premise turns out to be true then theoretically you could capture those database updates and build them into your XaaS workflow. All this seems like a lot of swings-and-roundabout motions to recreate the wheel (i.e., reconfigure). It may be simpler to figure out how you can use the built-in reconfigure day-2 action to avoid all that work. You mentioned it was a long story, but have you come to the conclusion that using the reconfigure built-in action is just not technically possible in your use case?

SeanKohler
Expert
Expert

Well I guess not particularly a long story... 

  • Big one is... Network configuration changes aren't allowed, and it appeared that they couldn't be excluded from Reconfigure? The NIC and IP you get during provisioning is the one you get... nothing more.
  • Disk size and count manipulation requires overcapacity in the blueprint with no managed placement with-in the VM  (SCSI bus, etc)
  • We do not extend blueprinting into other groups.  So if you want a machine with three vmdks, you either build the template VM with one disk and then perform day two operation to reconfigure disk layout (which also mounts and formats disks) or you can do it through XaaS add in form to composite blueprint on build.  (which essentially just runs its own day two operations POST)

disk.jpg

  • We have both Pets and Cattle in our VRA environment.  The pets need relatively standard mechanisms for dealing the vertical growth, and are maintained through ITIL process (change, etc).  Changes are targeted... e.g. Change Memory from 2-4 GB, and there was a concern that engineers would accidentally reconfigure other items.  (I know... I am not saying I agree with any of this... lol)

I don't think these concerns or this practice is uncommon.  This is why we see others using the VRO workflow to call a Reconfigure with property values.  They are building custom XaaS Day 2 operations to only manipulate the configuration items they want out of "Reconfigure" without actually exposing the Reconfigure action to end users.  Unfortunately, doing so creates its own challenges around entitlements. (for Reconfigure can only run as an account entitled to do so... usually a service account or other form of privileged user)

In our chosen path, we can have pretty compelling forms to update content with conditional fields (like no Change Req number for a Cattle machine, but yes for a Pet) and have targeted automation work against the backend VM to handle the condition change (grow the disk, mount the disk, format the disk, etc).  But this is at the expense of leaving VRA data behind until Data Collection.  We actually have thought a lot about direct database writes, but that isn't sustainable through upgrades and could present stability and support concerns.

Superb dialog though.  Much appreciated. I like having these discussions. If anybody has more thoughts, please do share.

(my coworker BW wrote this one--credit where it's due)

addDisk.jpg

Reply
0 Kudos
daphnissov
Immortal
Immortal

Ok, maybe it was more complex after all Smiley Happy

  • Big one is... Network configuration changes aren't allowed, and it appeared that they couldn't be excluded from Reconfigure? The NIC and IP you get during provisioning is the one you get... nothing more.

Well, in the case of network configuration changes (assuming different network selection here?), that's not really a "reconfigure" but a "redeploy" activity. There are lots more implications when it comes to changing network configuration that it does to rebuilding what was already built or adding vCPU, vRAM, etc. Lifecycle stages are invoked for possible extensibility, and that's not something a reconfigure is designed to do. And I have this conversation with customers fairly regularly where they think of vRA as just a public web portal to vCenter and expect to do anything and everything an admin can do with a VM after the fact. It's just not how cloud works and sometimes you have to redeploy.

  • Disk size and count manipulation requires overcapacity in the blueprint with no managed placement with-in the VM  (SCSI bus, etc)
  • We do not extend blueprinting into other groups.  So if you want a machine with three vmdks, you either build the template VM with one disk and then perform day two operation to reconfigure disk layout (which also mounts and formats disks) or you can do it through XaaS add in form to composite blueprint on build.  (which essentially just runs its own day two operations POST)

One of the ways I've dealt with this is to assign like individual 1 GB disks to the template but to not bring them online. This way, the SCSI controller and IDs you decide but for all intents and purposes they're useless unless extended, at which point you can select a preexisting disk with those attributes set. It doesn't help the capacity requirements in the blueprint, but again, my response is similar to the networking scenario. Blueprints are purpose-built for something and not just a blank check to whatever the user wants. This is really where IT process and governance come in.

  • We have both Pets and Cattle in our VRA environment.  The pets need relatively standard mechanisms for dealing the vertical growth, and are maintained through ITIL process (change, etc).  Changes are targeted... e.g. Change Memory from 2-4 GB, and there was a concern that engineers would accidentally reconfigure other items.  (I know... I am not saying I agree with any of this... lol)

That's an interesting one. So management was scared people with access to both pets and cattle deployments would...reconfigure the wrong ones??

I don't think these concerns or this practice is uncommon.  This is why we see others using the VRO workflow to call a Reconfigure with property values.  They are building custom XaaS Day 2 operations to only manipulate the configuration items they want out of "Reconfigure" without actually exposing the Reconfigure action to end users.  Unfortunately, doing so creates its own challenges around entitlements. (for Reconfigure can only run as an account entitled to do so... usually a service account or other form of privileged user)

This does happen sometimes, yes, but I see it more frequently done where they are trying to work around the limitations of reconfigure (due to vRA) and not trying to essentially re-work a wheel. Because very often the tasks being done in vRO are basically the exact same thing, but yet by circumventing vRA they now have a whole mess of coding to accomplish to make things copacetic, and that has implications on day 2 as well of vRA (vRA patching? upgrading? vRO upgrading? plug-in upgrading? How much is going to break now?).

I don't think anything I'm saying you don't already know, nor do I think it'll help in your original question and issue. I don't know if anything will help to further that mission, by the way, short of those database injections, but your points are all valid.

SeanKohler
Expert
Expert

Well, in the case of network configuration changes (assuming different network selection here?), that's not really a "reconfigure" but a "redeploy" activity.

Right!  Exactly. In the case of the D2 Reconfigure action, it allows you to change the network settings.  So the team fully anticipated that people would just go ahead and try that... muck everything up for their machine and maybe become a noisy neighbor in an IP schema that is automatically managed.

One of the ways I've dealt with this is to assign like individual 1 GB disks to the template but to not bring them online.

Clever.  I agree that new cloud models means shedding old ways.  (try selling that to my ITSM team though... heheh)

[Re Pets/Cattle... ] That's an interesting one. So management was scared people with access to both pets and cattle deployments would...reconfigure the wrong ones??

Reconfigure ones that require change control without proper controls in place (mandatory change numbers, etc)

I don't think anything I'm saying you don't already know, nor do I think it'll help in your original question and issue. I don't know if anything will help to further that mission, by the way, short of those database injections, but your points are all valid.

You have been tremendously helpful.  Thank you.  Way more helpful than GrantOrchardVMware​.  Smiley Happy (let's see if that gets him over here,,, lol)

Reply
0 Kudos
mbomb67
Contributor
Contributor

I had been looking for a way to do this for some time and actually had some success with it today... There are a lot of things that have not been taken in to account in this sample, and we had lots of issues in 6.x when we were forcing data collections of a single cluster, so I'm not entirely convinced that I will end up using this in production without quite a bit more polishing. Basically, I was able to dissect the OOB "Force data collection" workflow to filter it to only execute on the cluster (vCAC:HostMachine) specified. I can confirm that I was able to run a data collection in our Dev environment successfully. Hopefully this helps:

var modelName = 'ManagementModelEntities.svc';
var entitySetName = 'DataCollectionStatuses';

//Read a list of entities
var entities = vCACEntityManager.readModelEntitiesByCustomFilter(host.id, modelName, entitySetName, null, null);


for each (entity in entities) {
 if (entity.getProperty("EntityID") == cluster.hostId) {
  var filterSpec = entity.getLink(host,"FilterSpec")[0];
  var filterSpecGroup = filterSpec.getLink(host,"FilterSpecGroup")[0];
  var filterSpecGroupName = filterSpecGroup.getProperty("FilterSpecGroupName");
  if (filterSpecGroupName == "inventory") { //Other valid options: state, performance, inventory, VCNSInventory
   System.log("filterSpecGroupName: " + filterSpecGroupName);
   var entityKey = entity.keyString;
   System.log("Updating entity with key: " + entityKey);
   var links = null;
   var headers = null;
   var updateProperties = {
    "LastCollectedTime":null
   };
   vCACEntityManager.updateModelEntityBySerializedKey(host.id, modelName, entitySetName, entityKey, updateProperties, links, headers);
   break;
  }
 }
}

Reply
0 Kudos
mbomb67
Contributor
Contributor

My screenshot apparently didn't make it in to the original reply:

Reply
0 Kudos