VCloud Director 9.1 and CSE Kubernetes : Failure during Ubuntu Guest Customization

[fjmatos]

I´m testing  Vcloud Director Container Service Extesion for Kubernetes.

in a Vcloud Director 9.1GA .

I have followed this guide step by step.

https://vmware.github.io/container-service-extension/#installation

cse version 1.0.1 dev4

vcd-cli  version 20.2.0

python 3.6.4

the problem occurs  building the ubuntu template using cse install.

In the guest Customization phase of the ubuntu-temp vm,  the script fail with the following error:

waiting for process 1293 on vm 'vim.VirtualMachine:vm-7257' to finish (16)

exception, will retry in a few seconds, vm 'vim.VirtualMachine:vm-7257'

  exception: (vim.fault.GuestOperationsUnavailable) {

   dynamicType = <unset>,

   dynamicProperty = (vmodl.DynamicProperty) [],

   msg = 'The guest operations agent could not be contacted.',

   faultCause = <unset>,

   faultMessage = (vmodl.LocalizableMessage) []

}

Traceback (most recent call last):

  File "/root/.local/lib/python3.6/site-packages/vsphere_guest_run/vsphere.py", line 86, in execute_program_in_guest

    processes = pm.ListProcessesInGuest(vm, creds, [pid])

  File "/usr/lib/python3.6/site-packages/pyVmomi/VmomiSupport.py", line 580, in <lambda>

    self.f(*(self.args + (obj,) + args), **kwargs)

  File "/usr/lib/python3.6/site-packages/pyVmomi/VmomiSupport.py", line 386, in _InvokeMethod

    return self._stub.InvokeMethod(self, info, args)

  File "/usr/lib/python3.6/site-packages/pyVmomi/SoapAdapter.py", line 1366, in InvokeMethod

    raise obj # pylint: disable-msg=E0702

pyVmomi.VmomiSupport.vim.fault.GuestOperationsUnavailable: (vim.fault.GuestOperationsUnavailable) {

   dynamicType = <unset>,

   dynamicProperty = (vmodl.DynamicProperty) [],

   msg = 'The guest operations agent could not be contacted.',

   faultCause = <unset>,

   faultMessage = (vmodl.LocalizableMessage) []

There´s someone testing CSE , and  having similar problems? I think that the problem is in the code,

[V1rtualUser]

Hi, I am testing CSE and I have the exact same issue, only with the Ubuntu16.04 image. It's not installing VMware tools and can not perform

Guest operation inside the deployed VM.

waiting for process 1284 on vm 'vim.VirtualMachine:vm-134' to finish (46)

exception, will retry in a few seconds, vm 'vim.VirtualMachine:vm-134'

  exception: (vim.fault.GuestOperationsUnavailable) {

   dynamicType = <unset>,

   dynamicProperty = (vmodl.DynamicProperty) [],

   msg = 'The guest operations agent could not be contacted.',

   faultCause = <unset>,

   faultMessage = (vmodl.LocalizableMessage) []

}

Traceback (most recent call last):

  File "/root/.local/lib/python3.6/site-packages/vsphere_guest_run/vsphere.py", line 86, in execute_program_in_guest

    processes = pm.ListProcessesInGuest(vm, creds, [pid])

  File "/root/.local/lib/python3.6/site-packages/pyVmomi/VmomiSupport.py", line 580, in <lambda>

    self.f(*(self.args + (obj,) + args), **kwargs)

  File "/root/.local/lib/python3.6/site-packages/pyVmomi/VmomiSupport.py", line 386, in _InvokeMethod

    return self._stub.InvokeMethod(self, info, args)

  File "/root/.local/lib/python3.6/site-packages/pyVmomi/SoapAdapter.py", line 1366, in InvokeMethod

    raise obj # pylint: disable-msg=E0702

pyVmomi.VmomiSupport.vim.fault.GuestOperationsUnavailable: (vim.fault.GuestOperationsUnavailable) {

   dynamicType = <unset>,

   dynamicProperty = (vmodl.DynamicProperty) [],

   msg = 'The guest operations agent could not be contacted.',

   faultCause = <unset>,

   faultMessage = (vmodl.LocalizableMessage) []

  Anyone know of a fix for this?

Thanks

[jonathanw]

Yes, I have to fix this issue every time I (re)deploy the Ubuntu template, pretty sure I logged an issue on the CSE github, it got fixed for a while and has now broken again.

It appears to be caused by the template VM being restarted while packages are still being installed resulting in a corrupt package database.

If I remember rightly, the fix (well not really a fix, more a workaround is):

- Go through the install process and wait until that error appears.

- Log in manually to the ubuntu template VM (you need to grab the root password from the vCD guest properties).

- run 'dpkg --reconfigure' in the guest - it will come up with all sorts of warnings that you need to clear.

- Reinstall open-vm-tools (apt-get install open-vm-tools) - this usually won't work until you've fixed the dkpg issues.

- Once the open-vm-tools install has completed, you *may* need to do a 'systemctl start vmtoolsd' (can't remember offhand if this is done automatically by the install).

As soon as the open-vm-tools service is running again the CSE installation should continue and your template get generated correctly.

This is all from memory from the last time I rebuilt our templates, hopefully will be enough to get you up and running, but should probably be re-logged as an issue on the CSE github repo.

If you still have issues let me know and I'll re-run our Ubuntu template installation and write proper notes...