VMware Cloud Community
benjamin000
Enthusiast
Enthusiast
Jump to solution

Adding KVM Hypervisor

As VIO cannot be used for "proper" OpenStack usage due to there begin no Images that can be used for OpenStack and these images be used for SSH and/or Password Injection or can Instances be resized when using templates from vCenter. This overrides everything OpenStack and we have mistakenly installed and configured VIO and deployed over 180 Instances so I guess we are stuck with it.

To overcome this we want to install another hypervisor namesly KVM so we can use this hypervisor for the Instances that require resizing and other proper OpenStack options.

Can another hypervisor we added to an existing VIO deployment? IS it possible to deploy KVM in a host and then add that host to VIO ??

It is unfortunate that VMware do not provide Images and do not provide guides on using Images so that customers can use OpenStack with all the features of KVM.... I believe with the money that has been outlayed for NSX and all the other costs it would be nice to be able to use VIO in its fullest capacity. Maybe we are missing something but not having Linux Images that can be used for resizing, SSH Key injection is a massive downfall....If someone has a solution we would even be willing to pay handsomely for someone to create Ubuntu, CentOS and Debian images yet from what we have researched I do not think it is possible.

So if adding a KVM hypervisor to VIO is possible this would solve our issue as we can keep our normal Instances and new Linux Instances we can use KVM so customers can use SSH keys and benefit from image resizing.

Regards Ben McGuire
1 Solution

Accepted Solutions
benjamin000
Enthusiast
Enthusiast
Jump to solution

It appears that the only way to get this working was to convert the Openstack Cloud image BEFORE uploading to glance.

Convert via qemu-img and make sure that when uploading that "IDE" is used as qemu-img converts to IDE.

I am not sure why VIO did not convert properly but for those with the same issue just convert the OpenStack Image BEFORE uploading to glance.

Regards Ben McGuire

View solution in original post

Reply
0 Kudos
22 Replies
mvoelker
VMware Employee
VMware Employee
Jump to solution

First, are you aware that you can use most qcow2 and raw images with VIO?  Simply use the glance-import tool to import them into your cloud.  This allows you take advantage of lots of images from the ecosystem.  Check out the documentation here:

VMware Integrated OpenStack Documentation

Alternately, images can be converted manually using qemu-tools if you prefer not to use an import task for some reason.  To do this, have a look here:

OpenStack Docs: Converting between image formats

Any image with cloud-init should support keypair injection for SSH access, and you can also use cloud-init to inject passwords.  See for example:

Changing Passwords with cloud-init · Scott's Weblog · The weblog of an IT pro specializing in virtua...

For resize, the standard OpenStack resize API should work:

OpenStack Docs: Change the size of your server

Just note that you'll want to detach Cinder volumes before resizing and then reattach them after.

benjamin000
Enthusiast
Enthusiast
Jump to solution

Hello and thank you for the reply. Let me review and try some of your advice.

In saying this I have imported the qcow2 cloud images for ubuntu and others using both horizon and the cli but they do not boot when adding to VIO. I am an advid supporter of VIO as I am a VMware junkie but not having these images work is affecting the workings of my VIO deployment. Currently I use vSphere templates and link then into when creating and glance image and use temp password but recently have noticed the resizing Instances so not work this way nor does SSH Key injection. It is my understanding that VMware does not support injection but maybe with VIO it does.

Once I review your reply I will give it a go at 1. importing qcow2 images and boot and I will give you the error and 2. I will build my own using qemu tools which I have already done and received an error when using a Ubuntu Cloud Image ( img ) and using qemu to convert to vmdk even though glance converts it anyway. I will post the screenshot of the error when trying to boot that image. I have tried with LSIlogic and all other adaptors.

Ill be in touch tomorrow once I try things as it is better knowing that it CAN be dones rather than me hoping it can be done.

Thank you once again and hopefully I can share positive remarks tomorrow once I review everything.

Regards Ben McGuire
Reply
0 Kudos
benjamin000
Enthusiast
Enthusiast
Jump to solution

Just a quick update

I imported the Ubuntu 16.10.vmdk file from the Ubuntu Cloud Image site and when trying to launch an Instance I receive " No Valid Hosts Were Found"

It is my understanding that these images are ONLY for KVM and hence cannot be used in VIO with the esxi hypervisor.

And all the links you provided to have been to before I I have spent the past 12 months trying to find a solution to this. If you want to prove me wrong and create and I am happy to pay anyone for their time. Just PM me and we can talk.

Regards Ben McGuire
Reply
0 Kudos
xgao3
VMware Employee
VMware Employee
Jump to solution

Can you capture output of:

  • glance image show <image id>
  • nova scheduler log from both controllers.  Nova scheduler log should be /var/log/nova/nova-scheduler.log on the controller node.
Reply
0 Kudos
xgao3
VMware Employee
VMware Employee
Jump to solution

sorry 1st command is openstack image show <image id>

Reply
0 Kudos
benjamin000
Enthusiast
Enthusiast
Jump to solution

Hello Again. Thanks. Here is the first... Ill provide the other logs next

root@loadbalancer01:~# openstack image show 60ea1c30-a198-4b0e-88d7-1ae5bfbd33d3

+------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

| Field            | Value                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |

+------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

| checksum         | None                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |

| container_format | bare                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |

| created_at       | 2017-06-10T15:58:13Z                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |

| disk_format      | vmdk                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |

| file             | /v2/images/60ea1c30-a198-4b0e-88d7-1ae5bfbd33d3/file                                                                                                                                                                                                                                                                                                                                                                                                                                                   |

| id               | 60ea1c30-a198-4b0e-88d7-1ae5bfbd33d3                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |

| min_disk         | 0                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |

| min_ram          | 0                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |

| name             | Ubuntu 17.04 Cloud                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |

| owner            | 54b307bd475d4f359953d892e650b3ab                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |

| properties       | direct_url='vi://vcenter.XXXXXXXXXXX.local/Openstack Datacenter/vm/OpenStack/Project (54b307bd475d4f359953d892e650b3ab)/Images/60ea1c30-a198-4b0e-88d7-1ae5bfbd33d3?managed=True', locations='[{u'url': u'vi://vcenter.XXXXXXXXXXX.local/Openstack Datacenter/vm/OpenStack/Project (54b307bd475d4f359953d892e650b3ab)/Images/60ea1c30-a198-4b0e-88d7-1ae5bfbd33d3?managed=True', u'metadata': {}}]', vmware_adaptertype='lsiLogic', vmware_disktype='streamOptimized', vmware_ostype='ubuntu64Guest' |

| protected        | False                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |

| schema           | /v2/schemas/image                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |

| size             | 330916864                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |

| status           | active                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |

| tags             |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |

| updated_at       | 2017-06-10T16:00:29Z                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |

| virtual_size     | None                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |

| visibility       | private                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                |

+------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

root@loadbalancer01:~#

Regards Ben McGuire
Reply
0 Kudos
benjamin000
Enthusiast
Enthusiast
Jump to solution

Here is the last part of the scheduler log after I try and launch an instance with the Ubuntu 17.04 Cloud vmdk image

cheduler/host_manager.py:177

2017-06-10 16:08:49.730 4975 DEBUG oslo_concurrency.lockutils [req-0a35897e-5dc4-43c7-93ec-09804e0d289b 4a45cf83e6e54826b2fddc29bb385a71 54b307bd475d4f359953d892e650b3ab - - -] Lock "(u'compute01', u'domain-c47.c0c95b3e-0d6e-41c1-bac1-8c6953faed71')" released by "nova.scheduler.host_manager._locked_update" :: held 0.132s inner /usr/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:285

2017-06-10 16:08:49.730 4975 DEBUG nova.filters [req-0a35897e-5dc4-43c7-93ec-09804e0d289b 4a45cf83e6e54826b2fddc29bb385a71 54b307bd475d4f359953d892e650b3ab - - -] Starting with 1 host(s) get_filtered_objects /usr/lib/python2.7/dist-packages/nova/filters.py:70

2017-06-10 16:08:49.731 4975 INFO nova.scheduler.filters.retry_filter [req-0a35897e-5dc4-43c7-93ec-09804e0d289b 4a45cf83e6e54826b2fddc29bb385a71 54b307bd475d4f359953d892e650b3ab - - -] Host [u'compute01', u'domain-c47.c0c95b3e-0d6e-41c1-bac1-8c6953faed71'] fails.  Previously tried hosts: [[u'compute01', u'domain-c47.c0c95b3e-0d6e-41c1-bac1-8c6953faed71']]

2017-06-10 16:08:49.731 4975 INFO nova.filters [req-0a35897e-5dc4-43c7-93ec-09804e0d289b 4a45cf83e6e54826b2fddc29bb385a71 54b307bd475d4f359953d892e650b3ab - - -] Filter RetryFilter returned 0 hosts

2017-06-10 16:08:49.732 4975 DEBUG nova.filters [req-0a35897e-5dc4-43c7-93ec-09804e0d289b 4a45cf83e6e54826b2fddc29bb385a71 54b307bd475d4f359953d892e650b3ab - - -] Filtering removed all hosts for the request with instance ID '4c796ec0-8b1b-4a5f-93c3-bff44c210bfd'. Filter results: [('RetryFilter', None)] get_filtered_objects /usr/lib/python2.7/dist-packages/nova/filters.py:129

2017-06-10 16:08:49.732 4975 INFO nova.filters [req-0a35897e-5dc4-43c7-93ec-09804e0d289b 4a45cf83e6e54826b2fddc29bb385a71 54b307bd475d4f359953d892e650b3ab - - -] Filtering removed all hosts for the request with instance ID '4c796ec0-8b1b-4a5f-93c3-bff44c210bfd'. Filter results: ['RetryFilter: (start: 1, end: 0)']

2017-06-10 16:08:49.733 4975 DEBUG nova.scheduler.filter_scheduler [req-0a35897e-5dc4-43c7-93ec-09804e0d289b 4a45cf83e6e54826b2fddc29bb385a71 54b307bd475d4f359953d892e650b3ab - - -] There are 0 hosts available but 1 instances requested to build. select_destinations /usr/lib/python2.7/dist-packages/nova/scheduler/filter_scheduler.py:71

2017-06-10 16:09:23.587 4975 DEBUG oslo_service.periodic_task [req-7c7c0491-6bf1-4f56-9548-06c11c530362 - - - - -] Running periodic task SchedulerManager._expire_reservations run_periodic_tasks /usr/lib/python2.7/dist-packages/oslo_service/periodic_task.py:215

2017-06-10 16:09:42.395 4975 DEBUG oslo_service.periodic_task [req-7c7c0491-6bf1-4f56-9548-06c11c530362 - - - - -] Running periodic task SchedulerManager._run_periodic_tasks run_periodic_tasks /usr/lib/python2.7/dist-packages/oslo_service/periodic_task.py:215

Regards Ben McGuire
Reply
0 Kudos
benjamin000
Enthusiast
Enthusiast
Jump to solution

All my other images lauch just fine but those images I use as templates from vcenter and just create a link into glance but those images cannot use SSH Keys otr resizing does not work.... to resize i need to boot with gparted and do everything manually and the bad thing about that is it allocated the full storage rather than use thin disk.

Regards Ben McGuire
Reply
0 Kudos
benjamin000
Enthusiast
Enthusiast
Jump to solution

I was also looking at the vcenter tasks and the error there is interesting. I have attached the screenshot

Regards Ben McGuire
Reply
0 Kudos
benjamin000
Enthusiast
Enthusiast
Jump to solution

Hello

I have spoken to Trevor Roberts Jr from VMware and he said the qcow2 images will work as they will be converted upon upload. I have tried all distros and now in every distro I receive the error on boot " Operating System Not Found". I have attached a screenshot and from this it appears to be trying to boot from the network and not the HDD.

Regards Ben McGuire
Reply
0 Kudos
mvoelker
VMware Employee
VMware Employee
Jump to solution

Perhaps it would be useful if I stepped through a typical workflow or two so we can check if there are differences.  I'll be using a VIO 3.1 lab for the screenshots below.  I'll start by grabbing a generic Ubuntu 14.04 image in qcow2 format from Ubuntu's cloud image download site:

1.png

The first method we'll try is the import task method I mentioned earlier.   When you create an import task, you basically tell Glance "please go get the following image from the specified location and convert it for me".  That means it needs the image to be reachable via HTTP.  To take care of that, I'll just use a Python one-liner to temporarily start a webserver on port 12345 of the machine I downloaded the image to (you could also use a real HTTP server of course):

2.png

That done, we're ready to create the import task.  Notice here that I'm setting the adaptertype and disk type in the process.  An improperly set adapterType is the most common cause of the "Operating system not found" error when booting an instance (if you don't have it set on an image, you can add that property to it via the CLI, API, or Horizon without recreating the image):

3.png

This will take a few minutes to process and Glance downloads the image and converts it for you.  You can monitor progress via Glance or the CLI.  Once it's done, you should see the image available with all the proper attributes set:

4.png

And we can try booting it:

5.png

Since you asked about passwords here, I'll go ahead and use cloud-init to set the password for the Ubuntu user:

6.png

Once I click the Launch button, the image will start spawning:

7.png

Once it's done booting, I can use the Horizon console to verify that Ubuntu started up properly and shows me a login prompt:

8.png

Next, I'll try logging in with the credentials I set via cloud-init:

9.png

Success!

10.png

So, the import method appears to be working well here.  I may want to pre-convert my image to a VMDK instead though.  To see how that works, let's go back to the machine I downloaded the image to in the first place and use qemu-img to convert it (note: you'll need to "sudo apt-get install qemu-utils" if you haven't already).  Notice here that after I run qemu-img, I also run a printf command.  This is due to the fact that in this lab I'm using an older version of qemu tools which had a bug when creating a streamOptimized image.  Off the top of my head, I believe the fix we sent upstream for that got merged in version 2.5 (but don't quote me on it; I'm just working from memory here...if you're curious I'll be happy to check and get back to you):

11.png

I can then upload the image to Glance.  This time I'll just use the "openstack image create" command since we don't need to use an import task (although we could, if we wanted--import tasks don't have to convert between formats; it's just something useful they can do):

12.png

Once the image has been created, I'll boot a server from it.  Notice I used the config drive option here--that's not necessary (we didn't use it when booting the last instance); I was simply in the habit of typing it since I was working on something involving config drives earlier in the day.  Once I had the screenshot and I realized I'd used it, I figured we may as well leave it in there to demonstrate that config drive also works. 😃

13.png

And again, we'll switch over to Horizon to verify that Ubuntu started up correctly:

14.png

You also asked about resize, so let's take a look at that too.  First, we verify that the first instance we booted is an m1.small with 1vCPU, 2GB RAM, and 20GB of disk.  Then we'll click on "Resize Instance" in the dropdown menu on the right:

15.png

We'll ask for the instance to be resized to an m1.large with 4 vCPU's, 8GB of RAM, and 80GB of disk:

16.png

The instance should start resizing:

17.png

Eventually we'll be asked to confirm the resize:

18.png

Once we click the confirm button, we're finished and can see that the instance listed as m1.large:

19.png

If we log back in to the console, we can verify the new specs:

20.png

Reply
0 Kudos
mvoelker
VMware Employee
VMware Employee
Jump to solution

Ah.  If you're using images that were created from existing templates in vCenter: do you have cloud-init running on them?  Cloud-init is the "agent" which reads metadata that OpenStack provides to instances (such as SSH keys and passwords).  If the templates don't start cloud-init on bootup and try to read metadata either from the metadata service or a config drive, they won't get any of the metadata that OpenStack has provided for them there.  Try installing cloud-init before you import them to VIO.

Regarding resize: were you using the feature discussed here?

VMware Integrated OpenStack Documentation

If so, note that VM's imported from vSphere can't be resized (it's a limitation of the import process).

Reply
0 Kudos
benjamin000
Enthusiast
Enthusiast
Jump to solution

WOW....Thank you for your detailed answer.

Let me import the image using your guide.

I want to move away from vSphere linked clones as they limit many features in OpenStack as you described.

Give me a few hours and Ill report back with screenshots as I go through the process so you can see what to output os after each task.

Many thanks again....see you in a few hours.

Regards Ben McGuire
Reply
0 Kudos
benjamin000
Enthusiast
Enthusiast
Jump to solution

OK here is the import config....Notice the path that the image is saved........not sure if this is correct.....the datastore-h1 is for our glance images and is part of the Management Cluster and NOT the Management Cluster....Not sure if the whole issue is that the Compute Cluster cannot access the Glance storage on the other cluster. Ill post below and contine the process of booting the image.

+-------------------+----------------------------------------------------------------------------------+

| Property          | Value                                                                            |

+-------------------+----------------------------------------------------------------------------------+

| checksum          | 25da3acab885d89b7045f8844f586c2c                                                 |

| container_format  | bare                                                                             |

| created_at        | 2017-06-13T09:48:35Z                                                             |

| direct_url        | vsphere://vcenter.XXXXXXXXXXXX.local/folder/images/c7781224-0549-4dd6-9344-682e2 |

|                   | 1c48610?dcPath=Openstack+Datacenter&dsName=datastore-h1                          |

| disk_format       | vmdk                                                                             |

| id                | c7781224-0549-4dd6-9344-682e21c48610                                             |

| locations         | [{"url": "vsphere://vcenter.XXXXXXXXXXX.local/folder/images/c7781224-0549-4dd6- |

|                   | 9344-682e21c48610?dcPath=Openstack+Datacenter&dsName=datastore-h1", "metadata":  |

|                   | {}}]                                                                             |

| min_disk          | 0                                                                                |

| min_ram           | 0                                                                                |

| name              | Ubuntu-14.04-Trusty                                                              |

| owner             | f25adf22c293444bb8e2778df6a12ad9                                                 |

| protected         | False                                                                            |

| size              | 262472192                                                                        |

| status            | active                                                                           |

| tags              | []                                                                               |

| updated_at        | 2017-06-13T09:48:54Z                                                             |

| virtual_size      | None                                                                             |

| visibility        | private                                                                          |

| vmare-adaptertype | lsiLogic                                                                         |

| vmware-disktype   | streamOptimized                                                                  |

| vmware_ostype     | ubuntu64Guest                                                                    |

+-------------------+----------------------------------------------------------------------------------+

Regards Ben McGuire
Reply
0 Kudos
benjamin000
Enthusiast
Enthusiast
Jump to solution

Well I followed the steps and still get the No Operating System Found.

Here is the glance import command I used:-

pastedImage_0.png

Booted image and Horizon states instance is running:

pastedImage_1.png

Upon looking at the console it states the same error....

Maybe it is the glance store that I am using. As I have a lot of unused space in the Management Cluster I assigned the Glance storage there. This is local storage and NOT shared storage. Can it be possible that the Compute Cluster cannot access the image?  Maybe this is not it as I am sure there would be an error in horizon. Would it help if I change the glance storage to the compute cluster ?

pastedImage_2.png

As I am moving to a dedicated OpenStack control panel it is imperitive that I get this working. I have been using vSphere linked clones for 12 months with no issues but now I want to use OpenStack the way it was designed and not rely on linked clones.

Regards Ben McGuire
Reply
0 Kudos
benjamin000
Enthusiast
Enthusiast
Jump to solution

One thing I have noticed between the image below and your is that yours states that the image size is 2GB and mine is still the 250MB that the original images size is.

pastedImage_0.png

Regards Ben McGuire
Reply
0 Kudos
benjamin000
Enthusiast
Enthusiast
Jump to solution

Hello Again..

I am going to post screen shots of the VIO deployment so as to provide more information about hosts and datastores in case it helps

pastedImage_0.png

pastedImage_1.png

pastedImage_2.png

pastedImage_3.png

FYI the datastore datastore-h1   is on the Management Cluster on esxi-5

Regards Ben McGuire
Reply
0 Kudos
benjamin000
Enthusiast
Enthusiast
Jump to solution

This virtual size maybe the problem as I noticed in my image upload the virtual_size is zero but in the example provided above the image has an actual size and is not zero like mine below

pastedImage_0.png

Regards Ben McGuire
Reply
0 Kudos
benjamin000
Enthusiast
Enthusiast
Jump to solution

I also noticed the difference between the glance upload that the above tutorial provided is a file link for the location

My image location is a vcenter URL and not a file like you provided. Just trying to provide as much information to somehow try and get this solved.

pastedImage_1.png

Regards Ben McGuire
Reply
0 Kudos