vSAN Adding Glance Images ( http error Gone 410 )

[benjamin000]

We have just moved to vSAN storage which so far has gone perfectly however, when trying to import a Ubuntu image via the OpenStack command line we receive and error.

As you can see the error is about Adding images being dsiabled but we have been using VIO without vSAN storage with no issues when adding images. All Compute Hosts have access to the vSAN storage and we have added the vSAN Datastore within the OpenStack inteface

We have looked at the glance-api.conf file specifically the part regarding the VMware and Glance store.

As you can see we have adding the weight to the vsanDatastore of 200 so that the image upload selects the vsan datastore first.

We are using VIO 3.0 and NSX Networking. And FYI we DO NOT have vSAN enabled in the Management Cluster as we want the NSX and VIO VMs to use local storage.

[benjamin000]

Here is the error in vRealize Log Insight.

[benjamin000]

We managed to get the image imported by changing the glance-api.conf

The default setting is

default-store= vi

We changed it to

default-store= file

Now the image imports but the image is stored in /var/lib/glance/images and not on the vSAN datastore.

I think we may have to modify the glance-api.conf file so that it stores in a directory on the vSAN datastore. There is a setting in the configuration file.

We shall give it a go and report back soon.

[sabarim]

This error indicates that there is a configuration issue with glance-api. The exact trace or reason can be found at the very top of glance-api logs when it loads the 'vi' store. Can you look for that ?

I don't recall exactly but the trace must contain the words "BadStoreConfiguration" or "Failed to load driver". Sometimes this happens if the datastore got relocated under different datacenter or folder and glance isn't aware of the change.

Switching to 'file' store will store all your images locally on the controllers and doesn't use the VMware configuration. default_store='vi' is the supported store for VIO. If we fix the BadStoreConfiguration then you should see the image gets uploaded to the vSAN datastore.

[benjamin000]

Hello

Thank you for the advice. I am going to further investigate this however I have a few additional questions.

As we now are using vSAN does the stores still need to be = vi or something else ?

We did change the glance storage from datastoreh1 however now we have two  when we added the vsandatastore.

Do we need to create an images folder on the vsandatastore and make that configuration change in the glance-api.conf so that the directory the images are being stored is /images on the vsandatastore ?

I thought it may have to do with the VIO management cluster not having access to the vSAN datastore as this storage is on the Compute Cluster and the Management Cluster does NOT have access to the vSAN storage. Does this matter ?

Ill run some tests later and advise on the glance-api logs.

[benjamin000]

I am adding a shared NAS store to all hosts in the Compute Cluster to minimize vSAN issues first while I try and get rid of the error.

I also have another post about glance images not even working as when they are launched the boot screen displays "No Operating System Found" which means the image is not getting loaded or the VIO VM cannot reach or see the glance image. Hopefully vSAN storage and NAS shared storage will fix this as this was the main reason why I decided to move to vSAN.

Ill post the glance-api logs shortly as I am just waiting for the VIO services to restarts as that takes about 20 minutes.

[benjamin000]

Ok I got it working by changing to a shared NAS Datastore and then creating an /images folder on the NAS datastore. Now the glance images are being stored on the NAS device which is fine however when I try and boot an image every image says "Operating System Not Found "

[benjamin000]

In addition I was also using the glance image-create  command. I used openstack image create and that worked..Not sure if that command was the issue.

[benjamin000 · Accepted solution]

This issue was resolved by creating a folder  /images in the glance datastore and adding the path to this datastore in the glance-api.conf file.

Once the folder and path was added the upload went without a hitch.