VMware Horizon Community
lookandfeel
VMware Employee
VMware Employee

Remove a datastore from a storage group in App Volumes 2.14.2

Hi,

I have a storage group which consists of 3 datastores (App Volumes 2.14.2).

I want to remove only one of these datastores.

How can I achieve this without downtime in the environment?

I would be happy about a tip.

Best Regards,

Andre

11 Replies
Ray_handels
Virtuoso
Virtuoso

Appvolumes checks VCener to see what datastores are available. If you do want to remove the datastore you should first remove it from VCenter, otherwise my guess is that it will be recreated after a sync with vcenter is done.

After you removed the datastore you need to run a sync on the datastores and then a sync on the Appstacks. You will see that it still has 2 correct datastores and the datastore you removed will still be there but will be striked through and marked as unavailable datastore. To remove it you will need to remove the reference to that datastore for the appstack in the database, not other way of doing it.

Reply
0 Kudos
lookandfeel
VMware Employee
VMware Employee

Thanks your your answer.

That sounds unbeautiful.

I edited the storage group, changed the "storage selection" to 'Direct'. After that I set the checkmarks for the 2 remaining datastores only, not for the removing datastore.

In the GUI the storage group now consists of 2 datastores only.

But thats not the successful end.

I tried to remove some appstacks (vmdk files) which are still on the removed datastore per vSphere (Web) Client and get an error message:

"Deletion of file or directory [datastore name] blabla.vmdk was initiated from web client and completed with status 'failure'.

No other text or hint why I cannot delete these files from the datastore.

I renamed the folder /cloudvolumes214/apps/ to another one (/cloudvolumes214old/apps/), but the result is the same.

After some minutes I could see that a new folder was created in the (from storage group) removed datastore with the name /cloudvolumes214/apps.

So it seems that the App Volume manager is still using this datastore, but I have no idea why.

For sure, I could remove the datastore from vCenter also. But I have some important files on this datastore also and want to clarify why I cannot delete the unneeded files from that. Obviously there is an existing relation from App Volume Manager to this datastore.

Regards,

Andre

Reply
0 Kudos
Ray_handels
Virtuoso
Virtuoso

The reason that you can't remove the appstack is because it is marked as read only. You can change the .vmdk file (not the flat file) and change the value ddb.deletable=false to ddb.deletable=true, then you should be able to remove it.

And my guess is that because you still have appstack located on that datastore (even if they are no longer in the storage group) Appvolumes will still use it for storing it's appstack.

lookandfeel
VMware Employee
VMware Employee

Hi Ray,

you are right. Looking in the vmdk (descriptor) file I can see that the flat file is not deletable.

Trying to change this entry for one test file, after that I really can delete this.

Editing all affected vmdk files would be a high effort, so I will think about another process for that (maybe deleting the datastore and recreate this or similar if I backup the other relevant data from datastore).

The second behaviour I can reproduce - renaming the cloudvolumes folder and wait some minutes until this folder is re-created automatically.

In this folder /cloudvolumes/apps/ only one xxx.vmdk.metadata file is re-created. It is always the same file name and seems to be related to one specific appstack.

So I have to investigate here further.

One other thing I noticed is that if I take a look to the existing appstacks in the app volume manager GUI, for every appstack I can see 3 locations - and there is the (from storage group) removed datastore/location included, although this is no longer in use.

I am not sure if this could be a problem.

Regards,

Andre

Reply
0 Kudos
Ray_handels
Virtuoso
Virtuoso

One other thing I noticed is that if I take a look to the existing appstacks in the app volume manager GUI, for every appstack I can see 3 locations - and there is the (from storage group) removed datastore/location included, although this is no longer in use.

I am not sure if this could be a problem.

Yes this is a problem as it will most likely re-sync the appstacks to that datastore.

You either should remove those records out of the database or, if the environment is not that large, just recreate the database.Then make sure to remove the appstacks from the datastore you do not want to use anymore, create a storage group with the 2 appstacks you do wanna use and import the appstacks from those locations.

Reply
0 Kudos
lookandfeel
VMware Employee
VMware Employee

Yes this is a problem as it will most likely re-sync the appstacks to that datastore.

You either should remove those records out of the database or, if the environment is not that large, just recreate the database.Then make sure to remove the appstacks from the datastore you do not want to use anymore, create a storage group with the 2 appstacks you do wanna use and import the appstacks from those locations.

Ok, removing the records out of the database sounds not easy. But we have 140 AppStacks appr. 😞

For clarifying me, maybe one other opportunity would be to create a new storage group (maybe with 2 completely new datastores), copy all AppStacks maually to one of these new datastore, import the AppStacks and reconfigure the assignments.

And changing the default location in the storage configuration should be done also.

It is also a higher effort, but maybe one very clear solution.

Reply
0 Kudos
Ray_handels
Virtuoso
Virtuoso

Nope still wont work as you would still have a reference in the database and the appstack is still there. The storage group is only there to sync appstacks automatically.. If an appstack resides on a datastore it will be there, you cannot remove it.

You could however update all 140 appstacks and make sure to not sync it to the datastore you would like to remove but my guess is that it would be easier to recreate the database.

Reply
0 Kudos
lookandfeel
VMware Employee
VMware Employee

Hhm, thinking about these steps I would agree with you.

Maybe re-creating the database could be the most clearly way.

What do you think, how can I "re-create" a new database?

Simply create a new db on SQL server and re-configure the ODBC-connection, pointing to this new db, at the App Volume Manager server?

Reply
0 Kudos
da2125
Enthusiast
Enthusiast

mark the datastore as non-attachable, wait x days and/or run DB report to ensure no one has any AppStacks mounted from this datastore. Once confirmed, remove the datastore from the Storage Group, after x time with no issue, clean up the cloudvolumes folder structure on the datastore

Reply
0 Kudos
Ray_handels
Virtuoso
Virtuoso

What do you think, how can I "re-create" a new database?

Simply create a new db on SQL server and re-configure the ODBC-connection, pointing to this new db, at the App Volume Manager server?

Just install a new Appvolumes manager, during installation recreate the database, select the same datastore (make sure no machines are actually in that database) and import the appstacks. That should be enough. You can then reinstall the Appvolumes manager and point it to the new database.

mark the datastore as non-attachable, wait x days and/or run DB report to ensure no one has any AppStacks mounted from this datastore. Once confirmed, remove the datastore from the Storage Group, after x time with no issue, clean up the cloudvolumes folder structure on the datastore

No matter what you do the database will still hold a record to that file. If removed form the storage group it wont sync it automatically but than when you look at storage locations of the appstack it will still see that appstack but will mark it as not there and it will be striped through

lookandfeel
VMware Employee
VMware Employee

Hi Ray,

thanks for your answer.

On the one side it seems not simple, but on the other side replacing an App Volume Manager should be not so difficult when the new one will be connected to an existing database.

And I can re-import (rescan) also existing appstacks, so I have a valid procedure to get a cleared environment.

I will check your recommendations here and if we will run this work.

Maybe we can combine this with another task in the next months (have to move all appstack to a completely new storage) and so we can minimize the tasks.

Thank you very very much for your feedback!!

Best Regards,

Andre

Reply
0 Kudos