11 Replies Latest reply on Jun 6, 2019 7:42 AM by lookandfeel

    Remove a datastore from a storage group in App Volumes 2.14.2

    lookandfeel Enthusiast
    VMware Employees

      Hi,

      I have a storage group which consists of 3 datastores (App Volumes 2.14.2).

      I want to remove only one of these datastores.

       

      How can I achieve this without downtime in the environment?

      I would be happy about a tip.

       

      Best Regards,

       

      Andre

        • 1. Re: Remove a datastore from a storage group in App Volumes 2.14.2
          Ray_handels Master
          Community WarriorsvExpert

          Appvolumes checks VCener to see what datastores are available. If you do want to remove the datastore you should first remove it from VCenter, otherwise my guess is that it will be recreated after a sync with vcenter is done.

           

          After you removed the datastore you need to run a sync on the datastores and then a sync on the Appstacks. You will see that it still has 2 correct datastores and the datastore you removed will still be there but will be striked through and marked as unavailable datastore. To remove it you will need to remove the reference to that datastore for the appstack in the database, not other way of doing it.

          • 2. Re: Remove a datastore from a storage group in App Volumes 2.14.2
            lookandfeel Enthusiast
            VMware Employees

            Thanks your your answer.

            That sounds unbeautiful.

             

            I edited the storage group, changed the "storage selection" to 'Direct'. After that I set the checkmarks for the 2 remaining datastores only, not for the removing datastore.

            In the GUI the storage group now consists of 2 datastores only.

             

            But thats not the successful end.

            I tried to remove some appstacks (vmdk files) which are still on the removed datastore per vSphere (Web) Client and get an error message:

            "Deletion of file or directory [datastore name] blabla.vmdk was initiated from web client and completed with status 'failure'.

             

            No other text or hint why I cannot delete these files from the datastore.

            I renamed the folder /cloudvolumes214/apps/ to another one (/cloudvolumes214old/apps/), but the result is the same.

             

            After some minutes I could see that a new folder was created in the (from storage group) removed datastore with the name /cloudvolumes214/apps.

            So it seems that the App Volume manager is still using this datastore, but I have no idea why.

             

            For sure, I could remove the datastore from vCenter also. But I have some important files on this datastore also and want to clarify why I cannot delete the unneeded files from that. Obviously there is an existing relation from App Volume Manager to this datastore.

             

            Regards,

             

            Andre

            • 3. Re: Remove a datastore from a storage group in App Volumes 2.14.2
              Ray_handels Master
              Community WarriorsvExpert

              The reason that you can't remove the appstack is because it is marked as read only. You can change the .vmdk file (not the flat file) and change the value ddb.deletable=false to ddb.deletable=true, then you should be able to remove it.

               

              And my guess is that because you still have appstack located on that datastore (even if they are no longer in the storage group) Appvolumes will still use it for storing it's appstack.

              1 person found this helpful
              • 4. Re: Remove a datastore from a storage group in App Volumes 2.14.2
                lookandfeel Enthusiast
                VMware Employees

                Hi Ray,

                 

                you are right. Looking in the vmdk (descriptor) file I can see that the flat file is not deletable.

                Trying to change this entry for one test file, after that I really can delete this.

                Editing all affected vmdk files would be a high effort, so I will think about another process for that (maybe deleting the datastore and recreate this or similar if I backup the other relevant data from datastore).

                 

                The second behaviour I can reproduce - renaming the cloudvolumes folder and wait some minutes until this folder is re-created automatically.

                In this folder /cloudvolumes/apps/ only one xxx.vmdk.metadata file is re-created. It is always the same file name and seems to be related to one specific appstack.

                So I have to investigate here further.

                 

                One other thing I noticed is that if I take a look to the existing appstacks in the app volume manager GUI, for every appstack I can see 3 locations - and there is the (from storage group) removed datastore/location included, although this is no longer in use.

                I am not sure if this could be a problem.

                 

                Regards,

                Andre

                • 5. Re: Remove a datastore from a storage group in App Volumes 2.14.2
                  Ray_handels Master
                  Community WarriorsvExpert

                  One other thing I noticed is that if I take a look to the existing appstacks in the app volume manager GUI, for every appstack I can see 3 locations - and there is the (from storage group) removed datastore/location included, although this is no longer in use.

                  I am not sure if this could be a problem.

                  Yes this is a problem as it will most likely re-sync the appstacks to that datastore.

                  You either should remove those records out of the database or, if the environment is not that large, just recreate the database.Then make sure to remove the appstacks from the datastore you do not want to use anymore, create a storage group with the 2 appstacks you do wanna use and import the appstacks from those locations.

                  • 6. Re: Remove a datastore from a storage group in App Volumes 2.14.2
                    lookandfeel Enthusiast
                    VMware Employees

                    Yes this is a problem as it will most likely re-sync the appstacks to that datastore.

                    You either should remove those records out of the database or, if the environment is not that large, just recreate the database.Then make sure to remove the appstacks from the datastore you do not want to use anymore, create a storage group with the 2 appstacks you do wanna use and import the appstacks from those locations.

                     

                    Ok, removing the records out of the database sounds not easy. But we have 140 AppStacks appr. :-(

                     

                    For clarifying me, maybe one other opportunity would be to create a new storage group (maybe with 2 completely new datastores), copy all AppStacks maually to one of these new datastore, import the AppStacks and reconfigure the assignments.

                    And changing the default location in the storage configuration should be done also.

                    It is also a higher effort, but maybe one very clear solution.

                    • 7. Re: Remove a datastore from a storage group in App Volumes 2.14.2
                      Ray_handels Master
                      vExpertCommunity Warriors

                      Nope still wont work as you would still have a reference in the database and the appstack is still there. The storage group is only there to sync appstacks automatically.. If an appstack resides on a datastore it will be there, you cannot remove it.

                       

                      You could however update all 140 appstacks and make sure to not sync it to the datastore you would like to remove but my guess is that it would be easier to recreate the database.

                      • 8. Re: Remove a datastore from a storage group in App Volumes 2.14.2
                        lookandfeel Enthusiast
                        VMware Employees

                        Hhm, thinking about these steps I would agree with you.

                        Maybe re-creating the database could be the most clearly way.

                         

                        What do you think, how can I "re-create" a new database?

                        Simply create a new db on SQL server and re-configure the ODBC-connection, pointing to this new db, at the App Volume Manager server?

                        • 9. Re: Remove a datastore from a storage group in App Volumes 2.14.2
                          da2125 Novice

                          mark the datastore as non-attachable, wait x days and/or run DB report to ensure no one has any AppStacks mounted from this datastore. Once confirmed, remove the datastore from the Storage Group, after x time with no issue, clean up the cloudvolumes folder structure on the datastore

                          • 10. Re: Remove a datastore from a storage group in App Volumes 2.14.2
                            Ray_handels Master
                            Community WarriorsvExpert

                            What do you think, how can I "re-create" a new database?

                            Simply create a new db on SQL server and re-configure the ODBC-connection, pointing to this new db, at the App Volume Manager server?

                            Just install a new Appvolumes manager, during installation recreate the database, select the same datastore (make sure no machines are actually in that database) and import the appstacks. That should be enough. You can then reinstall the Appvolumes manager and point it to the new database.

                             

                            mark the datastore as non-attachable, wait x days and/or run DB report to ensure no one has any AppStacks mounted from this datastore. Once confirmed, remove the datastore from the Storage Group, after x time with no issue, clean up the cloudvolumes folder structure on the datastore

                            No matter what you do the database will still hold a record to that file. If removed form the storage group it wont sync it automatically but than when you look at storage locations of the appstack it will still see that appstack but will mark it as not there and it will be striped through

                            1 person found this helpful
                            • 11. Re: Remove a datastore from a storage group in App Volumes 2.14.2
                              lookandfeel Enthusiast
                              VMware Employees

                              Hi Ray,

                               

                              thanks for your answer.

                              On the one side it seems not simple, but on the other side replacing an App Volume Manager should be not so difficult when the new one will be connected to an existing database.

                              And I can re-import (rescan) also existing appstacks, so I have a valid procedure to get a cleared environment.

                               

                              I will check your recommendations here and if we will run this work.

                              Maybe we can combine this with another task in the next months (have to move all appstack to a completely new storage) and so we can minimize the tasks.

                               

                              Thank you very very much for your feedback!!

                               

                              Best Regards,

                              Andre