Migrate Datastore create duplicate and left behind .vmdk files

[Stan_Beta]

Dear All,

Need your help and advice...

I tried to migrate server datastore via vSphere client. The migrate worked, but after some time, I realize when browsing the datastore, there are couple of .vmdk files that left behind on the original datastore and also in the new datastore/folder, there's also multiple .vmdk more than it suppose to be

*seems to be duplicated and the names systematically changed

If I check on the migrated server Edit Setting, the left behind files path are not being used, also the duplicated file with the original names (I guess it's the original files) are not being used anymore.

After looking for some advises on Google, some says that the files used by online servers cannot be copied or cannot be added as a virtual storage of a new server and be powered on...

I tried those step and confirmed that the unused files can be copied and added as a new server storage.

But is that means that I can delete those file from the datastore (delete from disk option)?

Is there anything I must consider so it will not interfere with the migrated servers?

For example:

SVR-QA, only have 3 HD and after migrating, the files are:

- [Free-NAS01] SVR-QA_1/SVR-QA-000001.vmdk

- [Free-NAS01] SVR-QA_1/SVR-QA_1-000001.vmdk

- [Free-NAS01] SVR-QA_1/SVR-QA_2-000001.vmdk

But when I browse the datastore, I got:

*multiple .vmdk files in new datastore/folders

And also:

*left behind file in the old path

Somethings that make me confused are:
1. I can copied files that not mentioned on the server Edit Setting, but when I tried to copy the used ones, they give me error:

when I tried with other server files, they said:

2. When I tried to summarized the storage size from the used files, the total are 108.6 GB

But when I compared with the server storage information from the OS (Linux SUSE), they give the total about 200 GB more or less...

Really need help with this issue, because they're filling up our datastore and making warning and error of datastore usage...

Kindly help,

Regards,

Stan

[Alistar]

Hi Stan,

to answer your questions systematically:

The SVR_QA_1 folder was created because this name has already existed - maybe the last file was still locked but it failed to delete itself - this is the currently live VM and the "old" SVR_QA folder seems to contain an orphaned VMDK that can be deleted. You can align these names back via OVF export -> import.

The VMDKs with

- [Free-NAS01] SVR-QA_1/SVR-QA-000001.vmdk

- [Free-NAS01] SVR-QA_1/SVR-QA_1-000001.vmdk

- [Free-NAS01] SVR-QA_1/SVR-QA_2-000001.vmdk

Are snapshot copies of your Virtual Machine. I see they are more than 100GB in size, so I strongly recommend that you right-click on the running virtual machine out of business hours (or whenever the environment is about to be used the least, usually in the evening) and choose Snapshot -> Consolidate as pointed out in this KB http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=200363...

The space difference between what Datastore Browser tells you and what you actually see on your FreeNAS Server might be because of the difference in provisioning. Seeing that there are differences between Size and Provisioned size columns, you are using Thin Provisioning, correct?

You can not copy locked files because they are in use by the hypervisor. And rightfully so, losing touch with the VMs files would result in an OS crash. You can see which VMDK files are locked by logging in to your ESXi host via SSH as a root and doing lsof | grep SVR-QA_1 to list the files that are being actively used for your SVR-QA_1 VM. Try doing the same for SVR-WIN-IDS and see which ESXi host has them locked. Once you find out that no host has the leftover files locked and that all VMs have their right disks, the leftover files are safe to delete.

I hope you got a little insight into what has happened - good luck on cleaning up

[Stan_Beta]

Dear Alistar,

Thank you so much for you reply...

It make this case more understandable...

But does it means that all the 3 files used by SVR-QA are only used for the snapshot and the other files probably still being used by the server?

Can you help me with the steps to start cleaning-up the datastore?

Can I immediately delete the "old" SVR-QA folder and .vmdk file or I have to consolidate the snapshot first and check the locked files by the ESXi?

When you mean ESXi, does it mean the virtual server itself (SVR-QA, SVR-WIN-IDS, etc), the vCenter server or the host server where the virtual server runs?

*we use 6host blade server (svrbl01,02,...,06), and they can be accessed from vSphere client by IP, but I don't know whether they used Linux or not for the hostservers...

Thank you,

Regards,

Stan

[Alistar]

Hi again Stan,

the first step you should do is to Consolidate the Virtual Machine disk files - and the best would be to do it on all your VMs (if any snapshots are present) that were on the previously migrated datastore, just to be sure you reclaim all the space. Snapshots are delta disks for your point-in-time states of Virtual Machines that you can return to them for example should an application upgrade fail or other short-term VM-disk based change. You can safely do this step on all your current VMs if you don't need the snapshots anymore (consult with the app owner if you aren't their owner).

The second step to do would be moving the "orphaned" VMDKs from the "Original" datastore to a single folder, for example "temp_delete". If the move is successful you will know that the file is not locked and therefore not used. After all leftover files were moved without problem, you can delete the folder safely.

By the SSH session I mean the direct connection to the ESXi host itself (srvbl01,... etc.) - ESXi has a busybox console that can be operated in unix-like fashion, and there you can see what files are locked with the lsof | grep virtualmachinename.vmdk, but I guess you won't be needing them 🙂

Good luck!

[Stan_Beta]

Dear Alistar,

Thanks again for your reply.

I tried to check for the consolidate need by viewing the VMs information. And what I get is there are no needs for consolidation in all of the servers, whether the server has snapshot or not.

What does it means? If I try to consolidate them, will it help me retrieve back our storage?

Right now, I'm trying to move the "orphaned" files to my test folder. Hopefully, they all can be moved.

I also tried to access the host/the blade servers, but unfortunately, they all cannot be access through SSH (I'm using Putty)

Regards,

Stan