VMware Cloud Community
bcurran3
Contributor
Contributor
Jump to solution

Can't see copied VMDKs on new/different DATASTORE - HELP!

I searched and I searched and I searched. I found lots of info, but not the answer I need. I'm sure if I spent the next two days searching, I'd eventually find it. So I ask your pardon in advance if you've seen and answered this a million times before.

MOTIVE:

I had an ESXI server with a datastore comprised of 2 RAID 5s consisting of a total of 6x3TB drives giving me about 10TB of space. Well, it was full. So I decided to backup all the VMs and VMDKs (in multiple ways, wait, it's coming) and recreate one RAID5 disk using all the drives and recover 3TB of storage.Done. Seemed/s smart, I didn't have to buy new 4 or 5TB drives to replace them all and saved myself at least $1,000.Yeah me.

BACKUP IMPLEMENTATION:

I have five VMs running all the time, four servers, and one admin station. On the main file server I removed non-essential drives/VMDKs and started copying them down to a spare NAS for temporary storage. I used the VSphere Client, browsed the datastore, picked the VMDK files and saved them off. All cool with me based on past experience as I've been through this before to edit the VMDK text file to rename the files in the past. (Yeah, I know. Easier to SSH in and do it from the command line.) Dealing with so much data, backup and restore is literally taken weeks (only about 14MB/s transfer rates at BEST, and 7-8 MB/s norm.) My actual VMs and essential data I backed up with VEEAM.After everything was backed up, I recreated the new RAID and got my extra free storage. (Yeah team!)

RESTORE IMPLEMENTATION:

I  used VEEAM to restore all my VMs. It worked perfect. It's a great program. (The main file server with 3.5TB of essential data took 90+ hours to restore!) All VMs are up and running fine with my essential data. Now we start running into the problem. I used VSphere Client to browse the datastore, made a new folder called DRIVES, and have started to upload some of my previously removed VMDKs stored on my NAS. I went to add some of the uploaded drives back into my file server VM and, SURPRISE! ESXi doesn't see them. They show when browsing the datastore, but they don't have the option to add to inventory and aren't found when that database is rescanned. FAILURE! Trying to figure things out I looked at the VMDK text files and to find the difference. I hoped it was just a database signature/ID that needed to be updated to the current datastore and all would be edited and fixed fast. No dice. Comparing the information, I could see the differences but had no clue what they should be. So I created a new drive in an existing VM with the same size and parameters and looked at it, all different information in each file and I don't see a logical fix to the problem that way.

Yes, I know now that I should have used a SCP program to copy the files. In my experience, what I did would work but obviously my experience was limited in scope and it didn't work for this situation. I read the articles similar to my situation to use the Standalone Converter, but that doesn't work as it's looking for a VM (and thus a VMX file) to convert a machine instead of just a drive.

So I throw out my problem to the more experienced masses than I. How can I go about re-using my "old" downloaded VMDK drive files in my new datastore?

INFO:

ESXi 6.0U1

old drives h/w version 4

new drives h/w version 11

3.5TB down and 6.5 TB to go, about 8 drives.

Bill

1 Solution

Accepted Solutions
bcurran3
Contributor
Contributor
Jump to solution

Based on http://buildvirtual.net/recreating-a-missing-vmdk-descriptor-file/, I'm just deleting all the VMDK descriptor files and recreating them using vmkfstools. Size is right there in front of me, no problem. WHICH supported disk format I used, was a crap shoot. What was my mentality creating these HDDs five years ago? Thin provisioned? Zerothick? Eagerzerothick? Turns out I did them all zeroedthick.

So I've got four of the HDDs back up and working. A fifth one might be corrupt or linked to a snapshot that I didn't know exist and might possibly be effed. I'm re-uploading it now to know for sure. Three more need to be uploaded to the server and go through the process. Things are looking good! I'm a LOT less stressed.

The process to fix my problem ended up being this:

  • Upload old FILE-flat.vmdk files.
  • SSH into server, go to the path where the FILE-flat.vmdk files are.
  • rm all FILE-flat.vmdk files to oFILE-flat.vmdk.
  • ls -l to see file sizes of the oFILE-flat.vmdk files
  • use mkfstools to create new files (i.e. vmkfstools -c 137438953472 -d zeroedthick FILE.vmdk)
  • rm the new FILE-flat.vmdk
  • mv the oFILE-flat.vmdk to FILE-flat.vmdk
  • add HDD in VSphere Client to the appropriate VM.
  • Rinse and repeat.
  • Down a scotch

Thanks to all that helped.

Hopefully this will show up in searches for the next nitwit that goes through this pain.

-Bill

View solution in original post

Reply
0 Kudos
6 Replies
bcurran3
Contributor
Contributor
Jump to solution

BUMP!

No one knows how to deal with this problem?

Reply
0 Kudos
bcurran3
Contributor
Contributor
Jump to solution

mcrape
Enthusiast
Enthusiast
Jump to solution

Hi there - the KB site seems to be down for me at the moment, so I can't see what that article is.

The first thing that comes to mind is that either the file permissions are wrong, or the vmx file is corrupt. I believe the KB article for permissions is here: http://kb.vmware.com/kb/1904 (can't verify that at the moment though as the site is down).

Are you able to create a new VM and attach the existing disks to it? That would generate a new vmx file and might let you boot off the original VMDK.

Reply
0 Kudos
bcurran3
Contributor
Contributor
Jump to solution

mcrape - thanks for the attempt!

1> There are no VMX files, only  DATAFILE.VMDK and DATAFILE-FLAT.VMDK files. I think it's that VMX file that's killed me as VShphere doesn't download it using the methods I've stated. The KB article goes over recreating a VMX file. I found it late at night and put it off until later. I just viewed it again, it's up. Hopefully if you follow back here and click on it, you can see it and tell me if you think that's the solution or if I'm still chasing my tail. My gut is telling me right now that the KB article addresses the situation. No proof of concept until I can do it later. Smiley Happy

2> Permissions seem to be fine, all VMs and new HDDs with them work fine. The VMDKs can not be opened in Windows either using VMware Workstation 12, StarWind V2V Image Converter, etc. They all say the files are corrupt or unusable while VSphere itself just ignores them.

Reply
0 Kudos
bcurran3
Contributor
Contributor
Jump to solution

Based on http://buildvirtual.net/recreating-a-missing-vmdk-descriptor-file/, I'm just deleting all the VMDK descriptor files and recreating them using vmkfstools. Size is right there in front of me, no problem. WHICH supported disk format I used, was a crap shoot. What was my mentality creating these HDDs five years ago? Thin provisioned? Zerothick? Eagerzerothick? Turns out I did them all zeroedthick.

So I've got four of the HDDs back up and working. A fifth one might be corrupt or linked to a snapshot that I didn't know exist and might possibly be effed. I'm re-uploading it now to know for sure. Three more need to be uploaded to the server and go through the process. Things are looking good! I'm a LOT less stressed.

The process to fix my problem ended up being this:

  • Upload old FILE-flat.vmdk files.
  • SSH into server, go to the path where the FILE-flat.vmdk files are.
  • rm all FILE-flat.vmdk files to oFILE-flat.vmdk.
  • ls -l to see file sizes of the oFILE-flat.vmdk files
  • use mkfstools to create new files (i.e. vmkfstools -c 137438953472 -d zeroedthick FILE.vmdk)
  • rm the new FILE-flat.vmdk
  • mv the oFILE-flat.vmdk to FILE-flat.vmdk
  • add HDD in VSphere Client to the appropriate VM.
  • Rinse and repeat.
  • Down a scotch

Thanks to all that helped.

Hopefully this will show up in searches for the next nitwit that goes through this pain.

-Bill

Reply
0 Kudos
AnatolyVilchins
Jump to solution

Permissions seem to be fine, all VMs and new HDDs with them work fine. The VMDKs can not be opened in Windows either using VMware Workstation 12, StarWind V2V Image Converter, etc. They all say the files are corrupt or unusable while VSphere itself just ignores them.


>>> To be honest that is weird, I've seen StarWind V2V converter multiple times in the similar configuration and it worked like a charm. On my experience that should be some minor issue. Well, anyway, I know that you achieved your goal, which is good. Next time if you will face some issue with StarWind product, or you will have some concerns, feel free to post it to the StarWind community - it is really responsive and friendly imho StarWind Software • Index page

Kind Regards, Anatoly Vilchinsky