VMware Cloud Community
BHagenSPI
Enthusiast
Enthusiast

Cannot download vmdk from vsan datastore

I've got a vsphere 6.0U3 cluster, and vsan 6.2 Enterprise.

While working on an issue with me, vmware support made a copy of a 13TB vmdk to a different folder in my vsan as a backup. I am now trying to move that copy off my vsan because I'm now almost out of room and I need the space back.

I browse the datastore from the web-client, right-click the copied vmdk file (that's not attached to any vm), and click "download from datastore".

I immediately get 2 lines of feedback. Progress 100%, and "No file exists for given path -- No file exists for given path -- File open failed: Could not find the file".

I can download every other file from that folder (there are no other vmdk's).

Help? I'm stumped.

Thanks!

Reply
0 Kudos
4 Replies
BHagenSPI
Enthusiast
Enthusiast

Update:

If I ssh into one of my hosts and do an "ls -l" on the vsan folder that contains the 13TB vmdk, the size shows as 549k. ??? In the web client, it shows as 13TB.

If I use winscp, that same file shows as 1k.

If I attach it to a VM, it shows as 6.07TB (ftt=1)

What's going on here???

Reply
0 Kudos
TheBobkin
Champion
Champion

Hello BHagenSPI​,

"vmware support made a copy of a 13TB vmdk to a different folder in my vsan as a backup."

Using what method precisely? Was this copied from vSAN or somewhere else?

"I can download every other file from that folder (there are no other vmdk's)."

Are you able to download other vmdks from other VMs and do these result in usable disks? (determining whether downloading Objects to flats is working and/or potentially 2TB+ limiting)

"the size shows as 549k. ??"

Yes, because vSAN Disk Objects are not stored as -flat.vmdks but distributed Objects on vsandatastore and thus all you are measuring the size of here is the text files that point to these Objects.

"If I use winscp, that same file shows as 1k."

Same as above - you are just copying a descriptor, not data.

"If I attach it to a VM, it shows as 6.07TB (ftt=1)"

Is it Thin-provisioned? Are you looking at the space used on vsandatastore or the provisioned size of the vmdk?

Anyway, aside from the above (that don't fix the issue, just clarify why you are seeing this), what are you trying to achieve here?

Do you want the vmdk gone to free up the space? Do you want it transferred to alternative storage? Do you have a datastore attached to at least one of the hosts with enough space to store this disk? If no attached storage, do you have some alternative method of gettign this onto your desktop e.g. Network-based back-up?

Bob

Reply
0 Kudos
BHagenSPI
Enthusiast
Enthusiast

Hi Bob; thanks for the reply!

"vmware support made a copy of a 13TB vmdk to a different folder in my vsan as a backup."

Using what method precisely? Was this copied from vSAN or somewhere else?

It was copied from vsan to vsan from within the web client. We browsed the vSAN datastore to the folder containing the vmdk, right-clicked the .vmdk file in question, and did a "copy to" a different folder on the same vsan (the only vsan datastore we have).

"I can download every other file from that folder (there are no other vmdk's)."

Are you able to download other vmdks from other VMs and do these result in usable disks? (determining whether downloading Objects to flats is working and/or potentially 2TB+ limiting)

Good thought; nope...I cannot download any vmdks...same error as above. Obviously, this is the crux of the matter, and I'm having trouble finding solutions.

"the size shows as 549k. ??"

Yes, because vSAN Disk Objects are not stored as -flat.vmdks but distributed Objects on vsandatastore and thus all you are measuring the size of here is the text files that point to these Objects.

&

"If I use winscp, that same file shows as 1k."

Same as above - you are just copying a descriptor, not data.

Interesting; then the only way to copy an entire vmdk is thru the gui?

"If I attach it to a VM, it shows as 6.07TB (ftt=1)"

Is it Thin-provisioned? Are you looking at the space used on vsandatastore or the provisioned size of the vmdk?

I messed up that sentence. 🙂 From within Windows Server 2012, the drive shows as 6.07TB. When I edit the settings of the 2012 vm that it's attached too, I've got it set to FTT=1, the size of the hdd is reported as 6,215.0625GB, and the Virtual SAN storage consumption is 12.14TB.

Anyway, aside from the above (that don't fix the issue, just clarify why you are seeing this), what are you trying to achieve here?

Do you want the vmdk gone to free up the space? Do you want it transferred to alternative storage? Do you have a datastore attached to at least one of the hosts with enough space to store this disk? If no attached storage, do you have some alternative method of gettign this onto your desktop e.g. Network-based back-up?

The goal is to move this vmdk off of vSAN to free up space. I only have that much available space on the NAS box I use for backups. The reason for having a working copy of the vmdk is that we're about to try a procedure on the original vmdk and it's two snapshots, and if that for some reason fails catastrophically, we know that at least this vmdk is good...meaning, we'll "only" lose data from March to the current time; we won't lose the entire set of data. Many failure points have lead to this situation, all of which are being fixed. But "the perfect storm" happened, and this is the only "current" copy of this data that we have, so I'm being overly cautious. (All our backups also got smacked, so none of them go past January. Again, worst case all the way around!)

Reply
0 Kudos
TheBobkin
Champion
Champion

Hello BHagenSPI​,

"It was copied from vsan to vsan from within the web client. We browsed the vSAN datastore to the folder containing the vmdk, right-clicked the .vmdk file in question, and did a "copy to" a different folder on the same vsan (the only vsan datastore we have)."

So the original (base vmdk + snapshots) and this copy (base) vmdk are both stored on the same vsandatastore? Is the orginal still actively in use?

Can you please verify that there are indeed two sets of the Data-Objects by checking what Object the two copies are pointing to:

# cat /vmfs/volumes/vsanDatastoreName/Namespace1UUID/VM-Name.vmdk | grep vsan

# cat /vmfs/volumes/vsanDatastoreName/Namespace2UUID/VM-Name-Copy.vmdk | grep vsan

Basically ruling out that just the descriptor was copied here - if they point to different Objects with different UUIDs then this is not the case.

"Good thought; nope...I cannot download any vmdks...same error as above. Obviously, this is the crux of the matter, and I'm having trouble finding solutions."

What build version of vCenter and ESXi installed here?

Potentially not implicated here - but easy to check and thus worthwhile - try a different browser: Firefox, Chrome, IE11.

Even if this was able to download over Network to your desktop I wouldn't rely on this method for a large 6TB disk.

"Interesting; then the only way to copy an entire vmdk is thru the gui?"

No, there are other options. Not to question whomever did this (they likely had a reason), but personally I wouldn't use datastore browser to perform this - I would use VM-level clone operation or more preferable if just cloning a single vmdk (without its snapshots) # vmkfstools -i (-W vsan):

https://docs.vmware.com/en/VMware-vSphere/6.5/com.vmware.vsphere.storage.doc/GUID-01D3CF47-A84A-4988...

There are other potential GSS tools for cloning Objects also.

"The goal is to move this vmdk off of vSAN to free up space. I only have that much available space on the NAS box I use for backups."

Is this NAS box attached to at least one of the vSAN hosts and if not can you attach it?

If it is/can be then you can simply attach the copy vmdk to a VM and Storage vMotion it over or use vmkfstools -i to clone it over.

"...meaning, we'll "only" lose data from March to the current time; we won't lose the entire set of data."

Yikes

(All our backups also got smacked, so none of them go past January. Again, worst case all the way around!)

Double Yikes - I hope this  large vmdk isn't the backups of the thing it is backing up being stored on the thing it is backing up :smileygrin:

Bob

Reply
0 Kudos