All Posts

bigmaverick · ‎04-11-2022

@kevinols @bourne83 could you find a solution how to use vmPath in VixDiskLib_Open?

dbackup · ‎04-08-2022

I used NBD mode to read VM disk, and it printed a pile of this stuff. What does that mean and how do I remove it ? Although it does not seem to affect reading VM disk. 2022-04-08 14:49:2...

I used NBD mode to read VM disk, and it printed a pile of this stuff. What does that mean and how do I remove it ? Although it does not seem to affect reading VM disk. 2022-04-08 14:49:26.656@ubuntu1604@6872@LM_WARNING@.7002|VDDK_PhoneHome: VddkVacPushConfigData : Failed to push vendor product data at line 2044. 2022-04-08 14:49:26.869@ubuntu1604@6872@LM_INFO@.7002|VDDK_PhoneHome: VddkVacWriteLastMember : Empty field to serialize for key fail_function at line 266. 2022-04-08 14:49:27.084@ubuntu1604@6872@LM_WARNING@.7002|VDDK_PhoneHome: Curl perform failed in VddkVacHttpsPost with error code 77 at 92. 2022-04-08 14:49:27.294@ubuntu1604@6872@LM_WARNING@.7002|VDDK_PhoneHome: Curl perform failed in VddkVacHttpsPost with error code 77 at 92. 2022-04-08 14:49:27.295@ubuntu1604@6872@LM_WARNING@.7002|VDDK_PhoneHome: Curl perform failed in VddkVacHttpsPost with error code 77 at 92. 2022-04-08 14:49:27.295@ubuntu1604@6872@LM_WARNING@.7002|VDDK_PhoneHome: VddkVacPushData : Failed to post data to VC/PH at line 1706. 2022-04-08 14:49:27.576@ubuntu1604@6872@LM_WARNING@.7002|VDDK_PhoneHome: VddkVacPushBackupData : Failed to push backup data at line 2893. 2022-04-08 14:49:27.787@ubuntu1604@6872@LM_WARNING@.7002|VDDK_PhoneHome: Curl perform failed in VddkVacHttpsPost with error code 77 at 92. 2022-04-08 14:49:28.001@ubuntu1604@6872@LM_WARNING@.7002|VDDK_PhoneHome: Curl perform failed in VddkVacHttpsPost with error code 77 at 92. 2022-04-08 14:49:28.001@ubuntu1604@6872@LM_WARNING@.7002|VDDK_PhoneHome: Curl perform failed in VddkVacHttpsPost with error code 77 at 92. 2022-04-08 14:49:28.001@ubuntu1604@6872@LM_WARNING@.7002|VDDK_PhoneHome: VddkVacPushData : Failed to post data to VC/PH at line 1706. 2022-04-08 14:49:28.006@ubuntu1604@6872@LM_WARNING@.7002|VDDK_PhoneHome: VddkVacPushEnvironmentData : Failed to push env data at line 1807. 2022-04-08 14:49:28.219@ubuntu1604@6872@LM_INFO@.7002|VDDK_PhoneHome: VddkVacWriteMember : Empty field to serialize for key edition at line 217. 2022-04-08 14:49:28.430@ubuntu1604@6872@LM_WARNING@.7002|VDDK_PhoneHome: Curl perform failed in VddkVacHttpsPost with error code 77 at 92. 2022-04-08 14:49:28.640@ubuntu1604@6872@LM_WARNING@.7002|VDDK_PhoneHome: Curl perform failed in VddkVacHttpsPost with error code 77 at 92. 2022-04-08 14:49:28.641@ubuntu1604@6872@LM_WARNING@.7002|VDDK_PhoneHome: Curl perform failed in VddkVacHttpsPost with error code 77 at 92. 2022-04-08 14:49:28.642@ubuntu1604@6872@LM_WARNING@.7002|VDDK_PhoneHome: VddkVacPushData : Failed to post data to VC/PH at line 1706. 2022-04-08 14:49:28.854@ubuntu1604@6872@LM_WARNING@.7002|VDDK_PhoneHome: VddkVacPushProductInstanceData : Failed to push product instance data at line 1885. 2022-04-08 14:49:29.068@ubuntu1604@6872@LM_WARNING@.7002|VDDK_PhoneHome: Curl perform failed in VddkVacHttpsPost with error code 77 at 92. 2022-04-08 14:49:29.279@ubuntu1604@6872@LM_WARNING@.7002|VDDK_PhoneHome: Curl perform failed in VddkVacHttpsPost with error code 77 at 92. 2022-04-08 14:49:29.279@ubuntu1604@6872@LM_WARNING@.7002|VDDK_PhoneHome: Curl perform failed in VddkVacHttpsPost with error code 77 at 92. 2022-04-08 14:49:29.280@ubuntu1604@6872@LM_WARNING@.7002|VDDK_PhoneHome: VddkVacPushData : Failed to post data to VC/PH at line 1706. 2022-04-08 14:49:29.360@ubuntu1604@6872@LM_WARNING@.7002|VDDK_PhoneHome: VddkVacPushVendorProductData : Failed to push vendor product data at line 2181. 2022-04-08 14:49:29.481@ubuntu1604@6872@LM_INFO@.7002|Signal 15 (Terminated) catched in [starter].

dbackup · ‎04-07-2022

I have the same situation

bluefirestorm · ‎01-10-2022

There is no obvious API in the documentation to get the unallocated sectors. There is the QueryAllocatedBlocks call but the inverse call (QueryUnallocatedBlocks) is not there. I also don't see the re...

There is no obvious API in the documentation to get the unallocated sectors. There is the QueryAllocatedBlocks call but the inverse call (QueryUnallocatedBlocks) is not there. I also don't see the relationship of blocks to sectors written out anywhere. I have no idea how the writing of zeroes would be done with just purely VDDK API.

blackcomb84 · ‎01-09-2022

Thanks @bluefirestorm!! I was trying to mimic an unmap behavior by writing zeros and then issuing Shrink API from a backup-restore app. I tried to use the Shrink API as I could not find a direct unm...

Thanks @bluefirestorm!! I was trying to mimic an unmap behavior by writing zeros and then issuing Shrink API from a backup-restore app. I tried to use the Shrink API as I could not find a direct unmap API in the VDDK documentation. Given that Shrink API does not work if issued from a remote node, is there any other way we can unmap/punch hole a certain region of vmdk?

bluefirestorm · ‎01-07-2022

That seems to be the only difference for one to be local or remote disk (absence/presence of server credentials). The most explicit statement is in page 67 with the walk-through of the sample progra...

That seems to be the only difference for one to be local or remote disk (absence/presence of server credentials). The most explicit statement is in page 67 with the walk-through of the sample program. A call to VixDiskLib_Connect() creates a connection to disk. If host cnxParams.serverName is null, as it is without the -host argument, a connection is made to hosted disk on the local host. Otherwise a connection is made to managed disk on the remote host. With -ssmoref argument, advanced transport is used.

blackcomb84 · ‎01-07-2022

Correct if my understanding is wrong, If the connection is made without any host IP address(like the code snippet below) then Shrink API should reclaim the blocks. This means the program must be run...

Correct if my understanding is wrong, If the connection is made without any host IP address(like the code snippet below) then Shrink API should reclaim the blocks. This means the program must be run on the ESXi node itself where the disk is hosted. VixDiskLibConnectParams cnxParams = {0}; vixError = VixDiskLib_ConnectEx(&cnxParams, 0, ssMoRef, transportModes, &connection); Whereas if the connection is made from a remote node to a ESX host by providing host IP, credentials and Shrink API is called, it does not reclaim the blocks. So, for a shrink API to work the program should execute on the ESXi host directly like how a vmkfstools command is run?

bluefirestorm · ‎01-07-2022

From the PDF doc, there are two ways to open a disk. One as local and another as remote. Open as remote the VxDiskLib_Connect would have credentials passed while open as local disk have NULLs. If yo...

From the PDF doc, there are two ways to open a disk. One as local and another as remote. Open as remote the VxDiskLib_Connect would have credentials passed while open as local disk have NULLs. If you look at the documentation PDF TOC, some functions are explicit as local, "Grow an Existing Local Disk", "Shrink an Existing Local Disk" while other calls are "Read Sector from a disk" while another "Open a Local or Remote Disk". Apart from that, Shrink Disk documentation indicates it won't work for "Managed disks", which seems like another term for "remote disks".

blackcomb84 · ‎01-07-2022

"vmkfstools open as a local disk while your API is opening as remote disk" Sorry I fail to understand this. Its the same vmdk I used for both API and vmkfstools. The vmdk is hosted on a local VMFS d...

"vmkfstools open as a local disk while your API is opening as remote disk" Sorry I fail to understand this. Its the same vmdk I used for both API and vmkfstools. The vmdk is hosted on a local VMFS datastore which is created out of HDD attached to the ESXi host. I connect to the Vcenter(VixDiskLib_ConnectEx) for establishing the connection before opening the disk for Shrink. Does it make a difference? Could you kindly elaborate on the differences of remote disk vs local disk? By remote you meant managed disks and local for hosted disks?

bluefirestorm · ‎01-07-2022

The possible difference is that vmkfstools open as a local disk while your API is opening as remote disk. Notice both the documentation and the log snippet indicate "shrink an existing local disk". ...

The possible difference is that vmkfstools open as a local disk while your API is opening as remote disk. Notice both the documentation and the log snippet indicate "shrink an existing local disk".

blackcomb84 · ‎01-07-2022

Thanks for your response! Here's what I have tried now but still unsuccessful, Filled the entire disk(a 4GB VMDK) with zeros. From within the guest OS, did "dd if=/dev/zero of=/dev/sdd bs=1048576 ...

Thanks for your response! Here's what I have tried now but still unsuccessful, Filled the entire disk(a 4GB VMDK) with zeros. From within the guest OS, did "dd if=/dev/zero of=/dev/sdd bs=1048576 count=4096" Shutdown the VM and verified the disk has only zeros. # du -sh ubun-vm1-restore_2-flat.vmdk 4.0G ubun-vm1-restore_2-flat.vmdk # od -x ubun-vm1-restore_2-flat.vmdk 0000000 0000 0000 0000 0000 0000 0000 0000 0000 * 40000000000 Used the shrink API call to reclaim the blocks, ideally when shrink API returns all the blocks should have been reclaimed and usage should be 0. But it still shows 4GB used, # du -sh ubun-vm1-restore_2-flat.vmdk 4.0G ubun-vm1-restore_2-flat.vmdk The API call sequence used, 1. VixDiskLib_ConnectEx() 2. VixDiskLib_Open() with flags 0. Tried using flag VIXDISKLIB_FLAG_OPEN_SINGLE_LINK as well. 3. VixDiskLib_Shrink(vDiskHandle, shrinkProgressFunc, NULL) -> Call returned success 4. VixDiskLib_Close() 5. VixDiskLib_Disconnect() Snippet from vixDiskLib.log, 2022-01-07T07:44:59.714Z| host-5243| I125: Opening file [datastore-2M282402Q0] ubun-vm1-restore/ubun-vm1-restore_2.vmdk (vpxa-nfc://[datastore-2M282402Q0] ubun-vm1-restore/ubun-vm1-restore_2.vmdk@10.1.20.108:902) 2022-01-07T07:44:59.796Z| host-5243| I125: DISKLIB-LINK : Opened 'vpxa-nfc://[datastore-2M282402Q0] ubun-vm1-restore/ubun-vm1-restore_2.vmdk@10.1.20.108:902' (0x8): custom, 8388608 sectors / 4 GB. 2022-01-07T07:44:59.798Z| host-5243| I125: DISKLIB-LIB : Opened "vpxa-nfc://[datastore-2M282402Q0] ubun-vm1-restore/ubun-vm1-restore_2.vmdk@10.1.20.108:902" (flags 0x8, type custom). 2022-01-07T07:44:59.798Z| host-5243| I125: VixDiskLib: VixDiskLib_Shrink: Shrink an existing local disk. 2022-01-07T07:44:59.798Z| host-5243| I125: DISKLIB-LIB : Shrink (Synchronous) chain 7FFFD84C10C8. 2022-01-07T07:44:59.798Z| host-5243| I125: VixDiskLib: VixDiskLib_Close: Close disk. Tried the vmkfstools -K to reclaim the blocks on the same vmdk # du -sh ubun-vm1-restore_2-flat.vmdk 4.0G ubun-vm1-restore_2-flat.vmdk # vmkfstools -K ubun-vm1-restore_2.vmdk vmfsDisk: 1, rdmDisk: 0, blockSize: 1048576 Hole Punching: 100% done. # du -sh ubun-vm1-restore_2-flat.vmdk 0 ubun-vm1-restore_2-flat.vmdk What could possibly be going wrong here that VixDiskLib_Shrink() is unable to reclaim the zeroed blocks which "vmkfstools -K" is able to reclaim?

bluefirestorm · ‎01-06-2022

Somehow the first paragraph in your first post didn't enter my brain. Sorry about that. It is possible that (1) the write of zeroes through API is not working properly (2) something wrong with the...

Somehow the first paragraph in your first post didn't enter my brain. Sorry about that. It is possible that (1) the write of zeroes through API is not working properly (2) something wrong with the shrink call (3) both (1) or (2) I suppose the way you are testing this is create a large file in a sparse disk (e.g. create 2GB file in an 8GB sparse VMDK and then delete the 2GB file). To test for (1) You could create a large file with a fixed hexadecimal recognisable pattern (such as 0xDEADBEEF) and delete that file. After you done with the write of zeroes through API, check to see whether that obvious pattern still exists in the VMDK with a hexadecimal dump To test for (2) You create a large file with all zeroes (such using dd input from /dev/zero) and delete that file (to mimic the punching of zeroes) and see whether the shrink API call works. Did you try to close the virtual disk set after the writing of zeroes (to flush it to storage) and then re-open to shrink?

blackcomb84 · ‎01-06-2022

Yes, the API documentation do say that. But I did not see the desired result.

bluefirestorm · ‎01-06-2022

From the documentation https://developer.vmware.com/docs/14686/virtual-disk-development-kit-programming-guide--7-0-3--/GUID-2F6A805D-9A92-4A93-AB67-349432CEC5A6.html In the API, use VixDiskLib_Wr...

From the documentation https://developer.vmware.com/docs/14686/virtual-disk-development-kit-programming-guide--7-0-3--/GUID-2F6A805D-9A92-4A93-AB67-349432CEC5A6.html In the API, use VixDiskLib_Write() to zero out unused blocks, and VixDiskLib_Shrink() to reclaim space.

blackcomb84 · ‎01-06-2022

I use the VDDK library(version 6.7) API VixDiskLib_Write() to write zeros to the unused blocks of a vmdk (a thin provisioned disk). Then issue VixDiskLib_Shrink() to reclaim the blocks which has only...

I use the VDDK library(version 6.7) API VixDiskLib_Write() to write zeros to the unused blocks of a vmdk (a thin provisioned disk). Then issue VixDiskLib_Shrink() to reclaim the blocks which has only zero's, the call returns immediately with success but no blocks are reclaimed. The VM is in shutdown state during the whole process. But, if I use the command vmkfstools --punchzero <vmdk path> (the same vmdk where VixDiskLib_Shrink() did not reclaim), the blocks with zeros are reclaimed and only non-zero blocks are consumed. Isn't VixDiskLib_Shrink() supposed to work the same as --punchzero?

jamesray · ‎12-22-2021

I got the same problem. Do you have any solution for this situation ?

DevVM19912021 · ‎12-02-2021

When I use the vddk7.0.3 to Open disk with mode "independent persistent", but I got the error 2021-12-01T15:21:27.580+07:00 error -[18961] [Originator@6876 sub=vimaccess] GetFileName: Cannot create...

When I use the vddk7.0.3 to Open disk with mode "independent persistent", but I got the error 2021-12-01T15:21:27.580+07:00 error -[18961] [Originator@6876 sub=vimaccess] GetFileName: Cannot create disk spec for disk [datastore2] test.vmdk. Anyone, please help to fix this error?

ocdata · ‎11-23-2021

In my case, the issue was resolved by using VDDK 6.7 to backup a quiesced snapshot image on a vSphere server running 6.7. The moref snapshot field passed to *ConnectEx appears to have no effect in ei...

In my case, the issue was resolved by using VDDK 6.7 to backup a quiesced snapshot image on a vSphere server running 6.7. The moref snapshot field passed to *ConnectEx appears to have no effect in either 6.7 or 7.0.

dbackup · ‎11-23-2021

Yes, snapshot MoRef is needed to open quiesced snapshot file in some circumstances, like double snapshot files.

ocdata · ‎11-22-2021

Did you just have to pass in the snapshot ref name, like "snapshot-1234" into connectEX? I seem to be getting the same errors trying to open snapshot files from quiesced snapshots even when I supply ...

Did you just have to pass in the snapshot ref name, like "snapshot-1234" into connectEX? I seem to be getting the same errors trying to open snapshot files from quiesced snapshots even when I supply the snapshot ref.

All

All Posts

Re: VixDiskLib_Open fails in VDDK 7.0 using a vmPath in the vmxSpec

VDDK 7.0.3 print VDDK_PhoneHome: Curl perform failed in VddkVacHttpsPost with error code

Re: Cannot open disk with mode independent persistent

Re: VixDiskLib_Shrink does not reclaim any zero blocks

Re: VixDiskLib_Shrink does not reclaim any zero blocks

Re: VixDiskLib_Shrink does not reclaim any zero blocks

Re: VixDiskLib_Shrink does not reclaim any zero blocks

Re: VixDiskLib_Shrink does not reclaim any zero blocks

Re: VixDiskLib_Shrink does not reclaim any zero blocks

Re: VixDiskLib_Shrink does not reclaim any zero blocks

Re: VixDiskLib_Shrink does not reclaim any zero blocks

Re: VixDiskLib_Shrink does not reclaim any zero blocks

Re: VixDiskLib_Shrink does not reclaim any zero blocks

Re: VixDiskLib_Shrink does not reclaim any zero blocks

VixDiskLib_Shrink does not reclaim any zero blocks

Re: Fail to open disk with vddk 7.0.2. (Cannot get compatible version for vim)

Cannot open disk with mode independent persistent

Re: How to backup VM with two delta file through VDDK

Re: How to backup VM with two delta file through VDDK

Re: How to backup VM with two delta file through VDDK