I have Linux 64 bit VM. I have enable CTK and then taken a snashots.I have taken about 5 snapshots before that and deleted all.
Now by calling QueryDiskChangedArea("*") got a vector of offset to write and i did active blocks written to destination VM.I used base disk to open and write it.
When i power on the vm its showing the old files. New files whatever i copied between the snapshot creation and deletion is not showing.
Siva, you told that once we delete the snashot all the containts will be merged to base disk.If i power off the VM and open the current snapshot file and write it , its seems to be working.
Need a clarification on the steps.Can some one please clarify this.
with thanks,
Santosh
Hi,
Refer to the below text taken from VDDK Documentation. If you pass * it will return all the used blocks. Each snapshot object will have a identifier named changeID. You need to store it when you take a full backup and during the next QueryChangedDiskAreas, you need to send that instead of *. Also note, this should be done per disk, changeID will be different for each disk in the same snapshot itself.
Changed Block Tracking on Virtual Disks
On hosts running ESX/ESXi 4.0 and later, virtual machines can keep track of disk sectors that have changed. This is called changed block tracking. Its method in the VMware vSphere API is QueryChangedDiskAreas, which takes the following parameters:
■ |
■ | snapshot – Managed object reference to a Snapshot of the virtual machine. |
■ |
■ | startOffset – Byte offset where to start computing changes to virtual disk. The length of virtual disk sector(s) examined is returned inDiskChangeInfo. |
When you back up a snapshot for the first time, ChangeId should be unset, or unsaved, indicating that a baseline (full) backup is required. If you have a savedChangeId, it identifies the last time a backup was taken, and tells the changed block tracking logic to identify changes that have occurred since the time indicated by the saved ChangeId.
There are two ways to get this baseline backup:
./Siva.
1 |
Hi Shiva,
Thanks for reply. I have done some progress.Following is the code snippet using for querydiskchangedarea.
String changeId = "52 a0 df 53 4f 90 0b 19-7b 14 80 28 68 f7 d2 a3/13";
int diskDeviceKey = 2000;
DiskChangeInfo changes = null;
long position = 0, tot_len = 0;
do {
changes = vimPort.queryChangedDiskAreas(vmMor, snapshot, diskDeviceKey, position, changeId);
.........................
}
The issue is that if i pass "*" and write those block , its working fine. If i pass changedId as above (previous snapshot), queryChangedDiskAreas reporting vector of offset and length.But the files copied on source VM is not writing to destination VM. destination VM is on previous state itself. I have not encountered with any error though.
I got the change id and device key from the ESXi webservice. As of now its hard coded values.Let me know if i am doing any mistakes here.
I am not sure what could be the problem. This part had been pretty straight forward for me though. Say if you are creating a new file of about 2MB after a snapshot backup, and when you trigger the next queryChangedDiskAreas call, did you print the Start and lenght of the DiskChangeExtent structure? It should roughly correspond to the 2MB change you made. Other option to try would be to create a child disk to the pervious parent with the returned DiskChangeExtent and see how big the child disk gets.
Hi Shiva thanks for your reply...
Snapshot Name : TESTSNAP
START 526385152 LENGTH 65536
START 529137664 LENGTH 65536
START 529268736 LENGTH 65536
START 529924096 LENGTH 65536
START 530251776 LENGTH 131072
START 531693568 LENGTH 131072
START 794886144 LENGTH 65536
START 822280192 LENGTH 65536
START 914751488 LENGTH 65536
START 2673999872 LENGTH 131072
START 2677014528 LENGTH 65536
START 4821483520 LENGTH 65536
START 4823973888 LENGTH 65536
START 4824956928 LENGTH 65536
START 4826267648 LENGTH 65536
START 4826791936 LENGTH 65536
START 4969267200 LENGTH 983040
START 6969425920 LENGTH 65536
START 6969753600 LENGTH 65536
START 6970736640 LENGTH 65536
START 6971457536 LENGTH 65536
START 6972178432 LENGTH 65536
START 6977159168 LENGTH 65536
START 9116319744 LENGTH 131072
START 9118351360 LENGTH 65536
START 9118744576 LENGTH 65536
START 9121234944 LENGTH 131072
START 9121693696 LENGTH 65536
START 9154658304 LENGTH 65536
START 9253027840 LENGTH 131072
START 9691004928 LENGTH 65536
START 10179706880 LENGTH 131072
Start offset :smileylaugh: length 16106127360 tot len 3473408
I copied about 2.4MB file, QueryDiskChangedArea reported ~3.4MB. I have written these blocks from source VM to destination VM. Still its showing old files.
Shiva,
You mentioned about the creating child disk and parent disk. Can you please tell me the context where it will be used. As a part of CBT is it required to use it in some cases?
with thanks,
Sanotsh
You can se VixDiskLib_CreateChild function to create a child disk and write the incremental changes alone to that file. Then you should use VixDiskLib_Attach to parentdisk. Atleast you would know the latest changes are being made to the VMDK
Hi guys,
I have been observing your discussion and I am not quite sure what the overall goal is. To me it seems like PavanKumarMath is trying to write a replication application. At least I understand this from here.
@ PavanKumarMath
Now my question is: Do you want to replicate the VM directly from one server to another - where server can be vCenter, ESX or ESXi?
--
Thomas G.
I agree with u. But the thing is that if i pass "*" and write it to destination it is working fine. The only issue is that if i pass the revious snapshot Change_id.Then no change in the state of VM.Doesn't this rule out the above mentioned potential issue? I understand that CBT will be enabled only once. after that re-enable is not required ?
No CBT is enabled only one, just before full backup is taken. I think we are losing track of your actual objective? Thomas, who has answered below has helped me trouble shoot almost all the problems I faced, guess it is time to take his lead and answer his question to get back on track.
@Thomas,
You are right. I am writing the VM replication components based on CBT mechanism. Till now what i did that on the same ESXi machine : read the current VM's VMDK and write it to destination VM's VMDK. Copying only active blocks returned by QueryDiskChangedArea("*") API. This is working fine for me. If i pass the change id of previous snapshot and current snapshot MOR to QueryDiskChangedArea reporting the changed blocks.But when i read / write, changes are not replecting on destination VM. Still it is in previous state.
with thanks,
Santosh
Hi Santosh,
Copying only active blocks returned by QueryDiskChangedArea("*") API. This is working fine for me.
OK from now on I assume that this means that your baseline replication, i.e. the first replication run, works and that any following
If i pass the change id of previous snapshot and current snapshot MOR to QueryDiskChangedArea reporting the changed blocks.But when i read / write, changes are not replecting on destination VM. Still it is in previous state.
does not.
From here it seems to me that Siva has already successfully guided you to verify that the changeId based calls to QueryDiskChangedAreas(...) are done properly. So from now on I assume that you actually receive correct DiskChangeInfo objects, i.e. your handling of the source VM is assumed to be correct.
This brings me to destination machine. Following the discussion and also from this discussion's title it is suggested that there is something wrong with snapshot handling. So please, as properly as you can, step by step explain what you do to the destination machine. Especially:
And please really step by step, otherwise it will be very hard to help you with this problem, if at all 😉
Looking forward to hearing from you ...
--
Thoma G.
Thanks a lot for response.So far what i did that :
Create a dummy destination VM with same configuration as of source VM manually.
First pass :-
1. Create a snapshot of source VM.
2. Get the active blocks of Source VM.
3. Read the blocks by using nbd. Here opening the base disk (Not the snapshot disk file). In the vSphere client UI it is showing the snapshot file in the form of vmname/vmname-000001.vmdk is in use.Source VM is in powered on state.
4.Open the destination VM disk by using nbd. Write it to its base disk (vmname/vmname.vmdk) file. Destination VM is powered off.
Note:- For reading/writing using the VixDiskLibSample. Modified to read/write 1024 sectors at a time. This program will be running in the one of the VM on same ESXi host.Call it as a Appliance.
5. Remember the ChangeID of source VM snapshot. Change ID i am getting from the ESXi's Webservice.
6. Delete the Source VM snapshot.
After write is complete Power on the destination VM. It comes up with Source VM replica.
========================================================================================================================
Second pass:-
Write the files to source VM. I have written about 2.7 MB files.
1. Create a snapshot of Source VM.
2. Get the changed blocks by passing ChangeId of previous snapshot.QueryDiskChangedArea function reporting about 3.7 MB changed blocks.
And rest of the steps are same as above.
when i power on the destination VM. whatever files i copied during the second pass is not showing. BUT If follow the steps in the first pass, its shows the files whatever i copied.
==========================================================================================================================
To answer your questions i am not creating any snapshots on destination VM and its powered off.
Please let me know what could be the issue.If you need any clarification let me know.
with thanks,
Sanotsh
Hi Sanotsh,
Hhhhhmmmm your algorithm looks good to me at a frist sight. To be honest I am not sure what the problem is.
Very dumb idea, have you already tried it with a larger change, let's say 1 GB, just want to make sure that there is not any kind of caching problem around.
--
Thomas G.
One question. How and when is CBT enabled at the source in the above sequence?
Hi Siva,
Thanks for reply..
I enabled it during the first pass and reconfigured the VM. I have verified it on the vSphre client GUI also. Its showing ctkenable &scsi:ctkenable flag as true :
code snippet:
vmConfigSpec.setChangeTrackingEnabled(true);
ManagedObjectReference tmor
= vimPort.reconfigVMTask(
virtualMachine, vmConfigSpec);
with thanks,
Santosh