Snapshot bug in VIX 1.6.2?

[imhotep65]

Hello: Trying to determine if this is a VIX 1.6.2 bug, something that I am doing incorrectly in my code or not doing in my code.

I’m running on ESX Server 3i, 3.5.0, Build 123629.

Appreciate any help, suggestions, feedback.

What I am seeing is that, if a snapshot is taken for a gueston a different host than it was originally registered to and that guest is moved back to the original host, that new snapshot gets lost at least to the library. In this case, moved means "unregistered/registered" from one host to another.

This is the example (in this case guest is on a NAS, accessible by both host0 and host1):

Register guest (powered off, no snapshot) on host0, unregister from host0, register on host1, power up guest take snapshot of guest while registered on host1, poweroff guest, unregister from host1, register on host0 and try to revert guest to the snapshot (created while registered on host1) on host0 <--- this last step fails, the RevertToSnapshot API call returns a VIX_E_OBJECT_NOT_FOUND.

If instead of doing a revert step above, I try to find the created snapshot, using API calls such as GetCurrent, GetNamedSnapshot, even GetRootSnapshot if that snapshot turned out to be the only snapshot created in this instance, VIX cannot find it – returns invalid handle value. The snapshot does show up in the Snapshot Manager window on VI Client.

Note, if the application is restarted, the snapshots are found when the guest is registered on host0, but the application in which this

is happening is a long-lived application that is not stopped and restarted.

Thanks in advance.

[fixitchris]

Can you revert to snapshot from VI Client?

Are you snapshotting the RAM as well?

[imhotep65]

Hi: First, thanks for taking the time to respond to my post. In answer to your questions:

1) Yes, I can revert to the snapshot that "cannot be found" by VIX from the VI Client.

2) Yes, I am snapshotting RAM (using option VIX_SNAPSHOT_INCLUDE_MEMORY in CreateSnapshot API call).

So, I've attached a few screenshots from VI Client to illustrate what is happening.

The first screenshot (vi_SM01) shows the guest machine xppro02 (powered off) with a single root snapshot (snapshot: 0) on host0

The second screenshot (vi_SM02) shows the same guest machine xppro02 after it has been unregistered from host0, registered on host1, powered up, snapshot taken, powered off, unregistered from host1 and re-registered with host0 (as you see the root now has a child snapshot, "shapshot: 1", which is also the "current snapshot" as indicated by the "You are here" icon)

The last snapshot (vi_SM03) shows guest xppro02 back on host0 and an attempt to revert to the current snapshot obtained by GetCurrentSnapshot, which should be "snapshot: 1". This is where the RevertSnapshot returns a VIX_E_OBJECT_NOT_FOUND error.

It doesn't seem find or cannot find "snapshot: 1" which was created while xppro02 was registered on host1.

Related to your original question, I can <Go to> "snapshot: 1" from the Snapshot Manager.

Hope this makes the issue a bit clearer.

Thanks in advance.

[fixitchris]

If you run GetCurrentSnapshot against vi_SM03, does it return Snapshot 0?

Download VMwareTasks Lib from Codeplex and tell us what the Snapshot Collection contains against vi_SM03.

Chris

[imhotep65]

Hi: Let me add some more details that I left out of my original post. I'm doing develpment /testing of my VIX-based applications on Linux, specifically, Ubuntu 7.10 (Gutsy Gibbon) -- JeOS version. The results I've described are from a single run of a driver program that encapsulates some of the behavior in my applications in order to specifically diagnose the snapshot issue. I put in sleep calls in certain points in the code so that I could quickly grab a screenshot of the snapshot tree in the Snapshot Manager.

-

---

So this is the sequence of calls in the program:

Guest is registered on host0,

Guest moved to host1 (unregister on host0, register on host1)

Create snapshot on host1

Move guest back to host0 (unregister on host1, register on host0)

*Here: I check snapshot states, root, child BEFORE revert (see below)

Revert to snapshot created on host1.

**Here I check snapshot states, root, child AFTER revert (see below)

-

---

I put in calls to GetRootSnapshot/GetChild (to get child of root) and GetCurrentSnapshot when snapshot state was as shown in screenshot vi_SM02 on host0 (see above *):

GetRootSnapshot returned "snapshot: 0", the GetChild operation returned the error: VIX_E_FILE_NOT_FOUND (error text: A file by that name does not exist).

GetCurrentSnapshot returned the root, snapshot: 0, not the child of the root "snapshot: 1", which is the current snapshot as shown in the snapshot tree in vi_SM02.

Next, the revert is called and I get the state illustrated in vi_SM03 on host0:

Now, I call the same sequence of Snapshot API calls above on the state shown in vi_SM03 after the revert call (see above **):

GetRootSnapshot returned snapshot:0, the GetChild operation returned the error: VIX_E_FILE_NOT_FOUND (error text: A file by that name does not exist).VIX cannot find

the child snapshot even though it is in the snapshot tree in vi_SM02.

GetCurrentSnapshot returned the root, which is correct since VIX reverted from the root, snapshot:0, and not the child, snapshot: 1, as shown in the snapshot tree in vi_SM03,

which is not correct.

The sequence of API calls do not seem to work in this case and it seems like it should. I should be able to move guests between hosts and revert from snapshots created on any hosts.

Lastly, I ran a SEPARATE program against the state in vi_SM03 that opened the guest and gathered the snapshot state using the API calls above:

GetRootSnapshot returned snapshot: 0, the GetChild operations returned snapshot: 1 (no error as above) and GetCurrentSnapshot returned the root snapshot, snapshot: 0, which is

correct.

I would have to port the test program that I am using to diagnose this problem to Mono .NET on Linux to use VMWare Tasks library (if it works with it) or port it to a Windows box so that I can call the Snapshot Collection method before and after the Revert operation as describd above, or in a separate program, to determine the state of the snapshot tree. I can do this, but at the moment, my time is very limited.

Sorry about the lenghty post, I'm just trying to add as much detail as possible.

Thanks in advance.

[fixitchris]

Can you post the code?

[imhotep65]

Flow:

Register guest on host1 ("from" host)

Move guest to host2 ("to" host) -- unregister on host1/register on host2

Revert (if snapshot exists) or power-on (if no snapshot) on host2

Snapshot guest on host2, power-off guest

Move guest to host1 -- unregister on host2/register on host1

Attempt to revert to current snapshot on host1 (fails)

I am attaching the program to the post since the formatting doesn't look that great.

Thanks.

[imhotep65]

Hi: Any thoughts on this snapshot issue? Is this a VIX bug? There doesn't seem to be a good workaround for this. One possibility is disconnecting/reconnecting to host hoping that it invalidates state, but that is not a feasible option for me even if it worked.

Anyway to get more information from VIX -- a debug logging option that may provide more information?

[fixitchris]

The code looks good to me.

There is an issue with creating concurrent connections to multiple hosts on multiple threads. I am not sure if the scope is limited to multiple threads only though. Conn/Disconnect after each op would be the next thing I would do.

Can you output the contents of the snapshot collection after before each snapshot op you attempt?

[imhotep65]

Just an update: Invalidating the host context by disconnecting, then reconnecting seems to work. So a previous post I posted the flow of the operations and where the error was occuring (cut/pasting it here for convenience) -- ****marks original code, ++++marks changes/effect

Flow:

Register guest on host1 ("from" host)

Move guest to host2 ("to" host) -- unregister on host1/register on host2

Revert (if snapshot exists) or power-on (if no snapshot) on host2

Snapshot guest on host2, power-off guest

****Move guest to host1 -- unregister on host2/register on host1

+++Instead of doing the above, now its: Move guest to host1:

--Unregister guest on host2

--Disconnect from host1, reconnect to host1 (invalidate original host handle/context)

---Register guest on host1

****Attempt to revert to current snapshot on host1 (fails)

+++The revert to current snapshot on host1 succeeds using work-around.

The new snapshot tree in this case is vi_SM03_OK; as you can see, it reverted to snapshot: 1. In the original case, it failed to revert to the child snapshot in the snapshot tree in screenshot vi_SM03_FAIL. Also, the child could not be found by the library call GetChild(Root) (error returned was VIX_E_FILE_NOT_FOUND), after the revert, now with the work-around, the root and child snapshots are successfully found with no errors.

Invalidating that state via the disconnect/connect seems to fix it, but that work-around can be problematic in my system. In some cases, there may be 10's of guests registered to a host and if the work-around is used, that would mean I would have to go through and acquire new handles for all registered guests correct? The original handles related to the initial host handle are effectively rendered invalid via a disconnect; so I would need to get new handles via Open().

So, according to your post, there is an identified issue in the library that would cause this error?

I haven't gotten around to running VMWare Task Lib yet; I still need to convert my original code. Do you still think this is worth the effort? I have no problem doing this if it helps in diagnosing the problem.

Thanks in advance.

[fixitchris]

Thanks for doing all the research. This will definitely help VMware to make VIX better.

For all 10 guests you can use the same host handle, and unregister one by one. Then destroy the host handle and create a handle for host2 and reregister guests one by one.

This seems to be a similar issue to http://communities.vmware.com//thread/187177?tstart=0.

VMwareTasks will not solve your issue. Hopefully this will be resolved in 1.6.3.

I have run into a similar issue (creating two host handles at once) with vdisk-manager as well.

[imhotep65]

Thanks for the responses and help.

The only issue with unregistering/re-registering guests each time the host state needs to be invalidated is possible performace hits since after a Register, an Open has to be done to acquire a new handle for the guests assuming that the Disconnect invalidates the previous ones.

Anyway, I would have to implement this and determine if there are any performance issues; I'm just speculating now.

I hope this does get resolved in the next version. So far, I've found VIX to be a very nice, flexible and very practical library for my purposes.

Thanks again.

[fixitchris]

You would only invalidate the host handle after un/registering all 10 guests. No need to get a new host connection after the un/reg of each guest.

[mcr314]

Chris, any idea when 1.6.3 will be out?

Is there any chance for a patch for this specific problem?

(Is source for VIX available?)