Snapshot alarm but no snapshots exist

[m80arm]

Hi all,

Bit of a strange one here.  We have an alert to show any VM's that have a snapshot.  I noticed that one of our servers was showing this alert and when checking the datastore there were in fact some snaphots that hadn't consolodated properly.  No problem with this, I simply created a new snapshot and then deleted all snaphosts which resolved the issue.  The VM was still alerting to the snaphot so I browsed the datastore and could see the following

I've tried the following to resovle this:

1. Created another snapshot and delete all
2. Disable the alarm and re-enable it
3. vMotioned the VM to another host
4. Powered off the VM and re-added to the investory
5. Powered off the VM and sVmotioned the VM to another datastore

None of the above has worked.  When i sVmotioned the VM to another datastore I selected the option to provision the .vmdk's as thin but when checking after the sVmotion it's still showing as thick.  When attempting to sVmotion with the VM powered on I recevie the following error message:

Anyone has any other suggestions?

Thanks

Michael

[JimKnopf99]

Hi,

please take a look at that post

http://kb.vmware.com/selfservice/documentLinkInt.do?micrositeID=&popup=true&languageId=&externalID=1...

Also you could not do a sVmotion when there are Snapshots on the vm.

Frank

[m80arm]

Frank,

Thanks for the reply.  I actually dont think any snapshots exist as I can sVmotion and there are no delta files in the directory.  I think the snapshot alarm is a red herring:

What's confusing me is the 3 VMDK files with the -XXXXXXXXXXXXXXXXXX

The VM is using the base disks rather than the -XXXXXXXXXXXXXXX disks but I'm not to sure if I can just delete these disks

Michael

[a_p_]

Because the files have current time stamps, it looks like they are indeed in use. I'd recommend you take a look at the VM's latest vmware.log to see whether these files show up there.

André

[Gkeerthy]

are you using any backup software for taking the vm backup? these can also create snapshots.

[m80arm]

@ André - I think the timestamps are showing current as this was when the sVmotion was performed.  I'll check the log files and report back

@ Gkeerthy - Yes, we use Backup Exec 2010 but I don't think it'sa snapshot issue

Michael

[danw76]

I know this is not the correct place to be making this post, but I am getting constant emails everytime a new post is posted on this forum.  I have tried to adjust my settings so that I don't get emails for every new post, but surprisingly I can't find the option to change this preference.  I've spent the last 3 weeks accepting the fact that I'll have to delete each email as it comes in, which is very annoying.

Can anyone poit me directly to the place to change this preference?  Thanks.

Dan

[m80arm]

I've checked the latest vmware.log file and can see the following references to the strange disks:

Feb 28 13:59:22.531: vmx| DISK: OPEN scsi0:0 '/vmfs/volumes/4a28f09b-87f7b050-64a4-001e4f441d75/NEXDBTV01/NEXDBTV01.vmdk' persistent R[]

Feb 28 13:59:22.585: vmx| DISKLIB-VMFS  : "/vmfs/volumes/4a28f09b-87f7b050-64a4-001e4f441d75/NEXDBTV01/NEXDBTV01-delta.vmdk" : open successful (10) size = 6459273216, hd = 43337944. Type 8

Feb 28 13:59:22.585: vmx| DISKLIB-DSCPTR: Opened [0]: "NEXDBTV01-delta.vmdk" (0xa)

Feb 28 13:59:22.585: vmx| DISKLIB-LINK  : Opened '/vmfs/volumes/4a28f09b-87f7b050-64a4-001e4f441d75/NEXDBTV01/NEXDBTV01.vmdk' (0xa): vmfsSparse, 41931698 sectors / 20 GB.

Feb 28 13:59:22.598: vmx| DISKLIB-VMFS  : "/vmfs/volumes/4a28f09b-87f7b050-64a4-001e4f441d75/NEXDBTV01/NEXDBTV01-ac1eb6efaa73e4a-flat.vmdk" : open successful (14) size = 21469029376, hd = 41634009. Type 3

Feb 28 13:59:22.598: vmx| DISKLIB-DSCPTR: Opened [0]: "NEXDBTV01-ac1eb6efaa73e4a-flat.vmdk" (0xe)

Feb 28 13:59:22.598: vmx| DISKLIB-LINK  : Opened '/vmfs/volumes/4a28f09b-87f7b050-64a4-001e4f441d75/NEXDBTV01/NEXDBTV01-ac1eb6efaa73e4a.vmdk' (0xe): vmfs, 41931698 sectors / 20 GB.

Feb 28 13:59:22.598: vmx| DISKLIB-CHAINESX : ChainESXOpenSubChain: numLinks = 2, numSubChains = 1

Feb 28 13:59:22.636: vmx| DISKLIB-LIB   : Opened "/vmfs/volumes/4a28f09b-87f7b050-64a4-001e4f441d75/NEXDBTV01/NEXDBTV01.vmdk" (flags 0xa, type vmfs).

Feb 28 13:59:22.637: vmx| DISKLIB-LIB   : Content-ID check is now enabled.

Feb 28 13:59:22.638: vmx| DISK: Disk '/vmfs/volumes/4a28f09b-87f7b050-64a4-001e4f441d75/NEXDBTV01/NEXDBTV01.vmdk' has UUID '60 00 c2 92 c4 06 0e b3-98 03 88 9f aa 9d de 6e'

Feb 28 13:59:22.638: vmx| DISK: OPEN '/vmfs/volumes/4a28f09b-87f7b050-64a4-001e4f441d75/NEXDBTV01/NEXDBTV01.vmdk' Geo (2610/255/63) BIOS Geo (0/0/0)

Feb 28 13:59:22.639: vmx| DISKLIB-CTK   : Auto blocksize for size 41931698 is 128.

Feb 28 13:59:22.705: vmx| DISKLIB-CBT   : Initializing ESX kernel change tracking for fid 43337944.

Feb 28 13:59:22.705: vmx| DISKLIB-CBT   : Successfuly created cbt node 1d548da-cbt.

Feb 28 13:59:22.705: vmx| DISKLIB-CBT   : Opening cbt node /vmfs/devices/cbt/1d548da-cbt

Feb 28 13:59:29.042: vmx| DISKLIB-CBT   : Exceeded max. # of ioctl calls when looking for in use disk areas. Marking remainder of this from offset 538136064 as in use.

Feb 28 13:59:29.044: vmx| DISK: Change tracking for disk scsi0:0 is now enabled.

Feb 28 13:59:29.101: vmx| DISK: OPEN scsi0:1 '/vmfs/volumes/4a28f09b-87f7b050-64a4-001e4f441d75/NEXDBTV01/NEXDBTV01_1.vmdk' persistent R[]

Feb 28 13:59:29.149: vmx| DISKLIB-VMFS  : "/vmfs/volumes/4a28f09b-87f7b050-64a4-001e4f441d75/NEXDBTV01/NEXDBTV01_1-delta.vmdk" : open successful (10) size = 8707481600, hd = 638062256. Type 8

Feb 28 13:59:29.149: vmx| DISKLIB-DSCPTR: Opened [0]: "NEXDBTV01_1-delta.vmdk" (0xa)

Feb 28 13:59:29.149: vmx| DISKLIB-LINK  : Opened '/vmfs/volumes/4a28f09b-87f7b050-64a4-001e4f441d75/NEXDBTV01/NEXDBTV01_1.vmdk' (0xa): vmfsSparse, 104858304 sectors / 50 GB.

Feb 28 13:59:29.160: vmx| DISKLIB-VMFS  : "/vmfs/volumes/4a28f09b-87f7b050-64a4-001e4f441d75/NEXDBTV01/NEXDBTV01_1-9ace2ad6c00b0c6-flat.vmdk" : open successful (14) size = 53687451648, hd = 596250289. Type 3

Feb 28 13:59:29.160: vmx| DISKLIB-DSCPTR: Opened [0]: "NEXDBTV01_1-9ace2ad6c00b0c6-flat.vmdk" (0xe)

Feb 28 13:59:29.160: vmx| DISKLIB-LINK  : Opened '/vmfs/volumes/4a28f09b-87f7b050-64a4-001e4f441d75/NEXDBTV01/NEXDBTV01_1-9ace2ad6c00b0c6.vmdk' (0xe): vmfs, 104858304 sectors / 50 GB.

Feb 28 13:59:29.160: vmx| DISKLIB-CHAINESX : ChainESXOpenSubChain: numLinks = 2, numSubChains = 1

Feb 28 13:59:29.222: vmx| DISKLIB-LIB   : Opened "/vmfs/volumes/4a28f09b-87f7b050-64a4-001e4f441d75/NEXDBTV01/NEXDBTV01_1.vmdk" (flags 0xa, type vmfs).

Feb 28 13:59:29.223: vmx| DISKLIB-LIB   : Content-ID check is now enabled.

Feb 28 13:59:29.224: vmx| DISK: Disk '/vmfs/volumes/4a28f09b-87f7b050-64a4-001e4f441d75/NEXDBTV01/NEXDBTV01_1.vmdk' has UUID '60 00 c2 9a 6a 8b 54 7e-9d c3 6d a4 59 74 36 61'

Feb 28 13:59:29.224: vmx| DISK: OPEN '/vmfs/volumes/4a28f09b-87f7b050-64a4-001e4f441d75/NEXDBTV01/NEXDBTV01_1.vmdk' Geo (6527/255/63) BIOS Geo (0/0/0)

Feb 28 13:59:29.225: vmx| DISKLIB-CTK   : Auto blocksize for size 104858304 is 128.

Feb 28 13:59:29.268: vmx| DISKLIB-CBT   : Initializing ESX kernel change tracking for fid 638062256.

Feb 28 13:59:29.269: vmx| DISKLIB-CBT   : Successfuly created cbt node 23150eb2-cbt.

Feb 28 13:59:29.269: vmx| DISKLIB-CBT   : Opening cbt node /vmfs/devices/cbt/23150eb2-cbt

Feb 28 13:59:31.298: vmx| DISKLIB-CBT   : Exceeded max. # of ioctl calls when looking for in use disk areas. Marking remainder of this from offset 65626624 as in use.

Feb 28 13:59:31.305: vmx| DISK: Change tracking for disk scsi0:1 is now enabled.

Feb 28 13:59:31.337: vmx| DISK: OPEN scsi0:2 '/vmfs/volumes/4a28f09b-87f7b050-64a4-001e4f441d75/NEXDBTV01/NEXDBTV01_2.vmdk' persistent R[]

Feb 28 13:59:31.364: vmx| DISKLIB-VMFS  : "/vmfs/volumes/4a28f09b-87f7b050-64a4-001e4f441d75/NEXDBTV01/NEXDBTV01_2-delta.vmdk" : open successful (10) size = 100687872, hd = 141052211. Type 8

Feb 28 13:59:31.364: vmx| DISKLIB-DSCPTR: Opened [0]: "NEXDBTV01_2-delta.vmdk" (0xa)

Feb 28 13:59:31.364: vmx| DISKLIB-LINK  : Opened '/vmfs/volumes/4a28f09b-87f7b050-64a4-001e4f441d75/NEXDBTV01/NEXDBTV01_2.vmdk' (0xa): vmfsSparse, 20999004 sectors / 10.0 GB.

Feb 28 13:59:31.392: vmx| DISKLIB-VMFS  : "/vmfs/volumes/4a28f09b-87f7b050-64a4-001e4f441d75/NEXDBTV01/NEXDBTV01_2-c90359261cca5b0-flat.vmdk" : open successful (14) size = 10751490048, hd = 140396852. Type 3

Feb 28 13:59:31.392: vmx| DISKLIB-DSCPTR: Opened [0]: "NEXDBTV01_2-c90359261cca5b0-flat.vmdk" (0xe)

Feb 28 13:59:31.392: vmx| DISKLIB-LINK  : Opened '/vmfs/volumes/4a28f09b-87f7b050-64a4-001e4f441d75/NEXDBTV01/NEXDBTV01_2-c90359261cca5b0.vmdk' (0xe): vmfs, 20999004 sectors / 10.0 GB.

Feb 28 13:59:31.392: vmx| DISKLIB-CHAINESX : ChainESXOpenSubChain: numLinks = 2, numSubChains = 1

Feb 28 13:59:31.410: vmx| DISKLIB-LIB   : Opened "/vmfs/volumes/4a28f09b-87f7b050-64a4-001e4f441d75/NEXDBTV01/NEXDBTV01_2.vmdk" (flags 0xa, type vmfs).

Feb 28 13:59:31.411: vmx| DISKLIB-LIB   : Content-ID check is now enabled.

Feb 28 13:59:31.412: vmx| DISK: Disk '/vmfs/volumes/4a28f09b-87f7b050-64a4-001e4f441d75/NEXDBTV01/NEXDBTV01_2.vmdk' has UUID '60 00 c2 90 97 9f 64 ec-78 71 60 59 ec 49 50 86'

Feb 28 13:59:31.412: vmx| DISK: OPEN '/vmfs/volumes/4a28f09b-87f7b050-64a4-001e4f441d75/NEXDBTV01/NEXDBTV01_2.vmdk' Geo (1307/255/63) BIOS Geo (0/0/0)

Feb 28 13:59:31.412: vmx| DISKLIB-CTK   : Auto blocksize for size 20999004 is 128.

Feb 28 13:59:31.515: vmx| DISKLIB-CBT   : Initializing ESX kernel change tracking for fid 141052211.

Feb 28 13:59:31.516: vmx| DISKLIB-CBT   : Successfuly created cbt node 7aa4935-cbt.

Feb 28 13:59:31.516: vmx| DISKLIB-CBT   : Opening cbt node /vmfs/devices/cbt/7aa4935-cbt

Feb 28 13:59:32.778: vmx| DISKLIB-CBT   : Exceeded max. # of ioctl calls when looking for in use disk areas. Marking remainder of this from offset 816310784 as in use.

Feb 28 13:59:32.780: vmx| DISK: Change tracking for disk scsi0:2 is now enabled.

[a_p_]

@danw76

You can check your email notifications when you open "Profile" --> "Manage Email Notifications"

The direct link for you should be: http://communities.vmware.com/people/danw76?view=watches&numResults=15&watchObjectType=14

André

[a_p_]

Not sure what caused this issue (I'd guess the backup application did), but your current virtual disks have snapshots with the names in an unusual order, where the delta files have the names which are usually assigned to the flat file!? In addition to this the virtual disk size

NEXDBTV01.vmdk  -> NEXDBTV01-delta.vmdk

NEXDBTV01-ac1eb6efaa73e4a.vmdk ->  NEXDBTV01-ac1eb6efaa73e4a-flat.vmdk

NEXDBTV01_1.vmdk -> NEXDBTV01_1-delta.vmdk

NEXDBTV01_1-9ace2ad6c00b0c6.vmdk -> NEXDBTV01_1-9ace2ad6c00b0c6-flat.vmdk

NEXDBTV01_2.vmdk -> NEXDBTV01_2-delta.vmdk

NEXDBTV01_2-c90359261cca5b0.vmdk -> NEXDBTV01_2-c90359261cca5b0-flat.vmdk

What I would do is to run "Delete All" from the "Snapshot Manager" to merge the snapshot files into the base disks and then clean this up by either "Storage vMotion" the VM, which - depending on the version - should rename the files or clean this up manually (let me know if you need assistance with this).

André

[m80arm]

I've created a temp snapshot and "deleted all" and I'm currently in the process of sVmotioning the VM to another datastore with a different block size to ensure VAAI is not used in the copy.

I'll update tomorrow with the results.  It looks like the disks are finally changed to thin but the -flat file is still showing as -XXXXXXXXXXXXXX.

Michael

[m80arm]

After creating a new snapshot and sVmotioning the VM to another datastore with a different block size the VM is still showing the same disks

Anyone got anything else I can try before raising it with VMware support?

Michael

[a_p_]

Did you run the "Delete All" with the VM powered on? In this case the vmware??.log file should contain information what happened while deleting the snapshots.

André

[JimKnopf99]

Hi,

take a look at this post.

http://networkadminkb.com/KB/a347/how-to-fix-virtual-disk-is-in-link-cloned-mode-error-message.aspx

Frank

[JimKnopf99 · Accepted solution]

What you also could test is clone your vm or use the converter to build it new.

That should also remove all of your snaps.

Frank

[sparrowangelste]

Jimknopf's suggestion of using converter/cloning should work since the delete all snap doesnt

[m80arm]

@ JimKnopf99 - cloning worked.

Very strange but I thought the sVmotion would have produced the same reslts as a clone operation but obviosuly not.

Thanks for all your help guys.

Much appreciated.

Michael