Windows 2012r2 disk event id 157 ESXi 5.1 when creating a snapshot...

[deejerydoo]

My environment:

Host:

HP z800 with internal SATA drives. No iSCSI or fiber or anything fancy.

ESXi 5.1.0,799733

Guests:

Windows Servers:

2008R2 (SBS)

2012

2012R2

Creating snapshots on the 2008r2 and the 2012 servers has always worked without any issues. However, whenever I create snapshots of the 2012r2 servers I always get the following error messages:

Log Name:      System

Source:        Ntfs

Date:          23/10/2014 12:26:19 PM

Event ID:      137

Task Category: (2)

Level:         Error

Keywords:      Classic

User:          N/A

Computer:      host.domain.local

Description:

The default transaction resource manager on volume \\?\Volume{4d63c1a0-5a35-11e4-80c8-000c293e51b6} encountered a non-retryable error and could not start.  The data contains the error code.

Log Name:      System

Source:        Microsoft-Windows-Ntfs

Date:          23/10/2014 12:26:20 PM

Event ID:      140

Task Category: None

Level:         Warning

Keywords:      (8)

User:          SYSTEM

Computer:      host.domain.local

Description:

The system failed to flush data to the transaction log. Corruption may occur in VolumeId: \\?\Volume{4d63c1a0-5a35-11e4-80c8-000c293e51b6}, DeviceName: \Device\HarddiskVolume4.

(The specified request is not a valid operation for the target device.)

I have found the following VMware KB article that appears to cover this exact problem:

VMware KB: Creating a quiesced snapshot of a Windows virtual machine generates Event IDs 50, 57, 137...

However, the article is missing some critical information on this issue... The article does not state whether these alerts indicate if there is likely to be any adverse effect on the systems in question or not. Is this a real issue, that requires remedial action, or simply the misreporting of a non-issue? Also, this article refers to two MS KB articles. However neither of these two articles are relevant to my environment.

So, do we have a problem that needs to be fixed or do we just have yet another Microsoft system log error/warning crying wolf that won't get fixed until Windows 10 Server, if we're very lucky or Fortune 100?

[Vdex_ie]

Have you openned a support request with Microsoft on this issue?

"This is a known Microsoft issue. Microsoft is reviewing how to address this issue with their Products Group. VMware cannot control the generation of these messages in the Windows Event Logs."

I believe that VMwares Point of View is that this is a MS issue, however a active bug or collaboration is probably open from the wording in that kb so I would recommend subscribing to it.

Regards

Vdex

[deejerydoo]

Hi Vdex,

Yes, I saw that line in the article too. However, despite this I still felt I had to ask for further clarification/qualification.

I guess what I am after is unambiguous confirmation as to whether we should be snapshotting our Windows systems that are displaying these errors or not? This article does not mention whether it is safe to continue snapshotting or to avoid it until the bug is further investigated or resolved.

The article also mentions all the major variants of Windows server OSes, but not 2012R2 specifically. 2012R2 is specifically the ONLY platform in my environment that is throwing these warnings and errors.

Also the article states two workarounds which are so very specific, they're not likely to apply to a huge number of configurations in the real world. If VMware are unsure of any potential adverse effects to the Windows guests then they should er on the side of caution and warn people not to use snapshotting for the affected systems, in this article.

Cheers,

David.

[Vdex_ie]

Would you try snapshotting it without Queise I wonder would it have something to do with that?

[deejerydoo]

It is totally down to a bug between VMware snapshotting processes interacting with ShadowCopy. Snapshots without quiescing the drive work without any errors.

I have now tested in two different, but similar environments. In both my non 2012R2 systems are all quiescing and snapshotting without any issues, whereas the 2012R2 systems are all displaying the problem.

This directly contradicts the VMware article mentioned above that identified this as a problem in all Windows server OSes, except 2012R2!?

[Vdex_ie]

I see. I was always under the impression that VMware was not considered with the workings of the guest? That they provide the infrastructure and platform, that the bug would be on MS side.

[deejerydoo]

There would have to be interaction between the VMware ESXi disk management subsystems through the VMware disk drivers installed into the Windows host. This looks like some kind of timing issue from the Windows side, while the Volume Shadow Copy Service is changing the state of the hard drive between read and read/write. the Windows disk subsystems are seeing this as hardware disconnections from the VMware hard drives in Windows.

VMware will point the finger at MS and MS may well point it back at VMware. Lets hope they do the right thing, whomever has broken something. Although, this KB article has been around for quite some time! But the article, very annoyingly, doesn't have a creation date, only a last updated date. However, the update history started waaay back in July 2012! Looks like it may be a problem that keeps on rearing it's ugly head?

In the meantime, I still want to know from VMware, whether we should stop using/relying on snapshotting on affected systems or not? I also still want to know why their KB article doesn't list Win2012R2 as one of the affected OSes?

:smileyconfused:

Two critical omissions from their article.

:smileyangry:

[deejerydoo]

OK. With the lack of any reasonable outcome on this I am going to start posing this issue in other support fora. Starting with Spiceworks...

VMware ESXi Windows Server guest Snapshot, event IDs 50, 58, 137, 140, 157... - Spiceworks