ERROR: New-Snapshot (Operation is not valid due to the current state of the object)

[pagne]

Hello guys!

I've been googlin around a bit, but cannot find a solution to my problem. Hopefully one of you skilled people can point me in the right direction:)

I've created a powerCLI script to take a snapshot of our most important servers before we do our weekly maintenance. The idea to run the script as a regular powershell script and then have it

1. Add powercli snapins and connect to virtualcenter servers
2. query the VC servers for VMs in a perticular folder
3. Run a foreach loop on these servers to create a snapshot with name $snapshotname

Currently i'm just running it on a test folder that I created in vSphere that contains 3 test VMs:

• Testmagne1 - Working normally
• Testmagne2 - I've renamed the folder for this VM in the datastore in order to generate an error.
• testmagne3 - Working normally

Since the servers (not the test servers, but the proper ones) are the most important servers in our environment I want to be able to catch exceptions and errors when i run this script. I do this with the Try - Catch - Finally functions.

The weird thing is this:

The output from Powershell/PowerCLI is different from the output in VirtualCenter(!)

In virtualcenter the output is as follows

• Testmagne1 - Snapshot is created successfully - As expected
• Testmagne2 - Snapshot creation fails as it cannot find the .vmx files - As expected
• Testmagne3 - Snapshot is created successfully - As expected

However, the output from powercli is slightly different:

• Testmagne1 - Snapshot is created successfully - As expected
• Testmagne2 - Snapshot creating fails as it cannot find the .vmx files - As expected
• Testmagne3 - Snapshot fails with the error message: Operation is not valid due to the current state of the object - Huh?

Apparently once the first error is generated all succeeding VMs get the error "Operation is not valid due to the current state of the object"

I've been pulling my hair out trying to figure this problem out, but I am not getting anywhere

Here's some additional information:

1. The Script (short version of it. It generates the same error):

--------------- START SCRIPT-----------------

$ErrorActionPreference = "Stop"

Add-PSSnapin VMware.VimAutomation.Vds, VMware.VimAutomation.Core | Out-Null

Connect-VIServer VIRTUALCENTER1, VIRTUALCENTER2 | Out-Null  # Sensored the VirtualCenter names

$servers = Get-VM -location "testfolder"

$snapshotname = "Testsnapshot"

$verifycreatesnapshot = Read-Host "You are about to create snapshots for $servers. Do you want to continue? y/n"

    if ($verifycreatesnapshot -eq "y"){

        Write-Host "Creating snapshots..."

        ForEach ($i in $servers) {

            Write-Host "Creating snapshot for $i"        

            Try {New-Snapshot -VM $i -Name $snapshotname | Out-Null}

            Catch{$_.exception | select * ; Write-Host "Failed to create snapshot for $i";} 

        }

        Write-Host "Command finished"

    }

    else{Write-Host "Operation cancelled by user"}

Read-Host "End of script. Press ENTER to close"

------------------ END SCRIPT -------------------

2. The error messages from PowerCli:

Error for Testmagne 2 (as expected):

File                    : [DATASTORE114] testmagne2/testmagne2.vmx

DynamicType             :

DynamicPropertyInternal :

FaultCause              :

FaultMessage            :

ErrorId                 : Client20_TaskServiceImpl_CheckServerSideTaskUpdates_O

                          perationFailed

ErrorCategory           : NotSpecified

TargetObject            :

RecommendedAction       :

SessionId               :

ConnectionId            : /VIServer=SENSORED:443/

Severity                : Error

Message                 : 31/10/2013 10:52:16    New-Snapshot        The operat

                          ion for the entity "testmagne2" failed with the follo

                          wing message: "File [DATASTORE114] testmagne2/testmagn

                          e2.vmx was not found"

Data                    : {ParameterValues}

InnerException          : VMware.Vim.VimException: The operation for the entity

                           "testmagne2" failed with the following message: "Fil

                          e [DATASTORE114] testmagne2/testmagne2.vmx was not fou

                          nd"

TargetSite              :

StackTrace              :

HelpLink                :

Source                  :

Failed to create snapshot for testmagne2

Error for testmagne3:

Creating snapshot for testmagne3

ErrorId           : Core_BaseCmdlet_UnknownError

ErrorCategory     : NotSpecified

TargetObject      :

RecommendedAction : Error occured while executing cmdlet: New-Snapshot. Check i

                    nner exception for more details.

SessionId         :

ConnectionId      :

Severity          : Error

Message           : 31/10/2013 10:52:17    New-Snapshot        Operation is not

                     valid due to the current state of the object.

Data              : {ParameterValues}

InnerException    : System.InvalidOperationException: Operation is not valid du

                    e to the current state of the object.

                       at VMware.VimAutomation.ViCore.Impl.V1.Task.ViCoreTaskCo

                    reServiceProviderImpl.BeginTaskCompletionPoll(List`1 taskLi

                    st)

                       at VMware.VimAutomation.Sdk.Impl.V1.Task.CoreTaskService

                    Impl.WaitForTask(IEnumerable`1 taskList, ProgressCallback p

                    rogressCallback)

                       at VMware.VimAutomation.Sdk.Util10Ps.BaseCmdlet.BaseCmdl

                    et.EndProcessingErrorHandled()

                       at VMware.VimAutomation.ViCore.Util10Ps.BaseCmdlet.BaseC

                    mdlet.EndProcessingErrorHandled()

TargetSite        : Void ThrowTerminatingError(System.Management.Automation.Err

                    orRecord)

StackTrace        :    at System.Management.Automation.MshCommandRuntime.ThrowT

                    erminatingError(ErrorRecord errorRecord)

HelpLink          :

Source            : System.Management.Automation

Failed to create snapshot for testmagne3

3. PowerCLI version

PowerCLI Version

----------------

   VMware vSphere PowerCLI 5.5 Release 1 build 1295336

---------------

Snapin Versions

---------------

   VMWare AutoDeploy PowerCLI Component 5.5 build 1262826

   VMWare ImageBuilder PowerCLI Component 5.5 build 1262826

   VMware License PowerCLI Component 5.5 build 1265954

   VMware VDS PowerCLI Component 5.5 build 1295334

   VMware vSphere PowerCLI Component 5.5 build 1295334

4. VirtualCenter version

VMware vCenter Server Version 5.0.0 Build 913577

vSphere Client Version 5.0.0 Build 913577

Hosts: VMware ESXi 5.0.0 Build 914586

If you need any additional information, just let me know

Any help is greatly appreciated

Thanks!

- Pagne

[LucD]

That error is rather general I'm afraid.

Does the inner exception show anything more ? Do a

$error[0].Exception

$error[0].Exception.InnerException

I tried to reproduce your test, but for me Step 3 works without a problem.

Is that from PowerShell v2 or v3 ?

Perhaps try starting your session with

powershell -version 2

Is the script running in a 32- or 64-bit environment ?

[pagne]

Thanks for your reply LucD!

The error commands you provided does not provide anything more than what I wrote initially:

Creating snapshot for testmagne3

ErrorId           : Core_BaseCmdlet_UnknownError

ErrorCategory     : NotSpecified

TargetObject      :

RecommendedAction : Error occured while executing cmdlet: New-Snapshot. Check i

                    nner exception for more details.

SessionId         :

ConnectionId      :

Severity          : Error

Message           : 31/10/2013 10:52:17    New-Snapshot        Operation is not

                     valid due to the current state of the object.

Data              : {ParameterValues}

InnerException    : System.InvalidOperationException: Operation is not valid du

                    e to the current state of the object.

                       at VMware.VimAutomation.ViCore.Impl.V1.Task.ViCoreTaskCo

                    reServiceProviderImpl.BeginTaskCompletionPoll(List`1 taskLi

                    st)

                       at VMware.VimAutomation.Sdk.Impl.V1.Task.CoreTaskService

                    Impl.WaitForTask(IEnumerable`1 taskList, ProgressCallback p

                    rogressCallback)

                       at VMware.VimAutomation.Sdk.Util10Ps.BaseCmdlet.BaseCmdl

                    et.EndProcessingErrorHandled()

                       at VMware.VimAutomation.ViCore.Util10Ps.BaseCmdlet.BaseC

                    mdlet.EndProcessingErrorHandled()

TargetSite        : Void ThrowTerminatingError(System.Management.Automation.Err

                    orRecord)

StackTrace        :    at System.Management.Automation.MshCommandRuntime.ThrowT

                    erminatingError(ErrorRecord errorRecord)

HelpLink          :

Source            : System.Management.Automation

I also tried with different versions of powershell, running on both 32 and 64 machines, but with no luck

Since you're not able to reproduce the problem, did you rename the VM folder of one of your machines to force an error?

Here's the deal: The error (Operation not valid due to...) only occurs after the first error. Meaning, once I get an error on testmagne2 (since I renamed the files) all the VMs that are processed after testmagne2 get the "Operation is not valid..." error even though the tasks complete successfully in virtual center.

Sorry if I was not clear on that

-Pagne

[LucD]

Yes, I did exactly the same as you.

1) Take a snapshot of the VM --> ok

2) rename the VM's folder and take a snapshot --> error

3) rename the VM's folder back to the original name and take a snapshot --> ok

Perhaps someone from the PowerCLI Dev Team can shed a light ?

[pagne]

And you still don't get the "Operation is not..." error on the last VMs? Hmmm weird

I tried again on a completely different, separated environment and got the same errors as before
VMware VirtualCenter Server 5.1.0 1364037 and VMware ESXi 5.1.0 1065491

[knikolov]

I also tried to reproduce the issue, but I couldn't. I created 3 VMs, renamed the folder of the second one and executed your script. The result was as expected: error for the second one and snapshots successfully created for the first and third vm. Do you do anything more than this?

[pagne]

knikolov wrote:

I also tried to reproduce the issue, but I couldn't. I created 3 VMs, renamed the folder of the second one and executed your script. The result was as expected: error for the second one and snapshots successfully created for the first and third vm. Do you do anything more than this?

Hey knikolov!

No, i do exactly that. If I copy-paste the script above and save it to a .ps1 file and then run it with powershell I get the errors above: VirtualCenter creates the snapshots (except for the one with the renamed folder) and everything is fine everything and as it is supposed to be. However, in the powershell window i get an error for all VMs after the one with the renamed folder.

When the script is finished and displaying the errors in the powershell window vSphere still has the last "create snapshot task" in progress. A few seconds later the task is completed and the snapshot is created.

To me (not and expert on this) it seems like this is happening

1. PowerCLI sends the "New-Snapshot" command to VirtualCenter
2. For some reason an error is generated and is reported back to powershell. Powershell displays the error (operation not valid....)
3. VirtualCenter receives the New-Snapshot command and executes it.
4. The New-Snapshot command completes and a new snapshot is created.

[knikolov]

I managed to reproduce it. The key to reproduce it seems to be $ErrorActionPreference = "Stop".

Without this it works just fine. We'll investigate further where the problem is.

As a temporary workaround you could set the error action preference to Continue and check for the $error variable instead of using try..catch mechanism.

[pagne]

Yes, in order to catch any exceptions/errors the errors has to be considered as terminating. The $erroractionpreference = "stop" takes care of this and considers all errors as terminating.

Great that you managed to reproduce the problem. Really appreciate your efforts!

I'll see if I can create a temporary workaround in the meantime. I guess checking whether the $error.count has increased after each New-Snapshot is a possibility.

[LucD]

Why don't you just add the -ErrorAction Stop on the New-Snapshot cmdlet, instead of setting it globally for the script ?

[pagne]

Yes, that is possible. The original script is a bit longer so I figured that it was easier to set the ErrorActionPreference right away. The output is the same though 😕

[knikolov · Accepted solution]

Setting -ErrorAction Stop locally on the cmdlet will not solve the problem. We have located and fixed the issue and the fix will be available in the next release. Until then you can use the $error variable to detect if the cmdlet has succeeded or not. You can clear the $error variable before calling the cmdlet and check if it's still empty after the call.

[pagne]

Alright awesome! I will try out the workaround in the meantime

Thanks a lot for your efforts both of you!:)

[godzone]

I have just installed 5.5 Rel 2 Patch 1 and am getting this same error, although it would appear that the snapshot is created regardless.  Is this error something that I can simply ignore ?  If the issue has been fixed, what version of powercli has the fix ?

[godzone]

Both New-Snapshot and Remove-Snapshot give the same 'Error'.

Is there a work around ?  Is this really just a 'reporting issue' as the cmdlets do seem to work as intended.

[godzone]

And another.  Set-NetworkAdapter gives the same error.

Am going to attempt to go back a few versions.

[godzone]

Noted the Get-NetworkAdapter also generates the error.

BUT also found that the 32-bit version of the CLI does not appear to have these issues.

[LucD]

Did you try the same with PowerCLI 5.8 R1 ?

[godzone]

Sorry, I wasn't aware that it existed.  Have just downloaded and will try again.

Yep all good now.

Thanks.

[RonPSSC]

I realize this is an old post but I also just came across these errors using PowerCLI v5.5. I'll not take credit for this "alternate" workaround but was advised that if you simply close and re-open your PowerCLI session, the error messages do not get generated. I tried this and can confirm this works.

Ron