Invoke-VMScript has stopped working from past couple days. The same VM has been working from past few months. But now when I run any Invoke-VMScript command, Invoke-VMScript: 6/24/2020 1:21:39 PM Invoke-VMScript A general system error occurred: vix error codes = (1, 0). Error is getting displayed
Invoke-VMScript -VM $TARGET_VM -ScriptText $Command_text -GuestUser $vm_user -GuestPassword '&rWn.
Based on the previous posts, I tried following things:
Here is the vmware.log excerpt:
2020-06-24T13:51:29.022Z| vcpu-0| I125: TOOLS autoupgrade protocol version 2
2020-06-24T13:51:29.022Z| vcpu-0| I125: Vix: [mainDispatch.c:4156]: VMAutomationReportPowerStateChange: Reporting power state change (opcode=2, err=0).
2020-06-24T13:51:29.022Z| vcpu-0| I125: TOOLS Received tools.set.version rpc call, version = 10304, setting type to 1 from guest OS
2020-06-24T13:51:29.022Z| vcpu-0| I125: Tools_SetVersionAndType did nothing; new tools version (10304) and type (1) match old Tools version and type
2020-06-24T13:53:50.874Z| vmx| I125: VigorTransportProcessClientPayload: opID=f0c9f8f-2e-0c76 seq=29344: Receiving GuestOps.CreateTemporaryFile request.
2020-06-24T13:53:51.023Z| vcpu-1| I125: VigorTransport_ServerSendResponse opID=f0c9f8f-2e-0c76 seq=29344: Completed GuestOps request with messages.
I don't think that VMware Tools from a snapshot cause any issues.
And if they did, the reinstall should have fixed that issue.
That logins don't work is understandable and is probably due to the AD secure channel being outdated/expired.
This connection is updated/refreshed automatically on an interval.
You should be able to reset the secure channel between the station and the AD DCs with the netdom command.
One step I would try at this point, would be to start ProcessMonitor from Sysinternals in the Guest OS.
Then do the Invoke-VMScript(Plus) call and check if the monitor shows any obvious errors.
LucD ProcessMonitor is showing lot of processes, could you please let us know which Process and Operation should I be looking for. For now, I filtered vmtoolsd.exe and there are lot of "Name not found' errors. Please suggest.
I turned on the Verbose switch, 'Created temp folder in guest OS' message is displayed but the folder name is not displayed. Below is the screenshot for the same:
Ok, the Verbose message should have listed the name of the directory (in $tempFolder).
Write-Verbose "Created temp folder in guest OS $tempFolder"
But nothing is shown, meaning that $tempFolder is empty.
So now we are sure that the issue is with the creation of the temporary folder.
You should have a look in the ProcessMonitor output for a CreateFile operation that failed
Below is the screenshot of ProcessMonitor. Filtered with 'CreateFile operation' and 'vmtoolsd.exe Process'. There are no failure events though
Filtered with 'CreateFile operation' and 'Result is not Success':
One more thing to investigate, on the VCSA can you open an SSH session?
Then go to folder /var/log/vmware/vpxd
Now change the VCSA logging level to verbose
Looks like there is some problem with the Domain which the VM was connected to. Tried creating a new user in AD and used the new user to Invoke-VM script but the new user also was getting the same error. Invoke-VM script was working fine with local admin. So switched to different AD, and now Invoke-VM script started working fine on the same VM.
There must have been some change to DC, but not sure what that is (tried comparing both the domains with gpresult but there was lot of differences so didn't proceed with the analysis).
We were still not able to find the root cause for this issue, but I'm not blocked anymore.
Thank you LucD for your help.
LucD faced this issue again, so tried 'Test-ComputerSecureChannel' command which you gave last time and it returned False.
Then tried 'Test-ComputerSecureChannel -repair' and after that it started working!
Thanks a ton, this time I'm glad this issue got resolved in few minutes instead of few days
Posting the solution here so that it might be helpful for someone else.
LucD, looks like the scripts are failing again with the same issue 'A general system error'. But if I do 'Test-ComputerSecureChannel -repair' then it starts working. But to execute 'Test-ComputerSecureChannel -repair' I have to login to the VM. Please suggest, if there is any VMWare PowerCLI command which can be used to do the same task (this would avoid me to manually login to VMs).
My scripts has to run with domain account.
This time I already logged into VM and ran the repair command. Next time when I get the same error, I'll try the below command with local admin first before logging into VM and execute repair command.
Invoke-VMScript -VM $TARGET_VM -ScriptText $Command_text -GuestUser "local_admin" -GuestPassword "admin_password"