Re: Invoke-VMscript on linux machine fails - An er...

pargit · ‎05-20-2019

Hi,

i have a post deploy script that uses the Invoke-VMscript to configure second network card, routes and other commands.

when running it against windows vm everything is ok. on linux machine i get this error after about 30-45 seconds:

Invoke-VMScript : 5/20/2019 11:03:50 AM Invoke-VMScript An error occurred while sending the request.

At D:\Automation\DeployVM\After_Install.ps1:265 char:13

+ Invoke-VMScript -VM $vm -GuestUser 'root' -GuestPassword 'Aa123456' ...

+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

+ CategoryInfo : NotSpecified: (:) [Invoke-VMScript], ViError

+ FullyQualifiedErrorId : Client20_VmGuestServiceImpl_DownloadFileFromGuest_DownloadError,VMware.VimAutomation.ViCore.Cmdlets.Commands.InvokeVmScript

Invoke-VMScript : 5/20/2019 11:03:50 AM Invoke-VMScript An error occurred while sending the request.

At D:\Automation\DeployVM\After_Install.ps1:265 char:13

+ Invoke-VMScript -VM $vm -GuestUser 'root' -GuestPassword 'Aa123456' ...

+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

+ CategoryInfo : NotSpecified: (:) [Invoke-VMScript], ViError

+ FullyQualifiedErrorId : Client20_VmGuestServiceImpl_DownloadFileFromGuest_DownloadError,VMware.VimAutomation.ViCore.Cmdlets.Commands.InvokeVmScript

i enabled vmtools logging on the vm, this is what i see right after the command arrives:

[May 20 10:35:31.597] [ debug] [vmsvc] RpcIn: received 152 bytes, content:"Vix_1_Relayed_Command "798bb0e89ee54ee8"\00\01\00\0d\d0\05\00o\00\00\003\00\00\00\1f\00\00\00\1d\00\00\00\0d\b5\00\00\00\00\01..."

[May 20 10:35:31.597] [ message] [vix] VixTools_ProcessVixCommand: command 181

[May 20 10:35:31.615] [ debug] [vix] VixToolsImpersonateUser: successfully impersonated user root

[May 20 10:35:31.615] [ debug] [vix] VixToolsCreateTempFile: User: root

[May 20 10:35:31.615] [ debug] [vix] VixToolsCreateTempFile: returning '/tmp/vmware-root/powerclivmware77'

[May 20 10:35:31.615] [ message] [vix] VixToolsCreateTempFile: opcode 181 returning 0

[May 20 10:35:31.615] [ debug] [vix] ToolsDaemonTcloReceiveVixCommand: command 181, additionalError = 0

[May 20 10:35:31.615] [ debug] [vmsvc] RpcIn: sending 41 bytes

[May 20 10:35:31.773] [ debug] [vmsvc] RpcIn: received 718 bytes, content:"Vix_1_Relayed_Command "798bb0e97d551e9c"\00\01\00\0d\d0\05\00\a5\02\00\003\00\00\00U\02\00\00\1d\00\00\00\0d\b9\00\00\00\00\00..."

[May 20 10:35:31.773] [ message] [vix] VixTools_ProcessVixCommand: command 185

[May 20 10:35:31.787] [ debug] [vix] VixToolsImpersonateUser: successfully impersonated user root

[May 20 10:35:31.787] [ debug] [vix] VixTools_StartProgram: User: root args: progamPath: '/bin/bash', arguments: '-c "bash > /tmp/vmware-root/powerclivmware77 2>&1 -c \"echo \\\"DEVICE=eth1\\\" >> /etc/sysconfig/network-script/ifcfg-eth1;echo \\\"BOOTPROTO=static \\\" >> /etc/sysconfig/network-script/ifcfg-eth1;echo \\\"NM_CONTROLLED=no\\\" >> /etc/sysconfig/network-script/ifcfg-eth1;echo \\\"PEERDNS=no\\\" >> /etc/sysconfig/network-script/ifcfg-eth1;echo \\\"ONBOOT=yes\\\" >> /etc/sysconfig/network-script/ifcfg-eth1;echo \\\"IPADDR=10.190.10.73\\\" >> /etc/sysconfig/network-script/ifcfg-eth1;echo \\\"NETMASK=10.190.10.0\\\" >> /etc/sysconfig/network-script/ifcfg-eth1\""', workingDir: ''

[May 20 10:35:31.788] [ debug] [vmsvc] Executing async command: '"/bin/bash" -c "bash > /tmp/vmware-root/powerclivmware77 2>&1 -c \"echo \\\"DEVICE=eth1\\\" >> /etc/sysconfig/network-script/ifcfg-eth1;echo \\\"BOOTPROTO=static \\\" >> /etc/sysconfig/network-script/ifcfg-eth1;echo \\\"NM_CONTROLLED=no\\\" >> /etc/sysconfig/network-script/ifcfg-eth1;echo \\\"PEERDNS=no\\\" >> /etc/sysconfig/network-script/ifcfg-eth1;echo \\\"ONBOOT=yes\\\" >> /etc/sysconfig/network-script/ifcfg-eth1;echo \\\"IPADDR=10.190.10.73\\\" >> /etc/sysconfig/network-script/ifcfg-eth1;echo \\\"NETMASK=10.190.10.0\\\" >> /etc/sysconfig/network-script/ifcfg-eth1\""' in working dir '/root'

[May 20 10:35:31.791] [ debug] [vix] VixToolsStartProgramImpl: started '"/bin/bash" -c "bash > /tmp/vmware-root/powerclivmware77 2>&1 -c \"echo \\\"DEVICE=eth1\\\" >> /etc/sysconfig/network-script/ifcfg-eth1;echo \\\"BOOTPROTO=static \\\" >> /etc/sysconfig/network-script/ifcfg-eth1;echo \\\"NM_CONTROLLED=no\\\" >> /etc/sysconfig/network-script/ifcfg-eth1;echo \\\"PEERDNS=no\\\" >> /etc/sysconfig/network-script/ifcfg-eth1;echo \\\"ONBOOT=yes\\\" >> /etc/sysconfig/network-script/ifcfg-eth1;echo \\\"IPADDR=10.190.10.73\\\" >> /etc/sysconfig/network-script/ifcfg-eth1;echo \\\"NETMASK=10.190.10.0\\\" >> /etc/sysconfig/network-script/ifcfg-eth1\""', pid 3707

[May 20 10:35:31.792] [ debug] [vix] VixTools_StartProgram: returning '3707'

[May 20 10:35:31.792] [ message] [vix] VixTools_StartProgram: opcode 185 returning 0

[May 20 10:35:31.792] [ debug] [vix] ToolsDaemonTcloReceiveVixCommand: command 185, additionalError = 0

[May 20 10:35:31.792] [ debug] [vmsvc] RpcIn: sending 12 bytes

[May 20 10:35:31.823] [ debug] [vmsvc] RpcIn: received 141 bytes, content:"Vix_1_Relayed_Command "798bb0eac74db7eb"\00\01\00\0d\d0\05\00d\00\00\003\00\00\00\14\00\00\00\1d\00\00\00\0d\ba\00\00\00\00\00..."

[May 20 10:35:31.824] [ message] [vix] VixTools_ProcessVixCommand: command 186

[May 20 10:35:31.836] [ debug] [vix] VixToolsImpersonateUser: successfully impersonated user root

[May 20 10:35:31.836] [ debug] [vix] VixToolsListProcessesEx: User: root

[May 20 10:35:31.836] [ debug] [vix] VixToolsListProcessesExGenerateData: found all 1 requested pids on the startedProcess list; finished

[May 20 10:35:31.836] [ message] [vix] VixToolsListProcessesEx: opcode 186 returning 0

[May 20 10:35:31.836] [ debug] [vix] ToolsDaemonTcloReceiveVixCommand: command 186, additionalError = 0

[May 20 10:35:31.836] [ debug] [vmsvc] RpcIn: sending 789 bytes

[May 20 10:35:32.792] [ debug] [vmsvc] Child w/ fd b exited with code=1

[May 20 10:35:32.792] [ debug] [vmsvc] Waiting on pid 3706 to de-zombify it

[May 20 10:35:35.193] [ debug] [vmsvc] RpcIn: received 0 bytes, empty TCLO poll

[May 20 10:35:36.896] [ debug] [vmsvc] RpcIn: received 141 bytes, content:"Vix_1_Relayed_Command "798bb0ebdd7c83e5"\00\01\00\0d\d0\05\00d\00\00\003\00\00\00\14\00\00\00\1d\00\00\00\0d\ba\00\00\00\00\00..."

[May 20 10:35:36.896] [ message] [vix] VixTools_ProcessVixCommand: command 186

[May 20 10:35:36.914] [ debug] [vix] VixToolsImpersonateUser: successfully impersonated user root

[May 20 10:35:36.914] [ debug] [vix] VixToolsListProcessesEx: User: root

[May 20 10:35:36.914] [ debug] [vix] VixToolsListProcessesExGenerateData: found all 1 requested pids on the startedProcess list; finished

[May 20 10:35:36.915] [ message] [vix] VixToolsListProcessesEx: opcode 186 returning 0

[May 20 10:35:36.915] [ debug] [vix] ToolsDaemonTcloReceiveVixCommand: command 186, additionalError = 0

[May 20 10:35:36.915] [ debug] [vmsvc] RpcIn: sending 798 bytes

[May 20 10:35:36.945] [ debug] [vmsvc] RpcIn: received 184 bytes, content:"Vix_1_Relayed_Command "798bb0ec7205f2c0"\00\01\00\0d\d0\05\00\8f\00\00\003\00\00\00?\00\00\00\1d\00\00\00\0d\bc\00\00\00\00\00..."

[May 20 10:35:36.945] [ message] [vix] VixTools_ProcessVixCommand: command 188

[May 20 10:35:36.958] [ debug] [vix] VixToolsImpersonateUser: successfully impersonated user root

[May 20 10:35:36.958] [ debug] [vix] VixToolsInitiateFileTransferFromGuest: User: root filePath: /tmp/vmware-root/powerclivmware77

[May 20 10:35:36.958] [ debug] [vix] VixToolsInitiateFileTransferFromGuest: returning '<fxi><Name>/tmp/vmware-root/powerclivmware77</Name><ft>0</ft><fs>518</fs><mt>1558337731</mt><at>1558337731</at><uid>0</uid><gid>0</gid><perm>33152</perm><slt></slt></fxi>'

[May 20 10:35:36.958] [ message] [vix] VixToolsInitiateFileTransferFromGuest: opcode 188 returning 0

[May 20 10:35:36.958] [ debug] [vix] ToolsDaemonTcloReceiveVixCommand: command 188, additionalError = 0

[May 20 10:35:36.958] [ debug] [vmsvc] RpcIn: sending 178 bytes

any help will be appreciated.

Thanks

Mor

LucD · ‎05-20-2019

Can you give some more information?
What PowerCLI version?
Which guest OS?
Can you share the script that fails?
Are the file(s) the script creates in the guest OS created? All of them?

Blog: lucd.info Twitter: @LucD22 Co-author PowerCLI Reference

pargit · ‎05-20-2019

Hi,

Powercli version is: VMware PowerCLI 11.1.0 build 11289667

Guest os is centos7.5

The part of the script:

$scripts_eng = @(

"echo `"DEVICE=eth1`" >> /etc/sysconfig/network-script/ifcfg-eth1;echo `"BOOTPROTO=static `" >> /etc/sysconfig/network-script/ifcfg-eth1;echo `"NM_CONTROLLED=no`" >> /etc/sysconfig/network-script/ifcfg-eth1;echo `"PEERDNS=no`" >> /etc/sysconfig/network-script/ifcfg-eth1;echo `"ONBOOT=yes`" >> /etc/sysconfig/network-script/ifcfg-eth1;echo `"IPADDR=$ipoob`" >> /etc/sysconfig/network-script/ifcfg-eth1;echo `"NETMASK=$networkoob`" >> /etc/sysconfig/network-script/ifcfg-eth1";

"ip route add 10.190.0.0/16 via $gwoob dev eth1";

"ip route add 10.200.0.0/14 via $gwoob dev eth1";

"ip route add 10.208.0.0/12 via $gwoob dev eth1";

"ip route add 10.110.0.0/16 via $gwoob dev eth1";

"echo -e `"10.190.0.0/16 via $gwoob dev eth1\n10.200.0.0/14 via $gwoob dev eth1\n10.208.0.0/12 via $gwoob dev eth1\n10.110.0.0/16 via $gwoob dev eth1`" > /etc/sysconfig/network-scripts/route-eth1";

)

foreach ($script in $scripts) {

Out-Log "`nLine $(__LINE__): Script = $script"

$vm -GuestUser 'root' -GuestPassword 'Aa123456' -ScriptText $script -ScriptType Bash -Confirm:$false

}

Ofcourse that there is connect-viserver, get-vm, etc…

As far as I see the files are not created in the guest os.

Thanks

Mor

LucD · ‎05-20-2019

I assume that there is an Invoke-VMScript missing on the 2nd to last line?

And the ForEach loop is probably over $scripts_eng instead of $scripts?

Since I have the impression from the debug log you showed earlier, that some lines seem to execute correctly, it would be interesting to know what is in the output produced by your Out-Log.

The last line in there should be the one that is causing the error I guess?

Blog: lucd.info Twitter: @LucD22 Co-author PowerCLI Reference

pargit · ‎05-20-2019

Yes, got lost on Paste

this is the full part...

the windows part is working ok.

if ($division -eq 1) {

$scripts = $scripts_eng

}

else {

$scripts = $scripts_mis

}

foreach ($script in $scripts) {

Out-Log "Line $(__LINE__): Script = $script"

Add-Content -Path "D:\Website\automation\$vmname.txt" -Value $script

if ($os = "Linux") {

Invoke-VMScript -VM $vm -GuestUser 'root' -GuestPassword 'Aa123456' -ScriptText $script -ScriptType Bash -Confirm:$false

}

else {

Invoke-VMScript -VM $vm -GuestCredential $localcred -ScriptText $script -ScriptType Bat -Confirm:$false

}

Start-Sleep -Seconds 2

}

Thanks

Mor

LucD · ‎05-20-2019

Thanks.
The Windows part are of course different statements.

What about this Out-Log line, does it show at which line you get the error?
It should show the line before the error message?

Blog: lucd.info Twitter: @LucD22 Co-author PowerCLI Reference

pargit · ‎05-20-2019

Hi,

the out-log function is something internal... it looks like this in the console:

05-20-2019 13:25:31 Line 263: Script = echo -e "10.190.0.0/16 via 10.190.10.1 dev eth1\n10.200.0.0/14 via 10.190.10.1 dev eth1\n10.208.0.0/12 via 10.190.10.1 dev eth1\n10.110.0.0/16 via 10.190.10.1 dev eth1" > /etc/sysconfig/network-scripts/route-eth1

Invoke-VMScript : 5/20/2019 1:26:41 PM Invoke-VMScript An error occurred while sending the request.

At D:\Automation\DeployVM\After_Install.ps1:266 char:13

+ Invoke-VMScript -VM $vm -GuestUser 'root' -GuestPassword 'Aa123456' ...

+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

+ CategoryInfo : NotSpecified: (:) [Invoke-VMScript], ViError

+ FullyQualifiedErrorId : Client20_VmGuestServiceImpl_DownloadFileFromGuest_DownloadError,VMware.VimAutomation.ViCore.Cmdlets.Commands.InvokeVmScript

you can see there is a gap of about 70 seconds between the log and the error. in this time there is a progress bar of the invoke-vmscript which stay on 0% till it fails.

thanks

mor

LucD · ‎05-20-2019

Is it always the same line where it fails?
Even if you rearrange the lines in the array?

Blog: lucd.info Twitter: @LucD22 Co-author PowerCLI Reference

pargit · ‎05-20-2019

it fails in all lines.

each one timed out after about a minute and gives the error.

LucD · ‎05-20-2019

Ok, can you check Re: Error running Invoke-Vmscript - An error occurred while sending the request. FullyQualifiedError...

The user was able to see what was going on when he used my Invoke-VMScriptPlus function.

Blog: lucd.info Twitter: @LucD22 Co-author PowerCLI Reference

pargit · ‎05-21-2019

Success..

for some reason when i tried yesterday to use your script i had class+transform errors, but today after reboot to my pc it works...

Thanks for the help... now i have to integrate it with my script...

pargit · ‎05-21-2019

so errors again, on my production vm,

At D:\Automation\DeployVM\Invoke-VMScriptPlus.ps1:1 char:1

+ class MyOBN:System.Management.Automation.ArgumentTransformationAttribute {

+ ~~~~~

The 'class' keyword is not supported in this version of the language.

At D:\Automation\DeployVM\Invoke-VMScriptPlus.ps1:10 char:11

+ [object] Transform([System.Management.Automation.EngineIntrinsics]$engineIntrin ...

+ ~~~~~~~~~

Unexpected token 'Transform' in expression or statement.

+ CategoryInfo : ParserError: (:) [], ParseException

+ FullyQualifiedErrorId : ReservedKeywordNotAllowed

this is what i had yesterday untill i rebooted. here it's not helping.

what is wrong?

LucD · ‎05-21-2019

That seems to indicate that you are using a rather old PowerShell version.

What does $PSVersionTable return?

Blog: lucd.info Twitter: @LucD22 Co-author PowerCLI Reference

pargit · ‎05-21-2019

it's windows 2012r2

PS M:\> $PSVersionTable

Name Value

---- -----

PSVersion 4.0

WSManStackVersion 3.0

SerializationVersion 1.1.0.1

CLRVersion 4.0.30319.42000

BuildVersion 6.3.9600.19170

PSCompatibleVersions {1.0, 2.0, 3.0, 4.0}

PSRemotingProtocolVersion 2.2

LucD · ‎05-21-2019

I'm afraid that the function requires PS v5.
Can you upgrade?

Blog: lucd.info Twitter: @LucD22 Co-author PowerCLI Reference

pargit · ‎05-21-2019

yes. after upgrade i can run it.

now i have some networking issues (i replaced the resolve-dns with get-vm -name $vminstance |select vmhost beucase of dns issues.)

now i'm waiting for fw rules from the source server to the esx...

i'll update but it looks like going to the right direction

thanks again.

All

Invoke-VMscript on linux machine fails - An error occurred while sending the request.