11 Replies Latest reply on Nov 20, 2019 12:09 PM by LucD

    Response status code does not indicate success (500 internal server error)

    jack_ne Lurker

      The script got executed and in the the destination VM the test1 directory got created but the script failed the below 500 error code.

      Can anyone please help, why we are getting the Response status code does not indicate success: 500 (Internal Server Error) even after the directory creation.

      Either in the vmware log are vmware tool log, could not see any error detail message. PowerCLI 11.5.0 is being instal

       

      Invoke-VMScript -ScriptText "mkdir c:\test1\" -VM 'TestMachine' -GuestUser 'Test' -GuestPassword 'Test@123' -Verbose

       

      VERBOSE: Performing the operation "Invoke-VMScript" on target "TestMachine".

      VERBOSE: 11/19/2019 3:32:25 PM Invoke-VMScript Finished execution

      Invoke-VMScript : 11/19/2019 3:32:32 PM Invoke-VMScript         Response status code does not indicate success: 500 (Internal Server Error).

      At line:1 char:1

      + Invoke-VMScript -ScriptText "mkdir c:\test1\" -VM 'TestMach ...

      + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

          + CategoryInfo          : NotSpecified: (:) [Invoke-VMScript], ViError

          + FullyQualifiedErrorId : Client20_VmGuestServiceImpl_DownloadFileFromGuest_DownloadError,VMware.VimAutomation.ViCore.Cmdlets.Commands.InvokeVmScript

        • 1. Re: Response status code does not indicate success (500 internal server error)
          LucD Guru
          vExpertCommunity WarriorsUser Moderators

          What vSphere are you running? The vCenter and the ESXi node on which the VM is running?

           

          Is port 902 to the ESXi node on which the VM is running open?


          The vmware.log file in the VM's folder should show you the execution sequence.

          It would be useful to see if all steps are executed or not.

          • 2. Re: Response status code does not indicate success (500 internal server error)
            jack_ne Lurker

            vCenter is 6.7

            ESXI is 6.0

            Guest VM running with windows 10, vmware tools 10309

             

             

            Regd Port 902 connection status:

            Below are the status from the windows powershell system where i invoke the below command:

            C:\Users\xxxx>curl vcenter.com:902

            curl: (7) Failed to connect to xxxxxx port 902: Connection refused

             

             

            C:\Users\xxxx>curl esxi.com:902

            220 VMware Authentication Daemon Version 1.10: SSL Required, ServerDaemonProtocol:SOAP, MKSDisplayProtocol:VNC , VMXARGS supported, NFCSSL supported/t

             

             

            vmware log:

            2019-11-20T14:57:59.761Z| vmx| I125: VigorTransportProcessClientPayload: opID=6657c189-82-d67e seq=42212: Receiving GuestOps.CreateTemporaryFile request.

            2019-11-20T14:57:59.840Z| vcpu-0| I125: VigorTransport_ServerSendResponse opID=6657c189-82-d67e seq=42212: Completed GuestOps request.

            2019-11-20T14:57:59.911Z| vmx| I125: VigorTransportProcessClientPayload: opID=b633ddd-e6-d682 seq=42226: Receiving GuestOps.StartProgram request.

            2019-11-20T14:57:59.947Z| vcpu-2| I125: VigorTransport_ServerSendResponse opID=b633ddd-e6-d682 seq=42226: Completed GuestOps request.

            2019-11-20T14:58:00.014Z| vmx| I125: VigorTransportProcessClientPayload: opID=7645b1fd-7d-d689 seq=42240: Receiving GuestOps.ListProcesses request.

            2019-11-20T14:58:00.037Z| vcpu-1| I125: VigorTransport_ServerSendResponse opID=7645b1fd-7d-d689 seq=42240: Completed GuestOps request.

            2019-11-20T14:58:05.055Z| vmx| I125: VigorTransportProcessClientPayload: opID=73036410-3f-d696 seq=42255: Receiving GuestOps.ListProcesses request.

            2019-11-20T14:58:05.134Z| vcpu-1| I125: VigorTransport_ServerSendResponse opID=73036410-3f-d696 seq=42255: Completed GuestOps request.

            2019-11-20T14:58:05.202Z| vmx| I125: VigorTransportProcessClientPayload: opID=237226c5-7a-d69a seq=42269: Receiving GuestOps.InitiateFileTransferFromGuest request.

            2019-11-20T14:58:05.229Z| vcpu-1| I125: VigorTransport_ServerSendResponse opID=237226c5-7a-d69a seq=42269: Completed GuestOps request.

            2019-11-20T14:58:05.455Z| vmx| I125: VigorTransportProcessClientPayload: opID=1b55e00d-dd-d6a8 seq=42285: Receiving GuestOps.InitiateFileTransferFromGuest request.

            2019-11-20T14:58:05.523Z| vcpu-0| I125: VigorTransport_ServerSendResponse opID=1b55e00d-dd-d6a8 seq=42285: Completed GuestOps request.

            2019-11-20T14:58:05.639Z| vmx| I125: VigorTransportProcessClientPayload: opID=29d24eb1-fe-d6af seq=42301: Receiving GuestOps.InitiateFileTransferFromGuest request.

            2019-11-20T14:58:05.710Z| vcpu-0| I125: VigorTransport_ServerSendResponse opID=29d24eb1-fe-d6af seq=42301: Completed GuestOps request.

            2019-11-20T14:58:05.831Z| vmx| I125: VigorTransportProcessClientPayload: opID=884a80f-13-d6b6 seq=42317: Receiving GuestOps.DeleteFile request.

            2019-11-20T14:58:05.897Z| vcpu-0| I125: VigorTransport_ServerSendResponse opID=884a80f-13-d6b6 seq=42317: Completed GuestOperations request.

             

            Note: the script created the dir in the guest VM but the response for the script error is producing the error. so all the invoking script is successfully getting executed but the response output it receives the 500 error.  -> Response status code does not indicate success (500 internal server error)

            • 3. Re: Response status code does not indicate success (500 internal server error)
              LucD Guru
              Community WarriorsUser ModeratorsvExpert

              Can you, on the system where you run the Invoke-VMScript cmdlet, disable all FW rules temporarily?

              Just to check if that allows the result to be send back.

              • 4. Re: Response status code does not indicate success (500 internal server error)
                jack_ne Lurker

                Thank you for the quick response Lucd,

                Earlier only, I have turned off the firewall rules in both source system and the destination guest windows VM. That did not help. It is the same result.

                • 5. Re: Response status code does not indicate success (500 internal server error)
                  LucD Guru
                  vExpertUser ModeratorsCommunity Warriors

                  Do the certificates on your ESXi nodes only contain the FQDN of the ESXi node?
                  Or do they also contain the IP address of the ESXi node (as a SAN)?

                   

                  You could try to run the same with my Invoke-VMScriptPlus function with the Verbose option.

                  That function has a few more options and more elaborate verbosity.

                  • 6. Re: Response status code does not indicate success (500 internal server error)
                    jack_ne Lurker

                    While executing the Invoke-VMScriptPlus, following are the output sen.

                    In the destination guest vm,under the folder path  C:\Users\xxxxxxs\AppData\Local\Temp\xxxxxx_41724vmware112 it created files with 0KB size and the script failed with the

                    Could not establish trust relationship for the SSL/TLS secure channel. and -1 byte payload.

                     

                    Script execution details blow:

                    PS C:\Users\xxxxxx> Invoke-VMScriptPlus -ScriptText "mkdir c:\xxxxxx-CI\tst-03\" -VM 'GuestVM' -GuestUser 'xxxxxx'  -Verbose

                     

                    cmdlet Invoke-VMScriptPlus at command pipeline position 1

                    Supply values for the following parameters:

                    GuestPassword: *********

                    ScriptType: powershell

                    VERBOSE: 11/20/2019 12:01:21 PM Get-View Finished execution

                    VERBOSE: 11/20/2019 12:01:21 PM Get-View Finished execution

                    VERBOSE: 11/20/2019 12:01:21 PM Get-View Finished execution

                    VERBOSE: 11/20/2019 12:01:21 PM Get-View Finished execution

                    VERBOSE: No GuestOSType value provided. Trying to determine now.

                    VERBOSE: It's a Windows guest OS

                    VERBOSE: Created temp folder in guest OS C:\Users\xxxxxxs\AppData\Local\Temp\xxxxxx_41724vmware112

                    VERBOSE: Created temp script file in guest OS C:\Users\xxxxxxs\AppData\Local\Temp\xxxxxx_41724vmware112\xxxxxx_41724vmware233.ps1

                    VERBOSE: Created temp output file in guest OS C:\Users\xxxxxxs\AppData\Local\Temp\xxxxxx_41724vmware112\xxxxxx_41724_outputvmware55

                    VERBOSE: Copying Data to C:\Users\xxxxxxs\AppData\Local\Temp\xxxxxx_41724vmware112\xxxxxx_41724vmware233.ps1

                    VERBOSE: PUT https://x.x.x.194/guestFile?id=4936&token=5202d06f-5b24-9018-c1f4-9c17e8e1bb474936 with -1-byte payload

                    Invoke-WebRequest : The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel.

                    At line:179 char:27

                    + ... copyResult = Invoke-WebRequest -Uri $fileInfo -Method Put -Body $Data

                    +                  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

                        + CategoryInfo          : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-WebRequest], WebException

                        + FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeWebRequestCommand

                     

                     

                    ScripText copy failed!Status

                    At line:182 char:17

                    + ...             Throw "ScripText copy failed!`rStatus $($copyResult.Statu ...

                    +                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

                        + CategoryInfo          : OperationStopped: (ScripText copy failed!Status  :String) [], RuntimeException

                        + FullyQualifiedErrorId : ScripText copy failed!Status

                    • 8. Re: Response status code does not indicate success (500 internal server error)
                      jack_ne Lurker

                      -NoIPinCert throws

                      VERBOSE: x.x.x.10.in-addr.arpa

                      You cannot call a method on a null-valued expression. error

                       

                       

                      PS C:\Users\xxxxx> Invoke-VMScriptPlus -NoIPinCert -ScriptText "mkdir c:\xxxx-CI\plus\" -VM 'GuestVM' -GuestUser 'Guestuser' -Verbose

                       

                      cmdlet Invoke-VMScriptPlus at command pipeline position 1

                      Supply values for the following parameters:

                      GuestPassword: *********

                      ScriptType: powershell

                      VERBOSE: 11/20/2019 2:00:17 PM Get-View Finished execution

                      VERBOSE: 11/20/2019 2:00:17 PM Get-View Finished execution

                      VERBOSE: 11/20/2019 2:00:17 PM Get-View Finished execution

                      VERBOSE: 11/20/2019 2:00:17 PM Get-View Finished execution

                      VERBOSE: No GuestOSType value provided. Trying to determine now.

                      VERBOSE: It's a Windows guest OS

                      VERBOSE: Created temp folder in guest OS C:\Users\xxxxxx\AppData\Local\Temp\xxxxxx_39772vmware48

                      VERBOSE: Created temp script file in guest OS C:\Users\xxxxxx\AppData\Local\Temp\xxxxxx_39772vmware48\xxxxxx_39772vmware97.ps1

                      VERBOSE: Created temp output file in guest OS C:\Users\xxxxxx\AppData\Local\Temp\xxxxxx_39772vmware48\xxxxxx_39772_outputvmware217

                      VERBOSE: x.x.x.10.in-addr.arpa

                      You cannot call a method on a null-valued expression.

                      At line:176 char:17

                      +                 $fileInfo.Url = $fileInfo.Url.replace($ip, $hostName)

                      +                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

                          + CategoryInfo          : InvalidOperation: (:) [], RuntimeException

                          + FullyQualifiedErrorId : InvokeMethodOnNull

                       

                       

                      VERBOSE: Copying Data to C:\Users\xxxxxx\AppData\Local\Temp\xxxxxx_39772vmware48\xxxxxx_39772vmware97.ps1

                      VERBOSE: PUT https://x.x.x.192/guestFile?id=4921&token=52eb13d0-5f05-1216-8644-bdf6aa1416f04921 with -1-byte payload

                      Invoke-WebRequest : The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel.

                      At line:179 char:27

                      + ... copyResult = Invoke-WebRequest -Uri $fileInfo -Method Put -Body $Data

                      +                  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

                          + CategoryInfo          : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-WebRequest], WebException

                          + FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeWebRequestCommand

                       

                       

                      ScripText copy failed!Status

                      At line:182 char:17

                      + ...             Throw "ScripText copy failed!`rStatus $($copyResult.Statu ...

                      +                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

                          + CategoryInfo          : OperationStopped: (ScripText copy failed!Status  :String) [], RuntimeException

                          + FullyQualifiedErrorId : ScripText copy failed!Status

                      • 9. Re: Response status code does not indicate success (500 internal server error)
                        LucD Guru
                        Community WarriorsvExpertUser Moderators

                        Are your ESXi nodes registered in the VCSA with their IP address and not their FQDN?
                        What kind of certificates are you using?
                        Self-signed?
                        Do they contain the IP address of the ESXi node

                         

                        How did you configure your PowerCLI options?
                        Is InvalidCertificateAction set to Ignore?
                        You can check with Get-PowerCLIConfiguration.

                        • 10. Re: Response status code does not indicate success (500 internal server error)
                          jack_ne Lurker

                          Hi LucD,

                           

                          The ESXi nodes are registered with FQDN in the VCSA. In our vCenter GUI, all the ESXI hosts are displayed with the names eg: xxxxx.xxx.xxx.com

                          The certificates are CA certified

                          There is no IP address of the ESXi in the vCenter.

                           

                          Yes PowerCLI Configuration options has been set to ignore.

                          Set-PowerCLIConfiguration -InvalidCertificateAction Ignore -Confirm:$false

                          • 11. Re: Response status code does not indicate success (500 internal server error)
                            LucD Guru
                            vExpertCommunity WarriorsUser Moderators

                            Then I don't understand why the following line is apparently using an IP address instead of a FQDN with the NoIPinCert switch.

                             

                            VERBOSE: PUT https://x.x.x.192/guestFile?id=4921&token=52eb13d0-5f05-1216-8644-bdf6aa1416f04921 with -1-byte payload