pamiller21
Enthusiast
Enthusiast

Scripts Failing Suddenly

Jump to solution

I have had a number of scripts fail recently, I run the script manually and here is the script and output:

#############################

# Connect to vCenter        #

#############################

Start-Transcript -Path 'C:\Temp\script.log'

Import-Module -Name VMware.PowerCLI

Set-PowerCLIConfiguration -DisplayDeprecationWarnings $false -InvalidCertificateAction ignore -confirm:$false

$vc = 'DNS'

$Cred = Import-Clixml C:\util\Scripts\creds\autoitjob.xml

Connect-VIServer $VC -Credential $Cred

#############################

#         Variables         #

#############################

$date=Get-Date -format "yyyy-MMM-d"

$datetime=Get-Date

$filelocation="C:\util\Scripts\Temp\Thin-$date.htm"

#############################

#          Content          #

#############################

$report = Get-VM | Get-HardDisk | where {$_.StorageFormat -ne "Thin"} | select Parent,StorageFormat,Filename

#$report = Get-VM | Get-HardDisk |

#where {$_.StorageFormat -ne "Thin"} |

#select Parent,StorageFormat,Filename

#############################

# Add Text to the HTML file #

#############################

$report | ConvertTo-Html -title "VMware Thin Provision Check" -body "<H1>Vmware Thin Provision Check</H1>" -head "<link rel='stylesheet' href='style.css' type='text/css' />" | Out-File $filelocation

ConvertTo-Html -title "VMware Thin Provision Check" -body "<H4>Date and time </H4>",$datetime -head "<link rel='stylesheet' href='style.css' type='text/css' />" | Out-File -Append $filelocation

ConvertTo-Html -title "VMware Thin Provision Check" -body "<H4>VM Count</H4>",$report.Count -head "<link rel='stylesheet' href='style.css' type='text/css' />" | Out-File -Append $filelocation

######################

# FTP HTML output #

#####################

#we specify the directory where all files that we want to upload 

$Dir="$filelocation"

#Below for test

#$Dir="C:/Users/administrator/Desktop/tmp/"   

#ftp server

$ftp = "ftp://IP/internal-backups/datacenter/vcenter/misc-reports/Thin/"

$user = "USER"

$pass = "PW" 

$webclient = New-Object System.Net.WebClient

$webclient.Credentials = New-Object System.Net.NetworkCredential($user,$pass) 

#list every sql server trace file

foreach($item in (dir $Dir "*.bak")){

    "Uploading $item..."

    $uri = New-Object System.Uri($ftp+$item.Name)

    $webclient.UploadFile($uri, $item.FullName)

}

##############################

# Disconnect session from VC #

##############################

disconnect-viserver -confirm:$false

OUTPUT:

PS C:\util\scripts\Reports\Daily> .\ThinReport.ps1

Transcript started, output file is C:\Temp\script.log

Scope    ProxyPolicy     DefaultVIServerMode InvalidCertificateAction  DisplayDeprecationWarnings WebOperationTimeout

                                                                                                  Seconds

-----    -----------     ------------------- ------------------------  -------------------------- -------------------

Session  UseSystemProxy  Multiple            Ignore                    False                      300

User                     Multiple            Ignore                    False

AllUsers

IsConnected   : True

Id            : /VIServer=vsphere.local\powercli@DNS:443/

ServiceUri    : https://vcenter6.ruralnex.com/sdk

SessionSecret : "7f2930a91d58a037dd232de769e26727c3338c71"

Name          : vcenter6.ruralnex.com

Port          : 443

SessionId     : "7f2930a91d58a037dd232de769e26727c3338c71"

User          : VSPHERE.LOCAL\PowerCLI

Uid           : /VIServer=vsphere.local\powercli@vcenter6.ruralnex.com:443/

Version       : 6.5.0

Build         : 8667236

ProductLine   : vpx

InstanceUuid  : e66b929f-6e6e-4791-aafc-621177fc31b1

RefCount      : 1

ExtensionData : VMware.Vim.ServiceInstance

Get-VM : 7/9/2018 11:01:54 AM   Get-VM          Value cannot be null.

Parameter name: array

At C:\util\scripts\Reports\Daily\ThinReport.ps1:22 char:11

+ $report = Get-VM | Get-HardDisk | where {$_.StorageFormat -ne "Thin"} ...

+           ~~~~~~

    + CategoryInfo          : NotSpecified: (:) [Get-VM], VimException

    + FullyQualifiedErrorId : Core_BaseCmdlet_UnknownError,VMware.VimAutomation.ViCore.Cmdlets.Commands.GetVM

Uploading C:\util\Scripts\Temp\Thin-2018-Jul-9.htm...

0 Kudos
50 Replies
LucD
Leadership
Leadership

Can you just start a PowerShell prompt, and do the Connect-VIServer.

Then from the prompt, just do a Get-VM.

Does that generate the same error as you are seeing in your script?

Do all cmdlets (for exmaple Get-VMHost, Get-Datastore, Get-Cluster...) come back with the same error?


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

0 Kudos
pamiller21
Enthusiast
Enthusiast

Alright did just that and everything worked, BUT get-vm....same error

0 Kudos
LucD
Leadership
Leadership

Do you by any chance have a Hyper-V module installed?

Can you do

Get-Command -Name Get-VM


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

0 Kudos
pamiller21
Enthusiast
Enthusiast

PS C:\> Get-Command -Name Get-VM

CommandType     Name                                               Version    Source

-----------     ----                                               -------    ------

Cmdlet          Get-VM                                             10.1.0.... VMware.VimAutomation.Core

PS C:\> get-vm

get-vm : 7/13/2018 3:12:37 PM   Get-VM          Value cannot be null.

Parameter name: array

At line:1 char:1

+ get-vm

+ ~~~~~~

    + CategoryInfo          : NotSpecified: (:) [Get-VM], VimException

    + FullyQualifiedErrorId : Core_BaseCmdlet_UnknownError,VMware.VimAutomation.ViCore.Cmdlets.Commands.GetVM

0 Kudos
LucD
Leadership
Leadership

Clutching at straws now :smileygrin:

Do you have anything in $PSDefaultParameterValues


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

0 Kudos
LucD
Leadership
Leadership

And while you're at it, can you also try

Get-VM -ErrorAction SilentlyContinue


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

0 Kudos
pamiller21
Enthusiast
Enthusiast

I don't believe there is anything in the default configs, and that command comes back with nothing, no error no data.

0 Kudos
LucD
Leadership
Leadership

This is becoming a real mystery :smileygrin:

Can you check in the vpxd logs if there are any entries created at the time you get the error?


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

0 Kudos
pamiller21
Enthusiast
Enthusiast

I don't see anything in the logs

0 Kudos
mkfm
Enthusiast
Enthusiast

Hi

just had the same issue, strangely just on 1 of 4 vCenters, the other VCs correctly returned all VMs.

After some testing i found, that a running VM clone task was interrupting the command. After the task is completed, everything works as expected.

Just did a clone in our Lab-vCenter and same error occurs while the task is running.

So check your running tasks

0 Kudos
pamiller21
Enthusiast
Enthusiast

Don't think it is that, I am running them at two different times at night. Here is another thing that doesn't make sense, I have about half my scripts not working and half working just fine. Here is my script that discovers if VMTools are out of date and it works fine:

#############################

#    Connect to vCenter     #

#############################

Import-Module -Name VMware.PowerCLI

Set-PowerCLIConfiguration -DisplayDeprecationWarnings $false -InvalidCertificateAction ignore -confirm:$false

$vc = 'IP'

$Cred = Import-Clixml C:\util\Scripts\creds\autoitjob2.xml

Connect-VIServer $VC -Credential $Cred

#############################

#          Content          #

#############################

$date=Get-Date -format "yyyy-MMM-d"

$datetime=Get-Date

$filelocation="c:\util\Scripts\Temp\tools-$date.htm"

$report = Get-View -ViewType VirtualMachine -Property Config.Template,Name,Guest.ToolsStatus -Filter @{"Config.Template"="False";"Guest.ToolsStatus"="^(?!toolsOk$).*$" ;"Name"="^((?!_replica).)*$"; "Runtime.PowerState"="poweredOn"} |

Select-Object Name,@{Name="ToolsStatus";E={$_.Guest.ToolsStatus}}

#############################

# Add Text to the HTML file #

#############################

$report | ConvertTo-Html –title "VMware VMTools Check" –body "<H1>VMware VMTools Check</H1>" -head "<link rel='stylesheet' href='style.css' type='text/css' />" | Out-File $filelocation

ConvertTo-Html –title "VMware VMTools Check" –body "<H4>Date and time</H4>",$datetime -head "<link rel='stylesheet' href='style.css' type='text/css' />" | Out-File -Append $filelocation

ConvertTo-Html –title "VMware VMTools Check" –body "<H4>VM Count</H4>",$report.Count -head "<link rel='stylesheet' href='style.css' type='text/css' />" | Out-File -Append $filelocation

######################

# FTP HTML output #

#####################

#we specify the directory where all files that we want to upload 

$Dir="$filelocation"

#Below for test

#$Dir="C:/Users/administrator/Desktop/tmp/"   

#ftp server

$ftp = "ftp://10.10.25.240/internal-backups/datacenter/vcenter/misc-reports/Tools/"

$user = "IP"

$pass = "PW" 

$webclient = New-Object System.Net.WebClient

$webclient.Credentials = New-Object System.Net.NetworkCredential($user,$pass)

#list every sql server trace file

foreach($item in (dir $Dir "*.bak")){

    "Uploading $item..."

    $uri = New-Object System.Uri($ftp+$item.Name)

    $webclient.UploadFile($uri, $item.FullName)

}

##############################

# Disconnect session from VC #

##############################

disconnect-viserver -confirm:$false

0 Kudos
LucD
Leadership
Leadership

Are it only the scripts that contain a Get-VM that fail?

And all with that same message?


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

0 Kudos
pamiller21
Enthusiast
Enthusiast

Seems that is correct

0 Kudos
LucD
Leadership
Leadership

Can you check if the following also gets the error?

Get-VM -Name *

And what about this one?

Get-View -ViewType VirtualMachine


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

0 Kudos
LucD
Leadership
Leadership

Btw, I found a rather old mention of exactly the same issue.

See Strange PowerCLI Problem with Get-VM

Unfortunately there is no solution present in the thread.

The initial analysis in that thread is quite interesting, the user tried to link the issue to a specific VM or a specific VMHost.

But it was not clear which of the 2 was actually causing the issue.


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

0 Kudos
nilly
Contributor
Contributor

Chiming in with what info I have... I am having the same issue.

Environment 1: Windows 2012 STD server

$PSVersionTable

Name                           Value

----                           -----

PSVersion                      5.1.14409.1012

PSEdition                      Desktop

PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}

BuildVersion                   10.0.14409.1012

CLRVersion                     4.0.30319.42000

WSManStackVersion              3.0

PSRemotingProtocolVersion      2.3

SerializationVersion           1.1.0.1

Get-PowerCLIVersion

PowerCLI Version

----------------

   VMware PowerCLI 6.5.1 build 5377412

---------------

Component Versions

---------------

   VMware Cis Core PowerCLI Component 6.5 build 6230110

   VMware VimAutomation Core PowerCLI Component 6.5 build 6234650

Environment 2: Windows 2012 R2 STD

$PSVersionTable

Name                           Value

----                           -----

PSVersion                      5.1.14409.1012

PSEdition                      Desktop

PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}

BuildVersion                   10.0.14409.1012

CLRVersion                     4.0.30319.42000

WSManStackVersion              3.0

PSRemotingProtocolVersion      2.3

SerializationVersion           1.1.0.1

Get-PowerCLIVersion

PowerCLI Version

----------------

   VMware PowerCLI 10.1.0 build 8346946

---------------

Component Versions

---------------

   VMware Cis Core PowerCLI Component PowerCLI Component 10.1 build 8377811

   VMware VimAutomation VICore Commands PowerCLI Component PowerCLI Component 10.1 build 8344055

----

- Environment 1 works just fine, across 3 different vcenters (older version of powercli obviously).

- Environment 2 works on 2 out of the 3 vcenters I have (newer version of powercli).

- Curiously, installing the older version of powercli (6.5.1, same as on environment 1) yielded the same results on Environment 2.

I was wondering about a corrupt powercli installation as well because the version number doesn't match with what shows on the PSGallery (10.1.1.8827524), even with using -AllowPrelease or -RequiredVersion. I've reinstalled probably about 10 times now.

0 Kudos
LucD
Leadership
Leadership

The version on the PowerShell Gallery is indeed the latest.

The difference between 10.1.0.8403314 and 10.1.1.8827524, is a newer version of the HorizonView module.

See this list from a station that has both releases.

PS C:\> Get-Module -Name VMware* -ListAvailable | Select Name,Version

Name                                Version      

----                                -------      

VMware.Hv.Helper                    1.1          

VMware.DeployAutomation             6.7.0.8250345

VMware.ImageBuilder                 6.7.0.8250345

VMware.PowerCLI                     10.1.1.8827524

VMware.PowerCLI                     10.1.0.8403314

VMware.Vim                          6.7.0.8343295

VMware.VimAutomation.Cis.Core       10.1.0.8377811

VMware.VimAutomation.Cloud          10.0.0.7893901

VMware.VimAutomation.Common         10.1.0.8342134

VMware.VimAutomation.Core           10.1.0.8344055

VMware.VimAutomation.HA             6.5.4.7567193

VMware.VimAutomation.HorizonView    7.5.0.8827468

VMware.VimAutomation.HorizonView    7.1.0.7547311

VMware.VimAutomation.License        10.0.0.7893904

VMware.VimAutomation.Nsxt           10.1.0.8346947

VMware.VimAutomation.PCloud         10.0.0.7893924

VMware.VimAutomation.Sdk            10.1.0.8342078

VMware.VimAutomation.Srm            10.0.0.7893900

VMware.VimAutomation.Storage        10.1.0.8313015

VMware.VimAutomation.StorageUtility 1.2.0.0      

VMware.VimAutomation.Vds            10.1.0.8344219

VMware.VimAutomation.Vmc            10.0.0.7893902

VMware.VimAutomation.vROps          10.0.0.7893921

VMware.VumAutomation                6.5.1.7862888

Try doing an Update-Module -Name VMware.PowerCLI to get the latest.

Since you have 2 platforms that behave differently, it might be interesting to check which .Net versions you have on both.
Can you run the following to determine the versions?

Code comes from PowerShell script to return versions of .NET Framework on a machine? - Stack Overflow

$Lookup = @{

   378389 = [version]'4.5'

   378675 = [version]'4.5.1'

   378758 = [version]'4.5.1'

   379893 = [version]'4.5.2'

   393295 = [version]'4.6'

   393297 = [version]'4.6'

   394254 = [version]'4.6.1'

   394271 = [version]'4.6.1'

   394802 = [version]'4.6.2'

   394806 = [version]'4.6.2'

   460798 = [version]'4.7'

   460805 = [version]'4.7'

   461308 = [version]'4.7.1'

   461310 = [version]'4.7.1'

   461808 = [version]'4.7.2'

   461814 = [version]'4.7.2'

}


Get-ChildItem 'HKLM:\SOFTWARE\Microsoft\NET Framework Setup\NDP' -Recurse |

   Get-ItemProperty -name Version, Release -EA 0 |

   # For One True framework (latest .NET 4x), change match to PSChildName -eq "Full":

Where-Object { $_.PSChildName -match '^(?!S)\p{L}'} |

   Select-Object @{name = ".NET Framework"; expression = {$_.PSChildName}},

@{name = "Product"; expression = {$Lookup[$_.Release]}},

Version, Release


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

0 Kudos
pamiller21
Enthusiast
Enthusiast

Ok here's the results:

Worked = Get-VM -Name *

Worked = Get-View -ViewType VirtualMachine

Failed = Get-VM | where { $_.Version -notmatch "v13" } | Select Name, Version

I also ran the Update-Module -Name VMware.PowerCLI, went through the process and failed

0 Kudos
LucD
Leadership
Leadership

Did you check the .Net versions?

Are there perhaps any .Net related updates pending?


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

0 Kudos
pamiller21
Enthusiast
Enthusiast

Sorry forgot that, here is the output:

PSChildName Version   Release

----------- -------   -------

Client      4.7.02053  460805

Full        4.7.02053  460805

Client      4.0.0.0

0 Kudos