I suspect that you're getting caught up in some Try/Catch stuff. For cmdlets its important that the error action is "stop" when you're using Try/Catch, and that's not the default for all cmdlets (you can try throwing an "-ErrorAction Stop" on all of the cmdlet calls).
Also the common coding pattern for controllable errors is to use -ErrorAction and -ErrorVariable to handle things gracefully rather than Try/Catch (try/catch also makes it hard to clean up after yourself (i.e. if "connect-viserver" fails then you don't need to do a "disconnect-viserver" in the catch, however if you ended up in the catch because Get-VM failed, then you should do a disconnect-viserver)). So as an alternative
Connect-VIServer $vcenter -User $vcenter\$invuser -Password $invpass -ErrorAction SilentlyContinue -ErrorVariable Err
If ($Err -ne $Null) {
# Write logs to say connect failed or do whatever else you want to do. Since this is granular you can write something specific about the error in the log rather than just "Exception"
}
Else {
# continue on with the rest of your processing. If you're worried about Get-VM or other cmdlets failing, then use the same coding pattern to check for errors
}