The esxcli command you can do via the Get-EsxCli cmdlet, for the mv and python commands you will need to use SSH I'm afraid.
With the Posh-SSH module you can do that from your script.

See for example Use Posh-SSH instead of PuTTY

Thanks, Ill check out Posh-SSH

Here is what I came up with, I have not used it yet..but I think it will work

Connect-VIServer 'XXXXXXXXXXXXX'

$hosts = Get-Cluster "XXXXXXXXXX" | Get-VMHost
$cred = Get-Credential
$cmd = "mv /vmfs/volumes/INF03-PMAX-DS01/ucs-tool-esxi_1.2.1-1OEM.zip /tmp"
$cmd1 = "python /opt/ucs_tool_esxi/ucs_host_inventory.py"
    
foreach ($host in $hosts) {

    Get-VMHost $host | Get-VMHostService | Where-Object { $_.Key -eq "TSM-SSH" } | Start-VMHostService
    $session = New-SSHSession -ComputerName $host -Credential $cred -AcceptKey
    Invoke-SSHCommand -SSHSession $session -Command $cmd
    Get-esxcli software component apply -d /tmp/ucs-tool-esxi_1.2.1-1OEM.zip
    Invoke-SSHCommand -SSHSession $session -Command $cmd1 
    
    Remove-SSHSession -SSHSession $session | Out-Null 

}

Any thoughts on that piece of work lol?

I'm still trying to get this working correctly. Here is my latest, and the error I am getting:

Connect-VIServer 'XXXXXXXXXXXXX'

$hosts = Get-Cluster -name "XXXXXXXXXXX" | Get-VMHost
$cred = Get-Credential
$cmd = "mv /vmfs/volumes/INF03-PMAX-NonProd-DS01/ucs-tool-esxi_1.2.1-1OEM.zip /tmp"
$cmd1 = "python /opt/ucs_tool_esxi/ucs_host_inventory.py"
$arguments = @{
    'VIBURL' = '/tmp/ucs-tool-esxi_1.2.1-1OEM.zip'
}
    
foreach ($esx in $hosts) {

    # Get-VMHost $esx | Get-VMHostService | Where-Object { $_.Key -eq "TSM-SSH" } | Start-VMHostService
    $session = New-SSHSession -ComputerName $esx -Credential $cred -AcceptKey
    Invoke-SSHCommand -SSHSession $session -Command $cmd

    $esxcli = Get-EsxCli -VMHost $esx
    $esxcli.software.component.apply.Invoke($arguments)

    Invoke-SSHCommand -SSHSession $session -Command $cmd1 
    Remove-SSHSession -SSHSession $session | Out-Null 

}

The copy seems to work, but the esxcli command is failing with this:

MethodInvocationException: C:\Temp\vSphere\Install-UCSTools.ps1:19
Line |
  19 |      $esxcli.software.component.apply.Invoke($arguments)
     |      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     | Exception calling "Invoke" with "1" argument(s): "A specified parameter was not correct: argument[1]"

Any thoughts on how I can get this working?

Thank You.

This seems to work:

Connect-VIServer 'XXXXXXXXXXXXX'

$hosts = Get-Cluster -name "XXXXXXXXXXX" | Get-VMHost

$cred = Get-Credential
$cmd = "mv /vmfs/volumes/INF03-PMAX-NonProd-DS01/ucs-tool-esxi_1.2.1-1OEM.zip /tmp"
$cmd1 = "python /opt/ucs_tool_esxi/ucs_host_inventory.py"
$arguments = @{
    'VIBNAME' = '/tmp/ucs-tool-esxi_1.2.1-1OEM.zip'
}
    
foreach ($esx in $hosts) {

    Get-VMHost $esx | Get-VMHostService | Where-Object { $_.Key -eq "TSM-SSH" } | Start-VMHostService
    $session = New-SSHSession -ComputerName $esx -Credential $cred -AcceptKey
    Invoke-SSHCommand -SSHSession $session -Command $cmd

    $esxcli = Get-EsxCli -VMHost $esx
    $esxcli.software.vib.install("/tmp/ucs-tool-esxi_1.2.1-1OEM.zip")

    Invoke-SSHCommand -SSHSession $session -Command $cmd1 
    Remove-SSHSession -SSHSession $session | Out-Null 

}

To use the Invoke method, you have to add the V2 switch

$esxcli = Get-EsxCli -VMHost $esx -V2