Thank you very much for helping me @LucD 

Is this something like this:
$esxcli = Get-EsxCli -VMHost <Host Name>
$esxcli.software.vib.remove --vibname 'scsi-mpt3sas'

But I am getting error. Not sure what will be the correct commands