What kind of error are you getting when you try to delete the vm?
You should not get any kind of NSX firewall error when deleting a vm from ESXi. Thats part of the magic of nsx, that the rules stop apliying to the object when it is gone in your inventory. (So its easy to have "firewall rules deletion")
Can you share specific task and error sshot?
I have the same question as Matthew. I can understand once the VM gets deleted that it will get removed from any group where it is defined as a VM, that's fine. If you have it defined as an IP address, how can you search to see if the IP address has been used in any rules via a group?
But also with Matthews point if you could see the rules that it is using before it is deleted, you could find a rule that now needs deleting as it was specific to that VM? Would this be covered by using the filter on the rule set? If so is there any way to prevent it showing rules where '* Any' is used.