Can't hot remove virtual disk from running VM after adding passthrough device

[geppi]

I have a VM with 2 virtual disks under ESXi 5.1.

When the VM is running I can hot remove disk 2 in the VM settings dialog.

However, the remove button is greyed out when the VM is running as soon as I add a PCI device that has been passed through by the ESXi host.

The passthrough device is a SAS controller without any disks connected.

Why is it not possible to hot remove a virtual disk that resides on a datastore that is not related to the passthrough device ?

Is there any parameter that can be modified to enable removal ?

[geppi]

I've worked a little more on this and still don't understand what's blocking the removal of a virtual disk from a running VM when it has direct-IO devices configured which are passed through by the ESXi host.

It even doesn't work when you go pretty low level on the ESXi shell with vim-cmd.

Got the VM-id with "vim-cmd /vmsvc/getallvms"

and tried to remove the second virtual disk with "vim-cmd /vmsvc/device.diskremove <VM-id> 0 1 n".

( <VM-id> beeing the number of the VM in question returned by the first command, 0 is the controller number, 1 the disk unit number and the n stands for not deleting the related vmdk file)

It works and removes the second virtual disk from the running VM when it does not have direct-IO devices configured.

As soon as I run the VM with a SAS controller connected that has been passed through from the ESXi host the command fails with:

"Reconfigure failed"

Why is it not possible to remove a virtual disk from a running VM in that case ?

What is preventing the vim-cmd command to work ?

Is there a setting that can be tweeked to enable the hot removal ?

[geppi]

Since I had posted this problem over the holidays and have not received a reply I'm trying my luck again to see if somebody has an idea or any hint ?

[Linjo]

When PCI Passthrough is configured many things are off the table like vMotion etc and you need to dedicate the entire allocated memory to the VM.

I am not 100% sure but this is possibly another effect of enabling passthrough.

// Linjo

[geppi]

Well, it is most definitely an effect of attaching a PCI device to the VM that has been passed through by the host.

If it's attached in the VM settings the hot remove of a virtual disk on any datastore is impossible.

As soon as it's removed from the configuration of the VM the hot removal becomes available again.

In this situation the SAS controller is still enabled for passthrough in the host configuration. It's just not connected to the particular VM.

But again, I don't understand why this is necessary, what prevents it and if there is any parameter or method to enable it.

[Rubeck · Accepted solution]

I am not 100% sure but this is possibly another effect of enabling passthrough.

From http://pubs.vmware.com/vsphere-51/topic/com.vmware.vsphere.networking.doc/GUID-BF2770C3-39ED-4BC5-A8...

The following features are unavailable for virtual machines configured with DirectPath:

■	Hot adding and removing of virtual devices
■	Suspend and resume
■	Record and replay
■	Fault tolerance
■	High availability
■	DRS (limited availability. The virtual machine can be part of a cluster, but cannot migrate across hosts)
■	Snapshots

If you are a Cisco shop you've might have options to overcome this.... See doc link..

/Rubeck

[geppi]

Thank you for the link to the documentation Rubeck.

It looks like I didn't search properly and was obviously too focused on storage (SAS adapter passthrough) to find it in the Networking chapter.

Since it's stated explicitely in the official documentation I'm afraid that nothing else than a rude hack could enable it, if at all....

...still wondering what the deeper reason is in my special case.

[Rubeck]

...still wondering what the deeper reason is in my special case.

Now you've got me thinking about this too... To start with, let us kneel before Linjo and hope he can provide us with some insight into this 🙂

/Rubeck