VMware Cloud Community
altapowder
Contributor
Contributor

Storage vMotion fails with error "The operation timed out"

I have been having some problems lately with storage vmotions in my environment.  I am trying to storage vmotion some of my vm's to some new thin-provisioned LUN's on an EMC CX4-120 array that is using the fully automated storage tiering (FAST) technology.  The problem occurs almost everytime I try to move a VM.  The storage vMotion will run for approximately 20 minutes and then will fail with a generic error message that just says "Operation Timed Out".  Has anyone run into this issue before and what did you do to fix it?

0 Kudos
4 Replies
DSTAVERT
Immortal
Immortal

A clone to the same destination as a test???  I would look for clues in logs. You could temporarily change the logging level in vCenter to Verbose or more to see if you get more information about the why. Is the VM heavy I/O?

Would be worth a call to VMware Support.

-- David -- VMware Communities Moderator
0 Kudos
depping
Leadership
Leadership

At what percentage is it stuck? And can you actually see it is copying?

Duncan (VCDX)

Available now on Amazon: vSphere 4.1 HA and DRS technical deepdive

0 Kudos
altapowder
Contributor
Contributor

The percentage seems to vary, but it always fails around 20-25 minutes.  I can see the data moving but it moves at a much slower rate than if I was doing a storage vMotion between two "thick" LUNs on the same SAN.  It seems to be much slower on the "Thin" LUN.

0 Kudos
peetz
Leadership
Leadership

Hi,

just a guess: Have you enabled Storage I/O control on the source and/or target datastore?

We discovered that this can significantly slow down or even lead to timeouts of Storage VMotion tasks if the storage is under load.

Obviously Storage I/O control also regulates and throttles Storage VMotion tasks. This sounds logical, but is something that we have not thought about before.

We temporarily disabled Storage I/O control, then the Storage VMotion tasks ran with expected speed and did not time out again. An alternative is to raise a specific timeout value (s. http://kb.vmware.com/kb/1010045), but be careful: This will lead to the effect that the virtual machine will be suspended for a very long time (up to 2 minutes in our tests!) durng the Storage VMotion task, if if cannot get enough I/Os.

- Andreas

Twitter: @VFrontDe, @ESXiPatches | https://esxi-patches.v-front.de | https://vibsdepot.v-front.de
0 Kudos