How to migrate large vmdk to new LUN

[FCE]

ESX 3.5 U4 build 158874, VC 2.5 U3 (after having to roll back from U4 due to the HA issues). Guest in question is a large file server with 3 vmdks, 16GB, 1536GB and 128GB with low disk I/O. Storage VMotion times out after 45-60 minutes when box is running leaving the machine a mess with .DMotion files in use rather than the root vmdks (solved using method found in http://communities.vmware.com/message/999890 rather than VMWare support suggested method of cloning disk). No errors in the host event log. With guest powered off "Relocate Virtual Machine storage" process times out after 45-60 minutes and rolls back, no errors in host event log. Source SAN EMC CX3-80, destination SAN Compellent FC disk. VMWare support tells me on my open ticket that there is no limit to how large of a machine can be Storage Motioned or Migrated offline but "practically I do believe that you met this issue" and suggested vmkfstools -i as the only option.

So a few questions for the masters:

1. What is the limit? I've successfully storage vmotioned machines with this exact same setup that have 512GB vmdks but have had no success with the 1536GB ones. Is the realistic limit 512, 1024 or something else? Is there a different limit for offline migrations than for storage vmotions? These servers are user home folder servers and if I had known that I'd never be able to move cleanly them I'd have created 8 or 9 smaller servers rather than the 3 really big ones.

2. vmkfstools -i requires the machine to be offline. In testing cloning the large volume took over 17 hours a rate of only 25MB/sec on average, monitoring the SAN showed that for the first 2 hours or so it ran at less than 10MB/sec before jumping up to fluctuate between 27-31 MB/sec. Both SANs have the capacity for much higher throughput and there are no disk I/O or SAN fabric bottlenecks exhibited anywhere in the systems. Why is this so slow and is there anything I can do to speed it up? Taking down the three home folder servers that I need to move for ~20 hours a piece is going to be frowned upon.

[azn2kew]

I'm not aware of Storage VMotion size limitation, but curious what Storage VMotion plugin did you use? Are you using Andrew Kutz Storage VMotion plugin located at www.lostcreation.com might give it a try. Have you thought of backup the whole vm and restore it to Compellent SAN not sure how fast but worth testing?

If you found this information useful, please consider awarding points for "Correct" or "Helpful". Thanks!!!

Regards,

Stefan Nguyen

VMware vExpert 2009

iGeek Systems Inc.

VMware, Citrix, Microsoft Consultant

[FCE]

I've gotten the same results from both the RCLI and from the SVMotion plugin, times out in no less than 42 minutes and no more than 63 at 10% according to VC and about 20-35 minutes after creating the 1536GB flat file for the large VMDK. Normally I use the plugin for all my storage vmotions but thought since I was getting the timeouts that maybe the RCLI would work better. Unfortunately no such luck, it's just much more time consuming to do through RCLI.

I also thought of trying to use a restored backup instead of ESX tools to do the migration, basically clone only the root disk C: and then build a new data disk and shadow copy disk. I'd have to take the share read only on the existing server, perform a backup, shut down, clone 16GB C: to new SAN, create new VM, add disk, restore backups, change share off of read-only. I'd lose my shadow copy history unless I also clone that volume but that's only 128GB so can be done in less than an hour. I think this process could complete in about 9 hours so about half the time it took to do the clone and the server would be online in read-only for ~1/2 of that and would be restoring folders alphabetically for the last part so it would offer less of a true downtime. Next week I'll be running the tests on that option to see what the actual time is for that process.

[depping]

Why not use VMware Converter if you can't have the downtime?

http://www.vmware.com/products/converter

As far as I know there's no limit in size for Storage VMotion. I've never encountered it.

Duncan

VMware Communities User Moderator | VCP | VCDX

-

---

Blogging:

Twitter:

If you find this information useful, please award points for "correct" or "helpful".

[Troy_Clavell]

I agree when moving that much disk, you would be better off using VMware Converter. It will also allow you to resize the partitions if you need to.

[FCE]

Converter will take longer than a restore from backup I think and still requires the same read-only on the share while it's going on to prevent users from adding or changing files between the time converter finishes and I can shut off the old guest. I haven't tried it yet but I also suspect I'll get that same unexplained slowness with converter that I did with a clone of the disk.

[JoJoGabor]

How about simply using something like Robocopy to Mirror the drive. EIther create a new VMDK on the new LUN and assign it to the VM, or create the new LUN and map it as an RDM, then use robocopy to copy between the two, then you can script the migration of the windows shares to the new folder when you are ready to switch over.

Or something like vizioncores vReplicator, will allow you to replicate the VM. THis will do the base replication, then do incrementals as necessary

[JoJoGabor]

On another note - how are you finding the Compellent with VMware - I;ve had a brief experience with Compellent and it looked really good, but a client had a few problems with it, but then they kept having Power outages. Interested to know its robustnedd and integration with Vmware.