My cold migration (ESX 2.5 to 3.01) experience and procedure

[FredPeterson]

Hello everyone, I thought I'd just create a new thread with the details of what steps I took for a successful cold migration of my VM's from one disparate SAN in one datacenter to another on the other end of our building. These steps are a mix of things I found in different threads that seemed to not all be complete in one place. These steps I wrote down I found to be the easiest and simplest to execute with little to no issue. Hopefully this will benefit others who are going through the same discovery process as I did.

1. Perform a VM Tools Install on VM to be migrated - either perform an upgrade or allow the "Modify" section to come up and remove the Shared Folders component - doesn't work in ESX anyway.

2. Power down the VM.

3. SSH to any ESX 3 host (preferrably destination host)

4. cd to /vmfs/volumes/VOL_NAME and mkdir VM_NAME

5. SSH to the ESX 2.5 host that hosted VM_NAME

6. cd to /home/vmware/VM_NAME

5. scp VM_NAME.vmx to any ESX 3 host to the appropriate VMFS volume. Example:

scp web01.vmx root@esx3:/vmfs/volumes/SAN/WEB01/web01.vmx

6. On the ESX 2.5 host cd to /vmfs/VOL_NAME

7. scp appropriate VMDK or DSK files to ESX 3 host - look at config in VirtualCenter 1.x if necessary if you don't know specific name.

Example:

scp web01.vmdk root@esx3:/vmfs/volumes/SAN/WEB01/web01.vmdk

8. Exit SSH session to ESX 2.5 host

9. On the destination ESX 3 host, cd to to /vmfs/volumes/VOL_NAME/VM_NAME

10. Using vi or nano, edit the VM_NAME.vmx file and remove all references to the old VMFS2 VOL_NAME in the lines containing references to the VMDK files.

Example from vmx file:

scsi0:0.name = "san:web01.vmdk" and remove "san:"

11. Save the changes to the vmx

12. On the destination ESX 3 host, register the VM while in an SSH session.

Example:

vmware-cmd -s register /vmfs/volumes/SAN/WEB01/web01.vmx

13. Exit the SSH session to the ESX 3 host.

14. In VirtualCenter, the VM will appear and be turned off. Right click on the VM and choose 'Upgrade Virtual Hardware'

15. Power the VM on and perform a VMWare Tools Install to be at the latest version, reboot and thats it.

You can take the opportunity to also rename your vmdk or dsk files to remain consistent with a VM's actual name, just remember to make the changes to the VMX file. I did this on a couple migrations without problem.

I ran into weird problems with the HGFS service somehow not being removed and Windows would complain. I could not for the life of me get this to not happen. I think it had more to do with when the VM I migrated was created and if its VMWare Tools were properly upgraded and installed at its creation then anything I did. Not all of the VMs I did recently had this issue but honestly, other then the Event Log message and the popup on reboot, its no big deal and I can live with it. At a later time I'm sure I will go back and retrofit it to stop the message from happening but right now its not important.

Any comments or changes to this procedure are welcome and I hope this helps others.

[bister]

Just being curious: Why don't you use Virtual Center to migrate VMs. I used the "clone VM" option to move VMs from one storage/host to another... I think the result is the same with less work...

Respectfully,

Christian

[FredPeterson]

Because I am keeping the two environments completely segrated, this includes VirtualCenter management.

In retrospect, if I'd put more time into the project (forcibly made to make this hapen due to the "urgency" of the datacenter move that was ridiculous) I probably would have consolidated all environments to one VirtualCenter instance, but I didn't.

[benny_hauk]

That is basically the procedure I am in the middle of. It's worked 100% so far. Only points where mine differ is that the VMFS2 and VMFS3 volumes are all readable by the ESX3 hosts, so from the host I can just do a 'cp' from one to another instead of having to do a 'scp' across the network (just copies from one LUN to another).

Also if the 2.5 version of the VM was using vmxnet, I go ahead and remove it and use the vlance in 3.0. It seems to be the way they suggest going now so as long as I've got the VM down I've been making this change. Just make sure you know the ip address to reassign once the new card is installed. By the way, once a card is removed from a Windows server (virtual or physical) and replaced with a new one getting the same ip address you will get a warning saying another adapter in that server has the same ip address, "are you sure you want to continue?". At one point I knew the registry keys to delete to actually remove the old NICs entries in the registry that don't get removed when the card gets removed. Anyone know these?

We ran into a problem where our dhcp VM (running a dhcp service from MetaIP) was choosing this "removed" virtual nic instead of the one that was actually in the VM and we had to remove these registry entries to fix the problem.

[bister]

That's a point. But wouldn't it still be easier to add an old ESX temporarily to the new VC to migrate the VMs off the ESX? But anyways... nothing else to append here

Respectfully,

Christian

[squidfishes]

I used the VMware convert tool to migrate from ESX 2.5 to 3.01. Worked beautifully.

[benny_hauk]

I've just now started using this to P2V a couple machines and so far like it better than P2V Assistant and Leostream's product. Very nice. Good idea to use it for the migrations since the VM can say online during the cloning process.

Unfortunately where it would really come in handy (large email and database servers) can't really be cloned while the application is up and running because the data changes during the migration. I suppose I could resync it when the migration is finished and just copy a little bit of the "changed" data when it's complete. Might be better than bringing to whole system down for 2 or 3 hours. I think I'll try using converter for the web servers, domain controllers, systems that aren't going to significantly change after hours anyway. Great idea. Thanks!

[dmanconi]

HI

Anyone migrated servers from ESX 2.5.4 to ESX 3.01 that have RDM connections? I have a server with RDM's to a MSA1000, and am not entirely sure how they will migrate.

My Setup is a Single ESX 2.5.4 server connected to a MSA1000. All disk space is allocated on the MSA to the ESX and other servers, so I have no spare disk to allocate for a new LUN to migrate the VM\s to.

My plan was to do the following:

1) use a swing ESX server to move the VMDK files to from the SAN,

2) rebuild the original ESX server with ESX 3.01

3) recreate the VMFS LUN, and migrate the vm's back.

4) Upgrade hardware if needed

4) Re-register them and we are away. Obviously update the VMtools as well.

The only issue I have is the RDM connections and how they will react. I am assuming as they are just RDM's that I can represent the LUNS from the MSA to the ESX server, and attach them to the respective VM's before starting them (a file and Exchange server in this case).

Anyone have any experience with this

Thanks

David

[benny_hauk]

I haven't yet come across this scenario in my reading of the documentaiton either. I'll be anxious to hear what others, more experienced or more learned, have to say. My assumption so far has been that migrating these will actually be easier (or at least a heck of a lot faster) since the actual data doesn't have to move anywhere (a raw LUN is a raw LUN regardless of whether it is accessed via a metadata vmdk file residing on aVMFS2 partition or a VMFS3 partition. Worst case I guess is that I copy the metadata vmdk file from the VMFS2 partition to the VMFS3 partition and ESX3 tells me this is an incompatible VMDK and so I have to create a new VMDK metadata file in ESX3 to point to the LUN. Not a bad worst-case really. In fact, I may just do it that way from the start just so it's a little cleaner (creating from scratch instead of moving existing then upgrading). Especially considering that the alternative is to copy gigabytes and gigabytes of VMDK files if RDM isn't being used. I'm actually looking forward to migrating our RDM-based VMs.

[SST1at]

Simply working. - Thank you for this add!!!

[Mayur_Patel]

hey Benny.Hauk:

To delete the "hidden nic", from the Windows server that is complaining:

- open a command prompt

- type in: set devmgr_show_nonpresent_devices=1

- type in: start devmgmt.msc

- this will start the device manager, click Show Hidden Devices on the View menu

-then all of the hidden device will show as grey'd out, you can delete them.

MS has a knowledge base article on their web site, but I don't remember the Q#.

Mayur

[benny_hauk]

This is fantasic! I finally found the registry keys that MetaIP support told us to delete, but I don't think I'll post that here since your solution is so much cleaner than a registry hack. Thanks a bunch!

[einstein-a-go-g]

Also if the 2.5 version of the VM was using vmxnet, I

go ahead and remove it and use the vlance in 3.0. It

seems to be the way they suggest going now so as long

as I've got the VM down I've been making this change.

Just make sure you know the ip address to reassign

once the new card is installed. By the way, once a

card is removed from a Windows server (virtual or

physical) and replaced with a new one getting the

same ip address you will get a warning saying

another adapter in that server has the same ip

address, "are you sure you want to continue?". At

one point I knew the registry keys to delete to

actually remove the old NICs entries in the registry

that don't get removed when the card gets removed.

Anyone know these?

Do it the recommended way, and at the command prompt type

type in )

Device Manager Console, from the "View" menu, select "Show Hidden Devices".

Remove the grayed out old VMware PCI and AMD Network cards.

I do this before assigning IP Addresses, because I've had issues in the past with default gateways disappearing between reboots.

whoops, already answered, would help if I read all the thread!

😮

Message was edited by:

einstein-a-go-go

[einstein-a-go-g]

We have a seperate VI 3.0 (esx3.0/vc2) and ESX 2.5.3/VC1.3 environment, we wanted to keep them that way, due to various issues with VC2 management.

I add the ESX 2.5.3 server temporaily to vc2, cold migrate VMs that need to be migrated that weekend, and then remove it from VC2 after migration, and then connect VC1.3 back to it.

But I've just found ESX Migrator, and it's EXCELLENT! Try it for free! 2 free Migrations from Vizioncore, migartes in real-time, and keeps a copy of the original server un-touched, just in case for rollback should something go wrong.!

[patk]

I have a very simliar procedure used except for the following differences:

*I only SCP the VMDK/DSK files from the source 2.5 host VMFS partition to the destination 3.0.1 host VMFS partition

*I then CP the VMDK/DSK from the source 3.0.1 host VMFS partition to a VMFS LUN using the sparse=never[/b] switch

*My last step is to create a new VM using the wizard and the existing imported VMDK/DSK disks. I will then upgrade the virtual hardware before powering on the machine.

Key questions referring to both procedures is:

1. Is it necessary to use the cp sparse=never[/b] switch when only moving between VMFS partitions?

2. Would it be beneficial to use the vmkfstools -i command when moving between local/SAN VMFS partitions?

Thanks,

Pat

[sbeaver]

I recommend using vmkfstools to move or migrate any vmdk files and do not use CP because of the locks on the files

[benny_hauk]

I've changed my procedures to use 'vmkfstools -i' in VI3 instead of 'cp'. It's much cleaner than the ESX 2.5 counterpart, but I still think that command needs a lot of work. I posted more detail here:

http://www.vmware.com/community/thread.jspa?threadID=78190