Copy .vmdk files from Fibre to Fibre instead of using Ethernet on ESX Server 2

[dmaster]

Hello All,

I was wondering if there is a way, a little program or whatever to copy data from one Fibre Channel SAN datastore to another Fibre Channel SAN datastore with ESX Server 2.

Just like you can do with Veeam FastSCP but then without the 2 GB limitation (doing an export of the vmdk files isn't an option).

In hour situation we have an datastore on a DMX 2000 storage system and a datastore on a DMX 3 storage system. I need to migrate VM's one by one on an ESX 2 host SAN attached (DMX 2000)

to an ESX 3 host also SAN attached (DMX 3). To migrate the data we arranged an shared LUN witch is masked to both ESX hosts..When the vmdk files are on the shared lun VMware dmotion will do the rest for me..

upgrade the ESX 2 VM to an ESX 3 VM and migrate the data through the SAN, this wil result in a much shorter downtime, than copy over the network with 100mbit..

Any Suggestions ?

Regards,

Dennis

[sbeaver]

That is about it except to upgrade the tools and virtual hardware. Is there any part of this you are concerned with?

Steve Beaver

VMTN Forum Moderator

*Virtualization is a journey, not a project.*

[Texiwill]

Hello,

Easy way to do it....

If you use the VIC, migrate the VM from hostA to hostA just change the LUN to which it is going. Then the copy happens entirely within the ESX server without touching the network and in effect fibre to fibre. Or use the commands 'mv', 'scp', or 'cp' command from the CLI.

Of course you will have to do this from an ESX v3 host that sees both LUNs.

Best regards,

Edward L. Haletky, author of the forthcoming 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', publishing January 2008, (c) 2008 Pearson Education. Available on Rough Cuts at http://safari.informit.com/9780132302074

Message was edited by: Texiwill

[dmaster]

Thanks for your suggestion but i canno't upgrade the ESX 2 server to ESX 3. a lot of VM's are running on the ESX 2 host and are only allowed to powerdown one at a time just for the migration.

[sbeaver]

If you use dMotion then nothing should have to go down you can do the migrations live.

Steve Beaver

VMTN Forum Moderator

*Virtualization is a journey, not a project.*

[dmaster]

The last section when the .vmdk files have been migrated to the "shared LUN" also on the ESX 2 server. I can do a cold migration of the VM to the final LUN wich is also visible , this results in a dmotion through the Fibre Channel SAN.

My VM will be automaticly upgraded to the ESX 3 layout.. stays left the VMware Tools Upgrade.. but since ESX 3.02 this is very easy..

So the biggest challange is to copy the data from datastore 1 to datastore 2 on the ESX 2.5.4 server. I canno't easily migrate the data because of the two diffrent physical storage systems.

[dmaster]

If you have no vmotion available, you canno't do a live migration with dmotion (only available in ESX 3 when you do an upgrade from ESX 2). but when you use a cold migration, the data will be migrated over the SAN instead of the ethernet..

[Texiwill]

Hello,

I had the same issue, only two machines and need to get ESX upgraded. I planned for a full outage of everything not just a short blip to reboot the servers because that will NOT happen. Here is the general scheme of the upgrade:

• Backup everything, ESX Server, VMs, etc (could be many hours for this to happen)

• Once you go to v3 there is no going back

• Prepare ESX server for upgrade,

• vmotion off VMs or powerdown.

• disconnect Fibre channel from Server, depresent LUNs, or something like this

• Run Upgrade or reinstall ESX Server (20 Minutes or longer depending if you make a mistake)

• Reboot Server and represent LUN/reconnect FC to server, rescan for all VMFS LUNs

• Configure Server (anywhere from 10 minutes to several hours), configure VC

• vMotion all VMs back to this server

• Repeat for next server

Work with VMs

• Shutdown each VM

• Upgrade VMFS-2 to VMFS-3 or migrate VMs to new VMFS-3 LUN

• Boot VMs, upgrade Virtual Hardware, reboot VMs

All told this can take anywhere from 30 minutes to all day to perform and there is no going back. For example one such upgrade required making new LUNs with new Drives on a small SAN, we ended up doing all the ESX upgrades then restoring from Tape all the VMs as there was no space to place to hold anything on the SAN.

If you are concerned with downtime I suggest practicing first so that you are aware of all the pitfalls.

Best regards,

Edward L. Haletky, author of the forthcoming 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', publishing January 2008, (c) 2008 Pearson Education. Available on Rough Cuts at http://safari.informit.com/9780132302074

[dmaster]

Hello Texiwill,

Thanks for your suggestion, but still this is not the answer i am looking for..

is there not a way to copy the .vmdk files for example with the "cp" command on the service console and/or give some parameters that it's uses the Fibre cards instead of the ehternet cards.

[sbeaver]

Not unless it see all LUNS in question and if that was the case you could use dMotion.

BTW when copying vmdk you do not want to use cp but use vmkfstools instead.

[dmaster]

Hello sbeaver.. thanx for helping..Our biggest problem is that we have two different san environments wich are fully seperated.

we have now one dmx 3 storage system wich FA ports are connected to both SAN Fabric switches. That's why we could create an new LUN wich can be accessible by ESX servers in both SAN's.

this is the LUN on wich i have to copy the vmdk files to. this LUN must be VMFS2. .. i canno't vmotion because there is no vmotion network between the ESX 2 and ESX 3 environment. plus with vmotion you don't copy/migrate the actual data.

The other problem is that only one VM at a time can be powered down on the ESX 2 server. so upgrade to ESX 3 is not an option (in this case i coulded minimal use the clone feature in vmdkfstools wich is not available in ESX 2 to reduce the down time).

Other problem downtime = max 4 hours .. biggest VM is 420GB.. max ethernet for all VM's is 100mbit.

After the data has been migrated to the special shared LUN i have to change the .vmx file of the VM to point to the new datastore.

Next i can do a cold migration with Virtual Center 2. upgrade to vmfs3, new virtual hardware and all of this is over the FC San.

That's why we realy wan't to use the power of the 2Gbit Firbe Channel cards. but how ?

I hoped for the Veeam fast SCP program to handle this but it has limitation to 2GB files so i was not able to copy the .vmdk files.

[sbeaver]

Do you research on dMotion. You can migrate the data while hot when going from ESX 2 to ESX3 which might save you a step.

[dmaster]

no i am not researching dMotion ?:|.. just finding a way to do the impossible..

Because on the old esx server we don't have a vmotion network..i was not able to do the hot migration.. but i'm sure it's working when that was available.

[sbeaver]

Not vMotion DMOTION (notice the D before motion)

[dmaster]

Oops sorry for the typo.. "post updated" ...B-)

let me explain the vmotion part.. i tried to do the hot migration with datastore migration "dmotion".. but virtual center refused to start the hot migration because of the not existing vmotion network..

that's why i used coldmigration and it uses the FC adapters to migrate the data to the new datastore.

[Texiwill]

Hello,

Use of 'cp' or 'mv' does not use the ethernet, you login to the service console and do the following:

cd /vmfs/

cp -r VMName.vmdk /vmfs/volumes/VMname

There is absolutely no network involved besides the ESX server and the fibre network already there. However this is not a 'fast' option as ESX throttles all copies/writes to the FC down to something painfully slow. If I change the 'cp' to 'scp' then there is also no network involved as no host was mentioned.

Yet if you do not have an ESX server on both SANs then this is also impossible.

Other options,

Mirror inside the SAN Fabric LUN to LUN on the ESX v3 SAN. This assumes they at least talk to each other on the same fabric.

Make a backup of the running VM to tape then restore the VM on ESX v3 import and launch.

Without the ability to do dMotion, vMotion or having an ESX server on both fabrics you have to do the migration by hand somehow. You need to move the data.

Best regards,

Edward L. Haletky, author of the forthcoming 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', publishing January 2008, (c) 2008 Pearson Education. Available on Rough Cuts at http://safari.informit.com/9780132302074

[dmaster]

Hello Texiwill..

thanx for the help.. i was not aware of the fact "That ESX throttles all copes/writes to the FC down to something painfully when using 'cp'or 'mv' on the service console".

i will test thursday a migration with vmkfstools .. i will post the results.. if that results in a very good speed.. i have my answer..

i didn't check it because the cold migration through virtual center took awfully long..

regards,

Dennis

[Texiwill]

Hello,

If you did a Cold Migration from HostA to HostA and only changed the Storage location of the VMs then you in effect did the same as a cp or use vmkfstools. If you however went from HostA to HostB then you went over the wire.... I suggest trying the first option. If you do try from the CLI, you will not need to use vmkfstools. A straight 'cp' from VMFS to VMFS is sufficient.

Best regards,

Edward L. Haletky, author of the forthcoming 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', publishing January 2008, (c) 2008 Pearson Education. Available on Rough Cuts at http://safari.informit.com/9780132302074

[dmaster]

Hello Texiwill,

I did a couple of tests..

using the "cp" command from datastore on DMX 2000 to datastore on shared LUN (DMX 2000 / DMX 3) took me 25 minutes for 9GB

using Virtual Center 2 with cold migrate this took me also 25 minutes

using vmkfstools on CLI this took me 15 minutes..

but this still this is not the proper speed..

when i do a dmotion (cold migration) from the datastore with the shared LUN (DMX2000 / DMX 3) to the final datastore on DMX 3..

from the ESX 2 host to the ESX 3 host it took me only 5 minutes.. how is this possible ?

regards,

Dennis Derks