Vsc backup error because vm is currently migrating

[andwo]

Hi expert

I am currently install a new vSphere 6.5 Farm. Currently two esxi, two VMs, one Test Server (w2k12) and one is the vCenter Server (w2k12).

Four Datastores from NetApp over nfs. Both VMs are migrating between the datastores or esxi without Problems.

Now i install the NetApp VSC 6.2.1 and make a Backup Job for each datastore. The Backup fails when in the Datastore one of the VMs.

When i migrating the Maschine in another Datastore and Backup then, no Problem (the Datstore isnt empty, another VMs from older Farm is in it).

When Backup fails the Webclient says:

At least one datastore must be backed up. Either explicitly or implicitly through backing up a virtual machine

SMVI Log says:

2016-12-13 13:12:14,621 PROGRESS - Starting backup request

  2016-12-13 13:12:14,861 INFO - Generating backupName for the scheduleJob 17f0945f-0139-47e6-9753-736ffae05846 is backup_test_nst03_nfsq1_20161213131214

  2016-12-13 13:12:15,304 WARN - Virtual machine w2012-test1 is currently migrating and will not be backed up

  2016-12-13 13:12:15,307 INFO - Skipping datastore Ds_Sata_nst03_nfsq1 as it does not contain any valid virtual machine configurations.

  2016-12-13 13:12:15,309 PROGRESS - Backing up datastore(s) ([])

  2016-12-13 13:12:15,380 ERROR - At least one datastore must be backed up. Either explicitly or implicitly through backing up a virtual machine

  2016-12-13 13:12:15,387 ERROR - FLOW-11019: Failure in VmMapToStorage: At least one datastore must be backed up. Either explicitly or implicitly through backing up a virtual machine

  2016-12-13 13:12:15,389 ERROR - FLOW-11010: Operation transitioning to abort due to prior failure.

  2016-12-13 13:12:16,152 PROGRESS - Sent status notification message to [andre.worms@goebber.de]

  2016-12-13 13:12:16,161 PROGRESS - Storing logs for backup_test_nst03_nfsq1_20161213131214 in file .\repository\logs\17f0945f-0139-47e6-9753-736ffae05846\backup_backup_test_nst03_nfsq1_20161213131214.xml

The Migrating Process of the VMs is finished and success. Where can i see why VSC means the VMs is in Migration Status?

I have VSC 6.2.1 deinstalled and installed VSC 6.1. The Problem is the same thing.

Any Ideas?

Thanks

André

[SureshKumarMuth]

Looks like, this is as per design.Check the following  article if  it  matches

https://kb.netapp.com/support/s/article/why-is-the-error-virtual-machine-vm-name-is-currently-migrat...

Regards,

Suresh

[andwo]

Thanks Suresh, i have read this Artikel.

But My Problem ist why is the VM marked as "Migrating Status" ? Yes, i have the VM migrated. Yes, the Migration ist finished and sucessfull.

And VSC says by the Backup the VM ist currently in migration! Why? Where I can see in the Vm files the migration? Perhaps the status is not reset properly.?

Regards,

André

[SureshKumarMuth]

As per vCenter server and ESXi host, the migration status is completed. Somehow, this is not updated in the Netapp side where the vSC still thinks the state as Migrating.

Also validate from the vCenter that the virtual machine does not having any running migrate task. Check if it registered on proper host. Retry backup task and check if the issue persists.If the issue still persist, Can you restart the vSC service and check if that helps?

Regards,

Suresh

[JeffreyRusso]

I also had the same error:

WARN - Virtual machine <MYVIRTUALSERVER> is currently migrating and will not be backed up

I tried multiple things, rebooting vCenter, re-registering the plugin, reinstall VSC and recreating the job.  They all failed.  I even tried migrating the VM to another host and datastore.  Still didnt' work.  Finally I removed the VM from inventory and reregistered it in vCenter.  The backup worked again.

[Bolsinger]

The Solution works for me only until I VMotion the VM again. After the move is complete the issue apears again.

Do you have any more ideas? I tried a downgrade to 6.2.1 but no Change.

[JeffreyRusso]

I have a case open with NetApp and hopefully will hear back today.  They have been working on it for about a week now...

I have noticed in my case the message goes away after exactly 24 hours.  Once I hear back from NetApp I will update this thread.

[kneese]

We had a case open with Netapp. According to two engineers, this is a known bug. The bug ID is1050057 in case anyone wants to track it. They did not have an estimate of when it might get fixed.

[JeffreyRusso]

Is that bug id for the migration issue or the http 500 error using the log viewer.  The netapp forums reference that bug id back in November referring to to the log viewer.

[Shrikant_Gavhan]

Did you verify from esxi command line if any task still running or stuck for this VM.

Please try vim-cmd vmsvc commands to see if any active task against your VM.

You may have to use command help to find exact command.

[netrz]

Hi,

yesterday i upgraded VSCA to 6.5b and I installed ESXi 6.5 host-patches  which were shown by update manager as availble since march, 9th.

I just migrated a VM and I could SMVI-backup and restore it some minutes later. The problem seems to have diappeared.

andwo, jeffrusso: can you test this too?

[netrz]

I forgot, there is another factor which changed yesterday in my test environment:

I was / am affected by a problem, which resulted in an incomplete detection of ESX-Hosts and SVMs by VSC even though I am using cluster management LIF. The symptoms are discribed here: VSC 6.2.1 P1: Incomplete Netapp Storage System detection - NetApp Community

Yesterday I had a session with Netapp and narrowed down the cause for this problem. A driver issue prevents VSC getting from all necessary informations from ESXi-hosts. We tested a workaround and VSC was able to get the SVMs and NAS ressource informations from ESXi hosts. Unfortunately we cannot use the workaround permanently due to side effects. Theoretically this could also be a factor with the "migrating VM"-problem. So if you are affected, please check if your VSC shows symptoms as discribed in the link above.

[JeffreyRusso]

So I had upgraded one host to see if I could fix another issue with the lsi_mr3 driver causing VSC issues with a Cisco USC c220 m4.  The migration issue was still there, however I havent upgraded the vcsa to 6.5b yet.  I will and let you know how I make out.

Thanks!

[JeffreyRusso]

Was your driver issue with the lsi_mr3?  If so i reverted it back to a megaraid_sas driver and it fixed my host detection issues with VSC.

[netrz]

Hi,

yes our VSC-issue is related to the LSI-driver. Netapp's workaround was to disable this driver if you dont need it. But after next ESXi-reboot the RAID-controller was not shown at storage adpters anymore and vcenter Host Profile check complaind about some changed directories or IDs. We did not change anything further but after next reboot, ESXi installed the driver again automatically and VSC.log showed the Host discovery errors again. (But VSC itself still has the information about used SVMs, NAS-Volumes and related ESXi-Hosts.)

I googled a little bit and found out, that our Dell RAID controller uses an LSI controller chip. So I see no safe way to use netapp's workaround.

[JeffreyRusso]

NetApp gave me the same work around, but I wasn't going to disable the storage controller either.  I boot off the local storage.  So I did some googling as well...

I found my Cisco RAID controller (also LSI based) on VMware's site and saw multiple drivers listed.

VMware Compatibility Guide - I/O Device Search

Running the follow:

esxcli software vib list | grep mega

scsi-megaraid-sas              6.610.15.00-1OEM.600.0.0.2494585       Avago   VMwareCertified   2017-03-04 

scsi-megaraid-mbox             2.20.5.1-6vmw.650.0.0.4564106          VMW     VMwareCertified   2017-03-04 

scsi-megaraid2                 2.00.4-9vmw.650.0.0.4564106            VMW     VMwareCertified   2017-03-04 

lsu-lsi-megaraid-sas-plugin    1.0.0-7vmw.650.0.0.4564106             VMware  VMwareCertified   2017-03-04

I saw that I already had another supported driver installed (scsi-megaraid-sas, Avago bought LSI I believe).  So instead of removing the lsi_mr3 driver, I disabled it.

esxcfg-module -d lsi_mr3

I then rebooted and the RAID card was now using the other driver and the VSC discovery issue was resolved.  If I didn't have the other driver I would have had to manually install the VIB that I could download from VMware first.

I would suggest that you find your RAID adapter on the VMware website and select another driver to try...

Dell

VMware Compatibility Guide - I/O Device Search

[netrz]

I loooked into compatibility guide but if I interpret the result correctly there are only lsi-drivers for my RAID-adapter:

VMware Compatibility Guide - I/O Device Search

[JeffreyRusso]

Damn.  That stinks.  Hopefully someone will fix it soon.  Its been a known bug for quite sometime.  I am not sure who should fix it since NetApp was blaming Cisco, but you use Dell and have the same problem...

It appears from my testing that vCenter 6.5b does indeed fix the migration error for me as well.  I had previously fixed the driver issue and that didn't change the migration message.