Problem with FLR with vmware data protection 5.1

[MNCNE]

Hallo,

We have a problem with our vdp and after two days of reading manuals and kb article we dont know what to do anymore.

We have ESX 5.1 and vCenter 5.1 and using also the 2TB Data Protection 5.1.

The VDP is working to backup the hole machines and also to recover them without a problem.

But if we want to use the FLR option it doesn't work and we have no clue why.

What we tested right now was for example we made a backup from our vcenter and than from the vcenter we tryed to access the https:vdp-ip:8543/flr/ on the webinterface we tried the basic login and also the advanced login but both don't work.

We always get the message : "login failed cannot find vm at ip-of-vm".

We checked the firewall it is off. We also tried it from another client but no success and we can also see under restore in the webclient from vcenter the restorepoint of the full backup machines.

In the webclient on the vc we also the data protection and can connect and work with it.

Has anybody a clue?

The basic login we tried with: local administrator user and password, with domain\administrator, localmachinename\administrator

The advanced login we tried with: domain\administrator (which is taken in the configuration of the vdp to connect to vc which is working).

noting want to working and we dont find a error or logfiles to work with it.

thx for any help in advance

best regards

[GSparks]

Good day -

One issue we have defined with FLR returning this error can occur if you are using a Windows vCenter.

There is a known issue with Microsoft SQL Server 2008 RTM - 10.0.1600.22 (X64), that you may receive incorrect results when you run a complex query that contains aggregates functions, join functions, and distinct functions in sub queries. 

This may cause the FLR query to the Windows VC to return incorrect results and end up throwing the error "Login failed.  Cannot locate login client <IP address of VM> in VDP."

If you are running Windows VC with that version of Microsoft SQL server, then upgrading to Microsoft SQL Server 2008 R2 SP2 ( List of the bugs that are fixed in SQL Server 2008 R2 Service Pack 2  )  should fix the issue. 

You may also receive this error if you have multiple VM's in the vCenter's inventory with the same IP address.

[MNCNE]

Hallo,

thx for the answer, sorry that it needed so long time to reply but at the moment to many things are ongoing.

I checked the SQL Server and its was from the beginning installed the SQL 2008 R2 with servicepack 2.

We also deployed secound vdp appliance but also this one has the same error and we can not use the FLR functions so looks like there is no easy solution on that?

best  regards

[GSparks]

Try the following:

1) Note the IP address of the VM where FLR is not connecting.

2) Open a web browser to go to the vCenter's Managed Object browser (https://<vCenter IP Address>/mob)

3) Click on the following hyperlinks :

3a ) content

3b ) SearchIndex

3c ) FindAllByIp

The last hyperlink should open another pop up window (SearchIndex : FindAllByIp)

In this new window, there are three fields - datacenter, ip and vmSearch.

4) Clear out the XML value in the datacenter field.  I always highlight the XML and hit the Delete key, as hitting the space key will not leave it blank, but instead with a space, and that is misinterpretted sometimes.

5) In the IP field, enter the IP address of the VM where FLR is not finding it.

6) in the vmSearch field, type the value 'true'

7) Click on the "Invoke Method" hyperlink

This should open a box beneath the hyperlink (Method Invocation Result) where the 'val' field will show the names of the VM's with this IP address.

If multiple VM's are returned, that is your problem.

If a single VM is returned, then click on the hyperlink to take you to the VM and continue with step 8.

😎 Click on the config hyperlink.

9) On the VirtualMachineConfigInfo page, find the 'instance uuid' and note it.

10) Open an SSH or Putty session to the VDP

11) Run the command 'mccli dump clientcache > cc.txt'

12) View the cc.txt file, and look for the name of the VM returned in step 7 above.

If you do not find the VM in the list, then the VM has not been backed up by the VDP, and this is why FLR will not connect.  Run a backup of the VM and once that is complete, try again.

A 'normal' entry for the VM will look something like :

/vCenter/VirtualMachines/Win2008R2-GS01_UCDsdCa3U3WW3iJ2YzHs0Q

  fc6a9b5da90ba9ec259cf9980cc87652c6504d66

  type: VMACHINE

  vc_cid: 83b37ced5f59d8b88d36784947f10ed7bd823395

  vm name: Win2008R2-GS01

  instanceUuid: 5020ec74-26b7-5375-96de-22766331ecd1

Compare the instanceUuid value from the above file to that noted in the Windows vCenter.

12) If the instanceUuid values are not the same, then try running the command "mcserver.sh --restart".  After this completes, try FLR again.

13) If the above does not resolve the issue, then it would make the most sense to open an SR with VMware support.

[MNCNE]

Hallo thanks for your answer,

we tried to do it like you described but the problem is when we click on the "FindAllByIP" there is no popup which will open and he is doing nothing. Java is installed on the vc.

We tested a little bit more and it is really crazy.

The result at the moment is from our view server (2008R2Sp1) we can login on the https://vdp:8543/flr/ the basic and the advanced login is working but just with this credentials "domain\administrator" when we try it with the local administrator the error message is : "Login Failed. The username supplied is not a administrator".

On the view server it also works with the advanced login that we see the restore points from the other backup machines and even the file restoring is working.

Now the strange thing. When we try the same on our Win7 client or on our SQL Server(Windows2008R2Sp1) we got the error message "Login Failed. cannot locate vm at <ip address>" and there it don't work with the basic login and not with the advanced login also it doesn't matter which credentials we take domain or local admin.

All server are on the same level of windows updates and patches but just on the view server it is working and even from the vc it self it is not working.

I can also ensure that we have not used double ip addresses for any servers because this test environment is real small.

We also tried it in the meantime with the vdp 5.5.6 and there is the same result like with the 5.1

best regards

[MNCNE]

Hallo,

today in the morning I tested again from an other client the procedure from GSPARKS.

Right now it works from the new client that the popup will open and I can do a "FindByIP" query for the view server which is working he will give me a reply with "vm-774" also for our vMA virtual machine.

When I use the command mccli dump clientcache > clients.txt it will show me the list of his client cache and there are all clients in also the virtual machines where the FLR is not working.

In the vdp I have restore point for all machines and I can even restore the machines where the FLR is not working.

Is there a command to delete and redetect the clientcache or something?

I also open an ticket for the problem at the VMware support if there will be an outcome from the VMware technician I will let you all know.

Gsparks maybe you have an idea what else I could check?

[GSparks]

Can you check the User Account Control settings on your Windows machines?  They may be blocking access.

I have had one occurrence when the UAC needed to be disabled for the FLR to work on Windows machines.

[Vdex_ie]

with regard to your issue, logging in to the FLR restore portal. I know you're certain you're using the correct credentials but it can be a little confusing. When it asks for the local credentials it means a local account on that virtual machine. It can't be a domain account or other type of account. It must be a local account on the virtual machine you're using to log in to the FLR site.

[MNCNE]

Thx GSparks for the Answer, the vmware support told us the same but it doesnt make any changes on that problem :-(.

Sorry that the answer took so long but yesterday we had a power outage and had problems with timesync.

Today I tested everything also delete the old restorepoints and created a new fresh backup after the UAC was switched off on all computers but the same result and the same error messages "Login failed. Could not find vm at <ip>" and also it is still in the https://vcenter/mob / the same that for the server with the problem I can not find ip entrys.

The UAC solved just the problem that we where able to install on the sql server the vmware vdp sql agent without error message.

But also when I try to make a backup via SQL server he says there are no registered servers, on the sql firewall is open in all direction.

We also restarted after UAC was switched off the hole server environment to be on the save side the setting was applied.

What else we can check?

best regards

[MNCNE]

Thanks for the answer Vdex_ie.

@Vdex_ie: we know that it should be a local account from the vm but this is even not working on the servers where we can access the restorepoints. From the servers which are working the only way for normal and advanced login is to use domain\administrator and than he loads the restorepoint from this machine or in the advanced login from all other machines.

And we really see a relationship between the guide which was offered from GSparks and the vmSearch because the machines which are working we can find via ipaddress and the not working machines we are not finding with the "Invoke Method".

Thats why we think it is related to the problems from the MOB maybe somehow it needs to be reinitialization?

[ToniCapablo]

Good morning,

I found a similar problem with newer versions:

ESXi version: VMware ESXi, 6.5.0, 5969303

vCSA version: VCSA 6.5.0.130000

vSAN version: 6.6.1

vSphere Data Protection version: 6.1.7

I did all the checks with vCSA MOB and finally I found that Instance UUID is different between the vCSA and VDP.

The next action to do as you say is do a last command in VDP (step 13) but first of all, I must consider what implies this action. What exactly do this command?

Regards,