cef2lion
Contributor
Contributor

SRM vSphere issue

SRM installed in vSphere enviroment. We have two Dell AX/4 SANs. The protected site has 5 luns all with VMs. Dell setup the mirror of all 5 luns. They are in sync. We installed SRM on the protected and recovery sides. We then installed the x86 EMC solution enabler and the EMC SRA.

SRM and the EMC Insight tool are only seeing the third lun of the 5 luns. We setup consistency groups on the san for the luns. SRM and Insight now see the 5th lun and no others.

I have a case open with EMC. They only directed me to VMware. VMware opened a case and did a WebX and saw nothing and suggested going back to the SAN vendor. We have a local VMware tech account manger come out and look. He could not find anything wrong. We spoke with a local Dell tech account manager who is working with the VMware tech account manager. They are planning on visting next week but have nothing to go on. VMware and EMC keep contacting about wanting to close the cases.

I saw a post on the VMware forums last summer that had the same issue. The person never posted back that it was solved. I sent a PM but user has not been on since late in 2009. It was titled "SRM sees only one replicated LUN / EMC AX4-5FC" and posted on March 31, 2009.

Craig

0 Kudos
23 Replies
JeffDrury
Hot Shot
Hot Shot

Craig,

A few quick things:

1. Do the replicated datastores have installed VM's present? If there are no VMDK files or if the OS is not installed on the VM then SRM will not recognize the datastore. If you only create the VM's in the VIC and do not install the OS then SRM will not recognize the datastore.

2. Is recover point set to allow the CG's to be managed by SRM? This wouldn't prevent the datastores from being displayed but would prevent the mounting of the datastores when a test is run.

3. Have you stopped and started the SRM service?

These are shots in the dark but I thought I would give it a try.

0 Kudos
cef2lion
Contributor
Contributor

Thanks for the suggestions.

1. Yes the datastores all have VMs repsetn.

2. Will check on this.

3. I have rebooted SRM a number of time. Uninstalled and reinstalled as well.

Craig

0 Kudos
TimOudin
Hot Shot
Hot Shot

If I may, are you not seeing the datastore during a test or not seeing the LUN at all from the ESX host? After storage preparation phase does `esxcfg-volume -l` give you any information about exposed snapshot LUN? Could this possibly be an issue with snapshot access since this is configured on a per LUN basis in vSphere?

I suppose VMware support would have already addressed this, just more shots in the dark...



Tim Oudin

Tim Oudin
0 Kudos
depping
Leadership
Leadership

Do you mean during the configuration of the SRA? Can you post the SRM logfiles or snippets? SRM first does a discover arrays and then a discover luns. discover luns should return a list with devices...

Duncan

VMware Communities User Moderator | VCP | VCDX

-


Now available: Paper - vSphere 4.0 Quick Start Guide (via amazon.com) | PDF (via lulu.com)

Blogging: | Twitter:

0 Kudos
cef2lion
Contributor
Contributor

I got a PM from the thread from last summer with same issue.

No, i did not solve it. It was only a test in lab.

Please let me know if you can solve this issue.

Good luck.

Where are the SRM logs? I don't think VMware ever looked them.

On the SAN Dell setup the 5 remote mirrors. Each of which have working VM's. The mirrors according to Navi Manger are shown to be in sync. Those mirrors on the recovery site are presented to the ESX hosts in that site. Under storage adapter I can see the mirrors. Under storage if I say add new storage they are visable but if you would try and add them it sees nothing and wants to format as a new disk. I don't actaully do that. I think that is how the recovery side should see them without SRM in action on the recovery site.

When configuring SRM with the VI plug-in. I can pair the sites ok. When I try and configure the array managers it sees nothing. I do a scan in the wizard after entering info about the protected and recovery array managers. It does see the EMC SRA. On the last screen I select rescan a number of times and nothing.

What is odd. If I go back to the Navi Manager and remove the consistency groups things change a bit. The SRM plug-in under array manager sees one device now instead off none. Seems pulling the consistency group changed something. The 1.4 release notes for the EMC SRA says later versions of the SRA would not require a consistency group. I added them anyway.

Craig

0 Kudos
bladeraptor
VMware Employee
VMware Employee

Hi

I am writing this as an EMC Employee

Can you please detail your configuration - versions of CLARiiON Flare / VMware ESX / Solutions Enabler / the SRA and how the MirrorView is set up including the creation of Snaps

Have you tried manually failing over one of the MirrorView volumes by promoting the DR volume and bringing it into the Storage screen (promoting the DR mirror will strip the 'read only' flags off and should make the VMFS volume accessible to the DR ESX hosts and allow you to register any Virtual Machines resident there)

Clearly if there are any VMs on this VMFS voulmes on the productin side these will need to be shut down and unregistered (as the Production on the VMFS volume on promoting back will probably get resignatured)

Enclosed is a guide written by Cormac Hogan

http://communities.vmware.com/docs/DOC-11544

Please review

Regards

Alex Tanner

0 Kudos
cef2lion
Contributor
Contributor

Both SANs are AX4/5

Clariion Flare code: 2.23.050.5.705

VMware ESX: 4.0 update 1

Vcenter: 4.0 update 1

Solution Enabler: 7.1.0

SRA: 1.4.0.15

On the solution enabler. I have it installed prior to SRA. Do you need a license code? Dell seemed to think that got taken care of with the Navi Express to Manager upgrade.

I have not tried to manually fail over. VMware technical account manager is coming back out next week. He visted this week but I was not present. He didn't review the SRM logs.

I do have the guide by Hogan. Been over it a number times.

As stated Dell did the mirror setup as part of the project we had them in for. They updated all code on SANs and the fiber channel switches. They setup the fiber channel zones as well. They setup some luns for snapshots but didn't setup the snapshots. There project was stated to mirror the luns to the B side and we setup the snapshots to present them the the B side ESX hosts for backup. My thought was they were doing to mirror to prepare us for SRM. I had to step out for a bit while the tech was here. I'm a bit fuzzy on consistency groups and SnapView/Snapshots. I know SRM requires special naming for snapshots. Are snapshots a one time capture or to they occur on a schedule once setup?

Craig

0 Kudos
bladeraptor
VMware Employee
VMware Employee

Hi

Can you please confirm that something similar to the steps below have been completed?

1. Install SRM and the EMC Clariion adapter on both sides - I take it that you have installed Solutions Enabler on both sides as well?

2. Set up a LUN (PR)on the primary CX-4 - and present this LUN PR to the production side ESX hosts, scann it into the cluster, build a VMFS partition on it and locate a VM Guest in the resulting VMFS partition

3. Set up a Secondary Image LUN (DR) on the secondary CX-4

4. Configured a Remote Mirror between these two LUNs (DR + PR) across the primary and secondary CX-4s

5. (Optional) configure a Consistency Group with the DR and PR LUNs - this is not essential with the SRA version EMC_MirrorView_Site_Recovery_Adapter-1.3.0.4 and above

6. Add the LUN DR from step 3 to the Storage Group for the ESX servers on the secondary CX-4

7. Creat a Snapshot of the step 3 LUN DR, making sure to includes VMWARE_SRM_SNAP in the name of the snapshot - ideally you need to create a snapshot on the primary side LUN PR as well (using the VMWARE_SRM_SNAP as part of the snap name) and add this to the primary site ESX hosts. You do not need to build any VMFS partitions on it - but it needs to be seen by the hosts at the HBA level

8. Add the Snapshot to the same Storage Group for the ESX servers on the secondary CX-4 as was done from step 6

9. As has already been suggested on the DR side you just need to add the remote target LUN and snapshot to the remote ESX host so they are seen by the HBAs (the storage rescan stage) - you do not need to build any VMFS paritions on these volumes (this will prove difficult as these elements are read-only - till a test or failover are initiated)

Regards

Alex Tanner

0 Kudos
cef2lion
Contributor
Contributor

Going over the 9 steps outlined to see if something was missed.

One issue that Dell is looking into. We did load the Solution Enabler from EMC however I was reading in the install guide that a post task is to install the license. We never got a license for solutions enabler. Dell did the quote on our configuration. We did license SRM itself through our school. Dell thought the license was in the EMC Navi Express to Manger upgrade. I'm thinking not as that upgrade is for the SAN only. The enabler is loaded on the SRM server and licensed there. Could this be our issue?

Craig

0 Kudos
bladeraptor
VMware Employee
VMware Employee

Hi

Solutions Enabler is a red herring IMHO. The license is a RTU and only called by the Solutions Enabler functionality for the server function with Storage Viewer

In terms of enablers - the AX need to be running full Navisphere and have both SnapView and MirrorView enablers installed

The types of errors you are getting point to the AX / ESX set up or the MV / ESX configuration setup

I would create a mirror for a single volume (LUN) - make sure the production ESX hosts can see it, can format it, can put an VMFS partition on the volume and then put a Virtual Machine in that data store

Create the mirror and present the LUN on the DR side to those ESX hosts at a physical (adpater level)

Promote the mirror on the DR side and bring in the volume via the storage wizard and make sure the VMFS datastore is accessible to the ESX hosts and that the Virtual Machine can be registered and powered on in the DR ESX environment

Can you please upload the SRM logs from both SRM servers to this forum so we can look at them?

The logs can be found at Documents and Settings\All Users\Application Data\VMware\VMware vCenter Site Recovery Manager\Logs or equivalent

Regards

Alex Tanner

0 Kudos
cef2lion
Contributor
Contributor

Alex

Going to run through the test you suggest.

Both AXs are running full Navi. We have MirrorView enablers on both AXs. We have SnapView just on the Recovery AX.

As side note on storage groups. On the Protected side. I have 4 storage groups. One for each of our 4 ESX hosts. In each storage group I add all 5 luns as all 4 ESX hosts are in a VMware cluster. When I was setting up the recover side I created just one storage group and put both ESX hosts on the recovery side in that storage group.

Craig

0 Kudos
bladeraptor
VMware Employee
VMware Employee

Hi

Is there a particular reason for why you are using multiple storage groups on the Production side - the only time that makes sense really these days is when you are ESX booting from SAN and for isolated examples such as UCS this makes sense (given the way they have dynamic profiles) - for the majority of customers one Storage group for all hosts in a cluster is enough

The difference with LUN IDs and synching all those LUNs across the Storage groups can create room for a lot of errors

If you can get snap view on the Production Side I would suggest it as without it you can not test and model failback

Can you please upload the SRM logs if possible?

Regards

Alex Tanner

0 Kudos
cef2lion
Contributor
Contributor

Having multiple storage groups on the production side goes back to the Navi Express interface. I think the interface had me thinking that was the way to go. I plan to change that. I can move VMs off one host at a time and collapse that storage group until I get to one.

I can upload the SRM logs if you tell me where they are at and the file name.

Craig

0 Kudos
bladeraptor
VMware Employee
VMware Employee

Hi Craig

Please see enclosed image on the SRM servers running SRM 4 please follow the path - there is a script to generate the logs that will put them on your desktop as Zip files

Regards

Alex Tanner

0 Kudos
bladeraptor
VMware Employee
VMware Employee

Hi Craig

Please see the enclosed image for the program files route to generate te logs

Regards

Alex Tanner

0 Kudos
bladeraptor
VMware Employee
VMware Employee

Hi Craig

Please see enclosed image for a steer on generating an SRM log bundle

Has a case been raised with VMware / Dell

Regards

Alex Tanner

0 Kudos
cef2lion
Contributor
Contributor

I have attached the recover site SRM logs. Or tried to. It would not let me upload the logs. Is there a size limit?

I first opened a case with EMC. They did nothing other then refer me to VMware. Opened a case there. They did a quick WebX session looked a few things over and referred me Dell who did the mirror work. I then spoke with local Dell and VMware people who are engaged on it at the moment. The VMware local technical account manager visted and didn't see anything. He plans to visit again. The tech on the VMware and EMC case has since asked to close the case since VMware in site. The cases have never been raised but they sure want to close them. It could very well be a config issue on my end but I'm at a lose in identifying it.

Craig

0 Kudos
bladeraptor
VMware Employee
VMware Employee

Hi Craig

Can you upload them to ftp.emc.com/incoming/alexT

username: anonymous

password:

Regards

Alex Tanner

0 Kudos
cef2lion
Contributor
Contributor

I uploaded the logs. If you want the protected side logs let me know and will do the same. Started collapsing the storage groups on the protected side down to one.

Craig

0 Kudos