lpkirby
Contributor
Contributor

Reservation Conflict on RHEL 6 with Oracle RAC

I've been searching for any hints I can find out my problem, but nothing has worked so I'm trying here.  I'm seeing this message in /var/log/messages:

Jul 13 10:45:59 oratest1 kernel: sd 3:0:1:0: reservation conflict
Jul 13 10:45:59 oratest1 kernel: sd 3:0:1:0: [sdc] Unhandled error code
Jul 13 10:45:59 oratest1 kernel: sd 3:0:1:0: [sdc] Result: hostbyte=DID_OK driverbyte=DRIVER_OK
Jul 13 10:45:59 oratest1 kernel: sd 3:0:1:0: [sdc] CDB: Write(10): 2a 00 00 08 00 11 00 00 01 00

I believe this is what is causing errors like this to show up in my Oracle alert logs:

<msg time='2012-07-13T14:09:01.596-05:00' org_id='oracle' comp_id='rdbms'

client_id='' type='UNKNOWN' level='16'

host_id='oratest1.harding.edu' host_addr='10.10.2.195' module=''

pid='5205'>

<txt>Errors in file /u01/app/oracle/diag/rdbms/awdv/awdv1/trace/awdv1_ckpt_5205.trc:

ORA-00206: error in writing (block 3, # blocks 1) of control file

ORA-00202: control file: &apos;+DATA/awdv/controlfile/current.283.787679375&apos;

ORA-15078: ASM diskgroup was forcibly dismounted

ORA-15078: ASM diskgroup was forcibly dismounted

</txt>

</msg>

This is the first time I've tried to set up Oracle RAC in VMware so I probably missed something very small, but I cannot for the life of me find what it is.

My setup is:

RHEL 6.3 (upgraded from a fresh 6.2 install)

vSphere 4.1

Oracle RAC 11.2.0.3

Latest VMware  Tools

I'm using vmdk files for all my disk groups.  The cluster starts just fine and I can connect to the databases, but they rarely stay up more than a few hours with these errors.

I'm not using fence_scsi or any of the sg3_utils stuff, etc...

I've got disk.locking="false" in my vmx file and all the multi-writer flags for the scsi drives used by ASM.

There are no errors in the vmware.log file.

I feel like I just set something up wrong and maybe VMware and the oracle cluster are fighting each other with locks?

Has anyone seen anything like this or have any suggestions where to look next?

0 Kudos
7 Replies
jhannan
Contributor
Contributor

Why are you using the flag disk.locking="false"? I am wondering if this is causing you issues. You should be able to setup the ASM drives with just the multi-writer flag enabled.

Jim Hannan | House of Brick | Solutions Architect
0 Kudos
lpkirby
Contributor
Contributor

I added it just a few minutes ago to see if it would help.  I didn't have it originally, but several other people have listed it as part of the vmx file so I decided to try it.  I'm still getting the errors and it didn't seem to change anything.

For example, it's listed in this thread:

http://communities.vmware.com/thread/398405?tstart=0

0 Kudos
wbassler
Contributor
Contributor

I had the same issue that I believe was just resolved by adding the multi-writer flag for each shared disk. Please also reference this link as well to enable SCSI reservation: http://stuff.mit.edu/afs/sipb/project/vmdialup/lib/vmware-console/help-manual/ha_configs_scsi-res_en... . In addition are your vms on the same host? I migrated the vms to the same host and it helped resolve prior shared disk issues.

0 Kudos
lpkirby
Contributor
Contributor

Thanks for your response.  I never did get this resolved.  At the time we were using ESXi 4.x and are now on ESXi 5.1.  I don't know if that would help, but I ended up using a couple physical blades as being in VMware didn't buy me what I thought it would.

0 Kudos
normpang
Contributor
Contributor

I am away from the College until Monday 22nd April 2013. I won’t have access to e-mail but I will respond on my return. If you matter is urgent please call the helpdesk on 8888 or contact Platform Services (is-platform-services@qmul.ac.uk).

thanks

Norm

0 Kudos
wbassler
Contributor
Contributor

I also had fits for weeks on this. I first moved the vms to same host which seemed to help communication issues I was having. I then added the scsiX:Y = "multi-writer" flag to each of my shared disks. I started to get reservation conflicts. I then added scsiX.sharedBus = "virtual" and disk.locking = "false" flags and this resolved the reservation conflict. I would be happy to shed some more light on this if you would like. My DBA says that everything is looking good and we havent had shared disk issues in a week so I we are good to go.

0 Kudos
Dynamidis
Contributor
Contributor

Worked like a charm. Thank you man... You saved us from disaster...

0 Kudos