Virtualization

I've been searching for any hints I can find out my problem, but nothing has worked so I'm trying here.  I'm seeing this message in /var/log/messages:

Jul 13 10:45:59 oratest1 kernel: sd 3:0:1:0: reservation conflict

I believe this is what is causing errors like this to show up in my Oracle alert logs:

<msg time='2012-07-13T14:09:01.596-05:00' org_id='oracle' comp_id='rdbms'

client_id='' type='UNKNOWN' level='16'

host_id='oratest1.harding.edu' host_addr='10.10.2.195' module=''

pid='5205'>

<txt>Errors in file /u01/app/oracle/diag/rdbms/awdv/awdv1/trace/awdv1_ckpt_5205.trc:

ORA-00206: error in writing (block 3, # blocks 1) of control file

ORA-00202: control file: &apos;+DATA/awdv/controlfile/current.283.787679375&apos;

ORA-15078: ASM diskgroup was forcibly dismounted

ORA-15078: ASM diskgroup was forcibly dismounted

</txt>

</msg>

This is the first time I've tried to set up Oracle RAC in VMware so I probably missed something very small, but I cannot for the life of me find what it is.

My setup is:

RHEL 6.3 (upgraded from a fresh 6.2 install)

vSphere 4.1

Oracle RAC 11.2.0.3

Latest VMware  Tools

I'm using vmdk files for all my disk groups.  The cluster starts just fine and I can connect to the databases, but they rarely stay up more than a few hours with these errors.

I'm not using fence_scsi or any of the sg3_utils stuff, etc...

I've got disk.locking="false" in my vmx file and all the multi-writer flags for the scsi drives used by ASM.

There are no errors in the vmware.log file.

I feel like I just set something up wrong and maybe VMware and the oracle cluster are fighting each other with locks?

Has anyone seen anything like this or have any suggestions where to look next?

Why are you using the flag disk.locking="false"? I am wondering if this is causing you issues. You should be able to setup the ASM drives with just the multi-writer flag enabled.

I added it just a few minutes ago to see if it would help.  I didn't have it originally, but several other people have listed it as part of the vmx file so I decided to try it.  I'm still getting the errors and it didn't seem to change anything.

For example, it's listed in this thread:

http://communities.vmware.com/thread/398405?tstart=0

I had the same issue that I believe was just resolved by adding the multi-writer flag for each shared disk. Please also reference this link as well to enable SCSI reservation: http://stuff.mit.edu/afs/sipb/project/vmdialup/lib/vmware-console/help-manual/ha_configs_scsi-res_enable_gsx.htm . In addition are your vms on the same host? I migrated the vms to the same host and it helped resolve prior shared disk issues.

Thanks for your response.  I never did get this resolved.  At the time we were using ESXi 4.x and are now on ESXi 5.1.  I don't know if that would help, but I ended up using a couple physical blades as being in VMware didn't buy me what I thought it would.

I also had fits for weeks on this. I first moved the vms to same host which seemed to help communication issues I was having. I then added the scsiX:Y = "multi-writer" flag to each of my shared disks. I started to get reservation conflicts. I then added scsiX.sharedBus = "virtual" and disk.locking = "false" flags and this resolved the reservation conflict. I would be happy to shed some more light on this if you would like. My DBA says that everything is looking good and we havent had shared disk issues in a week so I we are good to go.

Worked like a charm. Thank you man... You saved us from disaster...

I am away from the College until Monday 22nd April 2013. I won’t have access to e-mail but I will respond on my return. If you matter is urgent please call the helpdesk on 8888 or contact Platform Services (is-platform-services@qmul.ac.uk).

thanks

Norm