Do SCSI reservations applied from a Linux virtual machine work on virtual disks? Are there tricks/tips on how they have to be configured?
Here's the background after doing research on how we think this is supposed to work ...
We are running two ESX 3.5 (plus some patches) servers with a FC connected Sun 3510 storage array between them. A 1.4 TByte RAID-5 LUN is presented to both servers as a VMFS volume. There are two RH4_U6 32 bit virtual machines, one on each ESX server, with their own private OS virtual disk (scsi0:0/sda) and two other virtual data disks (scsi1:0/sdb and scsi1:1/sdc). The data disks were created thick and are presented to both VM's as independent-persistent. The scsi1 controller is defined as lsilogic and sharedBus is set to physical (since the VM's are on separate servers). After the VM's kickstart I can do normal setup things (fdisk, vgcreate, mke2fs, etc). Then the application load sequence starts which includes installing and configuring Sybase ASE 12.5 pointing to normal disk files stored in an ext3 file system in an LVM on one of the two virtual data disks (one VM uses one virtual disk, the other VM uses the other virtual disk). As Sybase is initializing itself (running dataserver, I think), the kernel throws errors and the process hangs:
Journal commit I/O error
scsi1 (0,0,0): reservation conflict
SCSI error <1 0 0 0> return code 0x18
Buffer I/O error, lost page write
VM1 complains about the disk "assigned" to it (sdb, as shown above) and VM2 complains about the disk assigned to it (sdc, not shown but same messages except different SCSI target). They never complain about the "other" disk (the one the other server is working on and that they should not be paying any attention to).
Both drives need to be presented to both VM's for failover, although I don't know anything about how that's done or Sybase in general. The virtual machines need to be on separate servers, also for failover. I have almost no control over the application and how it's being loaded, although if there were some Sybase config file that needed tweaked that might be possible to sneak into the sequence. Changes to OS, ESX or .vmx files are completely under my control and can easily be handled.
I've run a number of tests with the following observations:
Failures happen very quickly if both VM's are roughly in sync, i.e. the Sybase initialization starts at close to the same time (with a couple of seconds).
I've seen failures when only one VM was running and the other was shut down, although the Sybase initialization usually runs quite a while in this case before failing.
Sometimes (not often) one of the two servers will complete successfully after the first has failed.
Putting both VM's on a single server works (although only tested once). I don't recall if I changed sharedBus to virtual for this case or not (probably not).
Putting both VM's on a single server and using local storage instead of SAN worked (although only tested once).
Putting the virtual disks on a SAN RAID-10 volume instead of the RAID-5 worked.
Recreating the entire environment on a completely different set of hardware (but exactly the same configuration) still fails, so I don't think this is a hardware problem.
Using raw LUN's presented via rdmp works. We used to run this way but requirements for these servers keep changing and we really, really, want the flexibility of virtual disks.
I saw a reference to "disk.locking = false" in one web search but it didn't help.
I'm not convinced there is really a difference between the RAID-10 and RAID-5 LUN's -- I think I may have just gotten lucky with the very few tests run on the RAID-10. I'm less certain about using the local storage.
I looked through the various VMware log files but didn't see anything particularly worrisome.
We have to make a decision on our storage layout very shortly (the next couple of days). I've submitted a support request to VMware but am hoping for help (again 
from the forum.
Would you think this setup should work (shared virtual disks supporting Sybase)?
Is there something else I need to do to allow SCSI reservations to work on a virtual disk?
Is this a Sybase specific thing and is there a workaround/fix?
.png "jhanekom"){kind=avatar}
Thank you for the reply.
As I said (buried in the original observations), we know raw LUN's work. That's how our storage used to be laid out. We just really hoped we could use virtual disks because our customers can't make up their mind on how many or what size the data areas need to be and virtual disks would allow a lot more flexibility (i.e. less hassle for me which is, of course, the most important goal ).
I assume by "cluster in a box" you mean all the VM's running on a single server? If so, I understand how that would make this work (and matches my test results) but we can't do that because the load has to be spread across physical servers for redundancy.
