Xtravirt Virtual SAN Appliance

[admin]

Hi Guys,

Over at Xtrairt we've just released a new free virtual appliance that I thought people might find interesting. It lets you use the local storage on two ESX hosts to create a virtual SAN. I've attached an overview diagram to explain how it works.

http://www.xtravirt.com/index.php?option=com_remository&Itemid=75&func=fileinfo&id=29

Summary: The Xtravirt Virtual SAN (XVS) appliance for VMware ESX3 Server is a free solution to provide the benefits of shared VMFS storage without the cost of a SAN – this allows the utilisation of otherwise unused local storage in the ESX server to facilitate enterprise level features such as vMotion, DRS and HA normally only available through the use of a shared storage device. All volume data is synchronously replicated between hosts, providing full fail-over capability with data integrity in the event of host, disk or appliance failure. The appliance is menu driven and has been designed to be as easy to configure as possible, and full documentation on the implemenation process is provided.

Questions/feedback welcome in this thread.

Cheers,

Alex

[azn2kew]

Wow, I can't wait to download and test it out. I've been using OpenFiler and FreeNAS for my alternatives but performance still in consideration. So how is performance looks like comparing to XVS and other commercial tools such as mentioned or SANMelody. This sounds like SANmelody and Lefthand's Network VSA but I could be wrong. I'll get some feedback after this weekend testings!

If you found this information useful, please consider awarding points for "Correct" or "Helpful". Thanks!!!

Regards,

Stefan Nguyen

iGeek Systems Inc.

VMware, Citrix, Microsoft Consultant

[azn2kew]

Quick questions, I've just read the manual and seems very straight forward to configure and 90% are similiar with OpenFiler process. I'm curious if you going to have web base management console like OpenFiler to manage volumes and LUNs etc... Does it support NFS or strictly iSCSI? If I need to add additional disk to XVS machine, than I will go through same process to get the LUN show up or it should automatically show up when I rescan the LUNs?

I like the LUN failover heartbeat feature, but want to know more details how it works and wondering any options or settings to decide which one is Active/Passive or not. What happen if we have multiple disks with different size for XVS appliance server?

XVS01:

disk01: XVS OS

disk02: 50GB LUN size disk creation

disk03: 100GB LUN size disk creation

disk04: 60GB LUN size disk creation

XVS02:

disk01: XVS OS

disk02: 40GB LUN size disk creation

disk03: 200GB LUN size disk creation

disk04: 80GB LUN size disk creation

How does it failover if it has different disk/LUN size?

If you found this information useful, please consider awarding points for "Correct" or "Helpful". Thanks!!!

Regards,

Stefan Nguyen

iGeek Systems Inc.

VMware, Citrix, Microsoft Consultant

[admin]

Quick questions, I've just read the manual and seems very straight forward to configure and 90% are similiar with OpenFiler process. I'm curious if you going to have web base management console like OpenFiler to manage volumes and LUNs etc...

Possibly at a later date but not in the immediate future.

>Does it support NFS or strictly iSCSI?

iSCSI only.

>If I need to add additional disk to XVS machine, than I will go through same process to get the LUN show up or it should automatically show up when I rescan the LUNs?

Currently each pair of XVS nodes supports only one LUN, therefore if you want multiple LUNs you'll need multiple XVS appliances per host.

I like the LUN failover heartbeat feature, but want to know more details how it works and wondering any options or settings to decide which one is Active/Passive or not.

The XVS is active/active, all reads come from the local disks, all writes go to both sides.

>What happen if we have multiple disks with different size for XVS appliance server?

XVS01:

disk01: XVS OS

disk02: 50GB LUN size disk creation

disk03: 100GB LUN size disk creation

disk04: 60GB LUN size disk creation

XVS02:

disk01: XVS OS

disk02: 40GB LUN size disk creation

disk03: 200GB LUN size disk creation

disk04: 80GB LUN size disk creation

How does it failover if it has different disk/LUN size?

The disk sizes on each XVS node must be the same size.

Hope this helps.

Cheers,

Alex

[admin]

Update: there sems to be a problem with the download link to the documentation, if you experience problems please use the link below to download the docs.

http://www.xtravirt.com/index.php?option=com_remository&Itemid=75&func=startdown&id=30

Thanks,

Alex

[mreferre]

Alex,

this might be a good scenario for your virtual appliance (?)

Massimo.

[admin]

Yep, that's exactly the sort of thing it's aimed at.

[stormlight]

Thank you, this is great.

Few questions:

Im trying to wrap my head around networking and what not in general with esx and this.

Shoud I add a Virtual Machine Port Group to my iscsi SAN Vswitch and run this virtual applience on that network?

Then point my esx host to the virtual ip that the XVS uses? If thats done when the two replicate back and fouth its using the nics that are binded to my SAN network correct?

If my physical SAN network is getting saturated is their any issues with running this in the vmotion network security wise? No one can get to our vmotion network unless they are on the esx host anyway. So the only contact would be the esx host software iscsi inatior passing data to this virtual san when it replicates.

thanks again

[SimonG-ioko]

I've got this setup on my home lab, vMotion etc. work ok (even over a 100mb/half hub )

I can't get my head around the failover though..

my setup

ESX 1 -> pointing at iSCSI IP on 1st SAN node running on itself

ESX 2 -> pointing at iSCSI IP on 2nd SAN node running on itself

if I reboot 2nd SAN node

does its iSCSI IP transfer to the 1st SAN as well to continue access or will ESX 2 loose access to its iSCSI LUN? - I can't really see whats going on from the console..

and secondly, when the 2nd SAN node comes back up and I start services it doesn't seem to do the sync and restart the iSCSI services it comes up 'outdated' - am I doing something wrong?

And I think you may need to mention in your docs that you need to open the firewall port for the ESX iSCSI client for to see the LUNs (or at least I did).

[stormlight]

cant answer all your questions.. i got mine above. However try a cross over cable between the two nics that this sits in. Since its only two node you can get much faster speeds then 100 mb half

also you may want to listen to the vmware virtulization podcast. The author of this is on it and talks a little about your question

[admin]

Thank you, this is great.

Few questions:

Im trying to wrap my head around networking and what not in general with esx and this.

Shoud I add a Virtual Machine Port Group to my iscsi SAN Vswitch and run this virtual applience on that network?

You can, or you can run it on your normal VM vSwitch - it's up to you, it depends how many NICs you have in your ESX host, if you can dedicate one to the iSCSI network great, if not then it should work anyway but GB is strongly recommended. The only requirement is that the two XVS nodes can talk to eachother.

Then point my esx host to the virtual ip that the XVS uses? If thats done when the two replicate back and fouth its using the nics that are binded to my SAN network correct?

Yes, you need an S/C and VMKernel PG with an IP in the same subnet as the XVS appliances so that they can talk to them both. So if you were using a dedicate vSwitch with a NIC bound to it, you'd need an S/C and VMKernel PG on that vSwitch.

If my physical SAN network is getting saturated is their any issues with running this in the vmotion network security wise? No one can get to our vmotion network unless they are on the esx host anyway. So the only contact would be the esx host software iscsi inatior passing data to this virtual san when it replicates.

No issue with this really, the only problem could be contention if you were writing a lot of data to the XVS at the same time as performning a vMotion. I think in reality it'll work fine.

Hope this helps.

Cheers,

Alex

[admin]

I've got this setup on my home lab, vMotion etc. work ok (even over a 100mb/half hub )

Heh I bet performance will be, erm, interesting.

I can't get my head around the failover though..

my setup

ESX 1 -> pointing at iSCSI IP on 1st SAN node running on itself

ESX 2 -> pointing at iSCSI IP on 2nd SAN node running on itself

if I reboot 2nd SAN node

does its iSCSI IP transfer to the 1st SAN as well to continue access or will ESX 2 loose access to its iSCSI LUN? - I can't really see whats going on from the console..

Yes, the 1st XVS node will assume the Virtual IP of the 2nd SAN node and ESX 2 will start recieving all of it's iSCSI data from the 2st XVS node until the 2nd XVS node is brought back online and is back in sync.

and secondly, when the 2nd SAN node comes back up and I start services it doesn't seem to do the sync and restart the iSCSI services it comes up 'outdated' - am I doing something wrong?

It should auto resync, sometimes if no data has changed the sync is so fast you don't get to see the status. If the iSCSI service started, it'll be in sync, as it won't start the service until both sides are in sync. You can check this from the troubleshooting console with "cat /proc/drbd". PM me if you need more information.

And I think you may need to mention in your docs that you need to open the firewall port for the ESX iSCSI client for to see the LUNs (or at least I did).

Thanks, that's a fair comment, I'll get the guide updated.

Cheers,

Alex

[SimonG-ioko]

Thanks Alex,

This appliance is a great idea... I've got a couple of posts on my blog (http://vinf.net) on how my setup works, once I nail this failover thing in my lab setup I'll post it up..

A couple of other feedback items that might be useful;

Would be good to have some kind of simple status info when it goes back to the main menu screen, even if it's just something basic like

iSCSI status: started/not started

iSCSI address(es): 192.168.x.1, 192.168.x.2 etc.

Failover/Heartbeat status: primary, lost connection etc.

Writes/connections/sec etc.

Great work!

Now I just need to find a Gig switch somewhere... hmmmm

[SimonG-ioko]

cant answer all your questions.. i got mine above. However try a cross over cable between the two nics that this sits in. Since its only two node you can get much faster speeds then 100 mb half

Thats a good idea; hadn't thought of that - I've got 2 x HP D530 PCs doing my ESX stuff - they have GigE onboard I could prob run my iSCSI/VMKernel over a crossover on that, I have some spare 100Mb NICs that I could use to run my external VM networks into the external hub until I can find a switch.

Thanks!

[BrianRTS]

Looks nice. Are there any plans to support more nodes so more servers can utilize the SAN storage?

[admin]

Thanks Alex,

This appliance is a great idea... I've got a couple of posts on my blog (http://vinf.net) on how my setup works, once I nail this failover thing in my lab setup I'll post it up..

Cool, thanks, and glad you like it.

A couple of other feedback items that might be useful;

Would be good to have some kind of simple status info when it goes back to the main menu screen, even if it's just something basic like

iSCSI status: started/not started

iSCSI address(es): 192.168.x.1, 192.168.x.2 etc.

Failover/Heartbeat status: primary, lost connection etc.

Writes/connections/sec etc.

Yep that's a good idea, I'll work to include that in the next release, thanks for the input.

Cheers,

Alex

[admin]

Looks nice. Are there any plans to support more nodes so more servers can utilize the SAN storage?

Not in the short term, doing so requires writing a bespoke network RAID solution, a lot of work!

You can however run as many pairs of XVS appliances as you like, and there is nothing to stop you from pointing a third ESX host at a pair of XVS appliances on two other ESX hosts.

(tried to draw it in ASCII but the forum mangels it )

[stormlight]

if you where to point a 3rd ESX host to the VSAN what ip would you point it to? The primary or the # 1 vsan?

so what happens when #1 dies #2 picks up Vsan #1s ip? And all is good

[admin]

if you where to point a 3rd ESX host to the VSAN what ip would you point it to? The primary or the # 1 vsan?

You could point it at XVS node 1 or node 2 iSCSI target IP (the virtual IP) it doesn't really matter, if you introduced a 4th ESX host, you would want to point it at the other node from ESX3 to even the load, although it is not critical.

so what happens when #1 dies #2 picks up Vsan #1s ip? And all is good

Correct, if XVS node 1 dies XVS node2 takes over it's IP and all storage traffic continues to flow via node 2, when XVS node 1 is brought back online it's IP address fails back over after it is brought back into sync and processing is then handled by both nodes again.

[rocker77]

What about commercial usage. I think, that this is ideal solution for small bussines - two ESX servers with local storage, ESX standard + VC Foundation. HA solution for great price.

Do you offer commercial support for XVS or do you suggest this appliance only for testing purposes?