Lefthand VSA Questions

[hutchingsp]

I'm hoping to speak to a reseller tomorrow but in the meantime I've been reading up on the Lefthand VSA.

On the face of it, I go out and buy a bunch of "something" and stuff them full of disks, drop them around our campus (we have fibre pretty much everywhere) and they form a redundant cluster automagically?

Or you could buy two, have one at each end of your site and have one replicate to the other?

Sounds too good to be true, what have I missed?

Also I'm unclear whether you can create LUNs greater than 2tb within Lefthand and assign them directly to a server that would mount it directly using the MS iSCSI initiator (I know the limit of VMFS/VMDKs is 2tb each, I'm unsure how the VSA/Lefthand handles this though).

[Josh26]

Hi,

Everything you have talked about is accurate. You certainly can put two in random places around and have a clustered storage environment. And yes, you can place one over a WAN and have it replicate.

In terms of "too good to be true", the "what have I missed" here is to think about expected write performance when, not only are you using iSCSI, you are using iSCSI to a SAN that mirrors its write to another iSCSI SAN before returning to the OS that a write has completed. Depending on your environment, availability may be worth it and awesome, or the performance may be crippling.

[amvmware]

The lefthand solution is a software SAN solution - this means you can have effectively - not that you would want to, utilise spare local disk storage capacity on your servers to create a virtual SAN. Lefthand does offer hardware solutions - now based on HP DL120 servers.

The software works by stripping the data across all the disks on the nodes - a node is a single DL120 device. To provide protection at the hardware level you can configture normal H/W raid - 0, 1, 5 - this protects you from a failure of a drive on a single node. You can then configure S/W RAID levels when you create your LUN's and this data is stripped across all the disks in the nodes.

The data is always copied to 2 separate nodes - so the failure of one node will not prevent data access - but could reduce performance as the number of drives providing IO to the LUN's has reduced by 50 % - or however many nodes you have. Hardware today is pretty reliable so this scenario would only become an issue if you had the nodes separated either on the same site or between sites and then the underlying LAN becomes a consideration.

If you are looking at having the nodes in 2 separate locations you will also need a third server or computer that will run a VM that acts as the majority node or witness - if this VM can contact one of the nodes then it it deemd to be the live node in the event of a LAN or WAN failure splitting the nodes.

By adding additional nodes you do gain in terms of additional memory and processing power - so it scales.

I should also clarify that you can buy a single node solution - it is a case of what levels of redundancy do you want or need for the data.

Hope this helps.

[hutchingsp]

I guess I'm coming back to this a bit late, sorry, didn't get notifed via email of the reply.

I guess where I'm confused is how the VSA differers from the physical Lefthand SAN product?

In our situation, right now we have a primary SAN/ESX infrastructure which is to be replaced in a year or so's time, so any money spent on it now is quite a waste.

We have a project coming up iminently that requires up to 5tb of storage capacity. Because our primary SAN is due to be replaced, I'm looking at options to let us put in 5tb of capacity now, that could either stand alone, or that could scale and form a part of whatever solution we look to replace our primary SAN with.

The Lefthand stuff looked appealing as it appeared I could go and buy a HP server, stuff it with disks, install ESXi, buy a VSA license and I have 10tb of fairly smart storage now (snapshots etc.), and in a year or so's time if we were to look at Lefthand for our primary SAN, it could be moved to form, perhaps, a DR node on the other end of our site.

What I wasn't clear on is how it scales, for example if we needed to grow beyond 10tb total available capacity could we "simply" add nodes and "network RAID" them so 2x10tb nodes is either 20tb with no failover, 10tb with failover etc.

[amvmware]

My understanding is the difference is VSA will take local storage and present it as a SAN. While the hardware version is simply preconfigured HP server hardware. - I am not sure you can purchase VSA on its own any more, but if you purchase the lefthand solutions, then you do get free VSA licenses to use as you indicated.

The way you grow the solution is adding additional nodes.This then means you can spread your created luns across more disks and you get the additional processing and cache to improve performance. This lefthand claims makes the solution more scalable than a standard SAN as you increase the performance as you add nodes.

On the capacity - you need to be careful 10TB will very quickly reduce when you add HW RAID 5 to provide data resilience and redundancy for each node and SW RAID 2 luns to strip the data across the multiple nodes and provide data redundancy in case of a node failure.

If i am honest the down side is customers see it as a bit of a "blackbox solution" and don't understand how it works and maintains performance when compared to standard SAN solutions.

[erickmiller]

Out of curiosity, how does the VSA handle failures? For instance, if you had 2 nodes replicated, and one fails, or better yet, the connection between the two fails, how are these scenarios handled? This differentiates a lot of products. The re-sync process in some products actually requires syncing every single byte of one end to the other end depending on which node was considered failed "first". This could be a pretty serious limitation if you have a WAN connection and need to transport 10TB of traffic to sync two (or more) nodes.

From what I understand, the VSA is much better than other solutions, but thought I'd ask.

Eric K. Miller, Genesis Hosting Solutions, LLC

- Lease part of our ESX cluster!

[hutchingsp]

I think I shall download the VSA ESX demo and install a couple of nodes on our existing ESX boxes and try it out - hopefully seeing/interacting with it will fill in a lot of the blanks I have in my mind.

I take the point about HW RAID i.e. the SmartArray will lose you capacity, and on top of that the VSA Network RAID will lose you capacity, but on the face of it, it would still appear that a DL1xx loaded with spindles running ESXi doesn't cost a lot of money.

I'm still trying to work out what I would "lose" by buying a couple and sticking one at each end of our site with 1-10gbps fibre linking them?

Actually the more I think about it, we have a big site and as Erick says, I could see the greater risk being that of a core switch or link(s) than an actual node going down.

Basically we're in a bit of a strange position in that we (maybe I) would like to do replication, failover and all the "enterprise" stuff but arguably the business doesn't actually need it, so it may be difficult to justify Compellent/3PAR/Netapp by the time you factor in all their licenses, and there also seems to be quite a high cost to get a base platform installed, whilst the VSA stuff looks quite linear if I understand how it works correctly.

Oh, if you go to the HP VSA page there is a product install video that looks to talk you through a lot of this stuff - just started to watch it as I type, may be worth checking out.

[amvmware]

If you have a geographically dispersed set of nodes - ie 2 server rooms on the same campus, the way it works is.

1. You have a third server \ pc running the management software as a VM - this acts as the majority node witness. So it is recommended it is not located in the same server rooms as the node, but must have IP connectivity to them.

2. In the event of a network failure that splits the campus - the majority node witness comes into play and the node it can contact is live and the other one is "dead" - The node that cannot contact the majority node witnes you can configure to power down. - This is the basis for lefthands distributed VMWare HA solution - the fact that the VM's have benn forceably shutdown on the second node means HA will apply on node one as it assumes there has been a server failure and so starts the VM's.

How it resyncs - i was told that until all the copies of the data on the luns are alligned then the second node would not come back on line, it only resyncs the changes - so it does it as a background task. If you had a lefthand solution and you needed to resync 10 TB of data - that is a lot of data and you would not be looking at 2 nodes but 6 or 8 nodes or more.

As i have said previous it does appear to do the job - but explaining this to customers when they are used to standard SAN technology can be difficult - this is its biggest draw back - it is perceived as bleeding edge by a lot of customers and we have lost lefthand opportunities because cusotmers are more comfortable with the standard SAN approach. Explaining why you need so many disks for a standard SAN is easier to explain than why you need fewer for a lefthand solution. Why you have SAN headers is easier to explain than why a standard server with SAS disks can achieve the same thing.

[hutchingsp]

Thanks again.

Still hazy on the Lefthand overhead - let's say after HW RAID I presented 5x2tb VMDKs to a single VSA node, how much could I use if it's my only node?

I've read about the "witness" and it makes sense in that it acts as a monitor, though I'm a bit hazy on the VMWare HA - we use that on our current (small i.e. 2 server) ESX cluster, but if we were dedicating boxes to be VSA nodes and they would NOT be running server VM's we would be able to run the free (ignore support) ESXi wouldn't we, and then we'd use HA/Vmotion on the ESX heads running our servers?

From speaking to one vendor we deal with, I'm not sure many vendors are able to wrap their heads around the VSA - we were told "We wouldn't recommend it" but I didn't get much in the way of hard technical reason as to why, sort of got the impression it was a bit out of their comfort zone vs. a dedicated box from HP

[amvmware]

Your available storage capacity depends on which SW and HW RAID levels you choose, Lefthand recommend HW RAID5 and SW RAID 2.

To be honest your experience with resellers is not surprising - we only found the answer to alot of this as customers raised their issues and scenarios. It is getting pushed heavily by HP as they see it as a storage offering that falls between a MSA and an EVA.

I think HP do a try before you buy with VSA - so speak to a local lefthand reseller and see if this can be arranged for you.

[hutchingsp]

Thanks, just watched the full Lefthand video on the HP site and it makes a lot more sense now.

One thing that leaps to mind is if you put your VSA nodes onto a dedicated storage network, how would you manage the Lefthand SAN from a typical workstation - my PC for example wouldn't be multi-homed to the production LAN and the storage LAN so the only way I can see would be from a VM running on one of the ESX boxes using the Lefthand SAN for storage, since it would be multi-homed?

[amvmware]

My experience is with the appliances - all i know about the VSA is you can use spare local storage capacity on a ESX server to create your virtual SAN and spare storage capacity from anywhere lese if you want to - you just need sufficient VSA licenses.

Don't forget to leave points for helpful/correct posts.

[hutchingsp]

Downloaded and installed the VSA for ESX demo today. Quick and dirty, one VSA each on a couple of our existing ESX servers, along with one FOM, all just using our regular LAN and on a client the standard Microsoft iSCSI initiator.

I'm rather impressed so far - I have storage that's redundant as proven by the fact that if I do a continuous ping of the cluster virtual IP, and then power off the gateway VSA node, I get a couple of seconds break in connectivity and Windows stutters for a few seconds about accessing the LUN, then the FOM kicks in and moves the connection to the remaining VSA and it all carries on, and when I power the regular node back on it resyncs all the data.

So, I've not though too specifically here, but in principle I think I could:

Drop in a couple of VSA nodes on dedicated hardware, one at each end of our physical site, connected to a dedicated iSCSI network.

Have several ESX servers spread across both locations, using the VSA cluster for their shared storage.

If I lose a site or a VSA, I have storage level redundancy, and either manually or using HA I have ESX level failover.

What have I missed?

One thing I'm certainly not clear on is performance - it seems I could stuff a dedicated server with fast disks in RAID10 and use a decent RAID controller with BBWC, but presumably my bottleneck is that all the iSCSI connections being made from my servers (ESX or direct from in Windows) are being split across "only" 1gbps x Total VSAs in Cluster" - could I have a heavy use file server (virtual or physical) use multiple connections to get extra throughput?

Thanks.

[mreferre]

Sorry, late to the party.

I wrote a blog entry about this a year ago... http://it20.info/blogs/main/archive/2008/05/19/121.aspx

How does this differ from the standard product? I would say this is for minimal scenarios where you only have/need a couple of ESX servers and the burden of buying a couple of storage servers would be too much. I would avoid using something like this for a more "complex / scaled" scenario.

Massimo.